Google Colab for ML/AI and Jupyter Notebook based Computational Research
|
The Center for Research Computing is empowering ND researchers with enhanced capabilities for Machine Learning (ML) and Artificial Intelligence (AI) through Google Colab notebooks. Recognizing the growing computational demands of modern research, ND has been actively coordinating with ND Office of Information Technologies (OIT) to facilitate access to high-tier Colab services. Specifically, a pathway has been established to enable upgrades to individual ND accounts, granting access to the enhanced features of Google Colab Pro and Pro+. These upgraded plans offer significant advantages, including faster GPUs, more memory and longer runtime, which are crucial for computationally intensive tasks such as analyzing vast datasets and uncovering hidden patterns in data. This collaborative effort aims to provide researchers with dynamically scalable compute resources directly integrated with their existing ND credentials and Google Drive storage.
A concrete example of this initiative is the collaboration with Prof. Ryan Sensenig's Research Experience in Biology Laboratory class. The CRC team provided the spring biology course with upgrades to Google Colab Pro for the period they require more compute resources. This will allow students to leverage enhanced computational power for their research projects, involving tools like RootPainter for grassland biology image analysis. The ability to easily upgrade student accounts to Colab Pro or Pro+ on a monthly basis, funded through a pilot program, showcases the commitment of the CRC to providing the necessary infrastructure for cutting-edge research and education in areas that increasingly rely on ML and AI. This coordinated approach ensures a seamless experience for students and faculty, allowing them to focus on their research without the complexities of managing separate accounts or logins.
Computational capability details and pricing can be found at the following link. All ND students and faculty already have full access to the base level via your ND ID. For upgrades to higher compute levels; do not sign up directly; contact us (crcsupport@nd.edu) with an ND FOPAL. We are also inviting CRC funded 1 month compute upgrades for prototype projects. https://colab.research.google.com/signup
|
The Center for Research Computing continues to expand our Generative AI offerings on our CRC Hosted Open WebUI GenAI platform available here (on campus or VPN, Notre Dame Google login required). Currently 70 users have signed up on this platform.
February Highlights Include:
|
-
API Documentation - There has been increased interest in using the Open WebUI API to use the models programmatically. To assist in getting started with this process, a comprehensive user guide on accessing models via an Open WebUI API key is provided in the CRC user documentation.
-
Additional Models - Over 40 models are available on the underlying GPU server including models with parameters within 70B and 405B. Not all models are publicly available for all users. If you have a specific question about what is available for use, contact CRCSupport
- Open WebUI Updates - We have applied several software updates and regularly maintained the application and its resources in the past month. The current and latest version is v.0.5.20.
|
If you are interested in specific system capabilities of Open WebUI or have research applications that could utilize this CRC hosted service, email crcsupport@nd.edu for a consultation with one of our staff members.
|
|
|
Tech Tip: Using Rclone to Transfer Data Between ND Google Drive and CRC Systems
|
Rclone is a powerful command-line tool that simplifies the transfer and synchronization of files between cloud storage services, such as Google Drive, and CRC systems. Researchers often need to move large datasets between CRC systems and cloud platforms, and rclone streamlines this process by enabling fast, secure transfers without the need for a graphical user interface. Rclone is installed on CRC systems, and once you configure it with your ND Google Drive account, you can efficiently manage files and automate transfers. This makes rclone an essential tool for handling large datasets in the CRC environment. For more information on setting up and using rclone, visit this link.
|
|
|
Apply to the CRC Graduate Award for Computational Science and Visualization
|
Please consider submitting a nomination for the annual CRC Award for Computational Sciences and Visualization. This award is open to all current Notre Dame students seeking advanced degrees and recent graduates. The use of CRC hardware or software is not required for the nominated work. Awardees will receive a $1,000 cash award and a plaque.
The award recognizes outstanding contributions in all areas of computational science and visualization. The deadline for nominations is March 31, 2025, at 11:59 p.m. Eastern.
|
|
|
Top 10 Computation Users (February 2025)
|
|
|
Chemical & Biomolecular Engineering
497,953 CPU hours
|
Chemical & Biomolecular Engineering
486,050 CPU hours
|
Chemical & Biomolecular Engineering
246,530 CPU hours
|
Chemical & Biomolecular Engineering
244,379 CPU hours
|
Chemical & Biomolecular Engineering
241,474 CPU hours
|
| Physics
200,871 CPU hours
|
Aerospace & Mechanical Engineering
179,158 CPU hours
|
Aerospace & Mechanical Engineering
175,346 CPU hours
|
Chemical & Biomolecular Engineering
158,385 CPU hours
|
Aerospace & Mechanical Engineering
115,693 CPU hours
|
|
|
Top 10 Graphics Processing Unit (February 2025)
|
|
|
Chemical & Biomolecular Engineering
16,244 GPU hours
|
Computer Science and Engineering
4,516 GPU hours
|
Chemical & Biomolecular Engineering
4,332 GPU hours
|
Computer Science & Engineering
3,668 GPU hours
|
Chemistry & Biochemis
3,665 GPU hours
|
| Chemical & Biomolecular Engineering
3,601 GPU hours
|
Computer Science & Engineering
3,582 GPU hours
|
Chemistry & Biochemistry
3,055 GPU hours
|
Computer Science & Engineering
2,430 GPU hours
|
Computer Science & Engineering
1,980 GPU hours
|
|
|
| User Training Office Hours |
Every Wednesday and Thursday
2:00 – 3:00 p.m.
812 Flanner Hall (map)
|
The CRC offers multiple training opportunities for both new and existing users. We periodically provide short courses and other learning opportunities, which are advertised on our website and through email lists. In-person office hours are held every Wednesday and Thursday from 2:00-3:30 p.m. in Flanner Hall, room 812, on a first-come, first-served basis. You can also arrange a Zoom meeting at your convenience by emailing CRCsupport@nd.edu with your availability. We recommend bringing a laptop to in-person sessions.
|
- A CRC User Account is required to participate. If you need an account, please fill out and submit the CRC Account Request Form.
-
Office hours will be held in 812 Flanner Hall. Click here to register.
|
|
|
Manage your preferences | Opt Out using TrueRemove™
Got this as a forward? Sign up to receive our future emails.
View this email online.
|
940 Grace Hall University of Notre Dame | Notre Dame, IN 46556 US
|
|
|
This email was sent to mreill14@nd.edu.
To continue receiving our emails, add us to your address book.
| |
|
|