The CRC Software Development team hosted their annual reCon intra-conference on September 25-27. The team took this opportunity to “reCon”nect with one another, do “reCon”naissance on new technologies or on projects that peers may be working on, and “reCon”sider individual and collective performance compared to future goals. This marked the third consecutive year of hosting reCon for the CRC.
During the conference, Caleb Reinking gave a plenary talk on the current state of AI coding companion technologies, its impacts on the software engineering profession, and direct applications for the role the CRC plays on campus as research software engineers. Throughout the rest of the schedule, nearly every member of the Software Development team took the opportunity to share some of their work and experiences with other peers.
One of the other highlights of the week was the Design Thinking workshop, facilitated by Mendoza Professor Wendy Angst. Design Thinking is a methodology for thinking about problems with a user-focused mindset, emphasizing empathetic thought activities. During this workshop, four different faculty and staff members were interviewed to understand their perspectives on the problem being discussed. The software team enjoyed this workshop as an opportunity to collaborate as a whole on a CRC software project, learn new skills, and think about building software in a new way!
|
|
|
Tech Tip: Maximizing Your Research with HTCondor for High Throughput Computing
|
High Throughput Computing (HTC) is well-suited for researchers who need to run numerous independent tasks over extended periods, maximizing computational efficiency. At CRC, HTCondor is the tool that enables you to manage and distribute these tasks effectively. HTCondor harnesses idle computing resources, borrowing machines that are not currently in use by Grid Engine jobs.
|
Getting Started with HTCondor
|
To begin using HTCondor at CRC:
|
- Log in to the Condor frontend at condorfe.crc.nd.edu.
- Create a submission file specifying parameters like the executable file, output locations, and log configurations.
-
Submit your job with the condor_submit command, queuing your task on available resources.
- Monitor job status using condor_q. If a job is idle for too long, the condor_q -better-analyze command helps diagnose issues.
|
HTCondor’s scheduling ensures your jobs are queued and executed when resources are available.
| Submitting Large Job Sets
|
HTCondor supports batch submissions, allowing you to run multiple jobs with varying parameters in parallel:
|
-
Use the $(PROCESS) macro to automate the submission of hundreds of job variations at once.
- HTCondor efficiently distributes these jobs, maximizing resource utilization.
|
Running Multicore and GPU Jobs
|
CRC’s Condor pool supports both multicore and GPU-accelerated jobs:
|
- For multicore jobs, modify your submission file to request multiple CPUs, enabling parallel execution for your applications.
- For GPU tasks, request GPU resources for workloads such as molecular dynamics or machine learning. Including GPU requests in your submission file ensures optimal resource assignment.
|
Resource Sharing and Preemption
|
In shared environments like CRC, remember that if you are borrowing machines owned by others, some jobs may be preempted (kicked off) and rescheduled elsewhere. HTCondor efficiently handles this by migrating or rescheduling jobs to ensure minimal disruption.
|
HTCondor is essential at CRC for optimizing High Throughput Computing tasks. Whether running a few jobs or managing hundreds, its dynamic scheduling and robust job management features make it crucial for researchers. Just keep in mind the possibility of preemption in shared environments, and rest assured that HTCondor will keep your jobs running with minimal downtime. For more information on HTCondor, please refer to the documentation available at CRC HTCondor Documentation.
|
|
|
| User Training Office Hours |
Every Wednesday and Thursday
2:00 – 3:30 p.m.
812 Flanner Hall (map)
|
The CRC offers multiple training opportunities for both new and existing users. We periodically provide short courses and other learning opportunities, which are advertised on our website and through email lists. In-person office hours are held every Wednesday and Thursday from 2:00-3:30 p.m. in Flanner Hall, room 812, on a first-come, first-served basis. You can also arrange a Zoom meeting at your convenience by emailing CRCsupport@nd.edu with your availability. We recommend bringing a laptop to in-person sessions.
|
- A CRC User Account is required to participate. If you need an account, please fill out and submit the CRC Account Request Form.
-
Office hours will be held in 812 Flanner Hall. Click here to register.
|
|
|
Top 10 Computation Users (September 2024)
|
|
|
Civil & Environmental Engineering & Earth Sciences
630,427 CPU hours
|
Aerospace & Mechanical Engineering
604,450 CPU hours
|
Chemical & Biomolecular Engineering
459,312 CPU hours
|
Chemical & Biomolecular Engineering
444,554 CPU hours
|
Chemical & Biomolecular Engineering
408,989 CPU hours
|
| Biological Sciences
346,989 CPU hours
|
Civil & Environmental Engineering & Earth Sciences
294,417 CPU hours
|
Biological Sciences
270,580 CPU hours
|
Chemical & Biomolecular Engineering
263,647 CPU hours
|
Aerospace & Mechanical Engineering
240,746 CPU hours
|
|
|
Manage your preferences | Opt Out using TrueRemove™
Got this as a forward? Sign up to receive our future emails.
View this email online.
|
940 Grace Hall University of Notre Dame | Notre Dame, IN 46556 US
|
|
|
This email was sent to mhughe24@nd.edu.
To continue receiving our emails, add us to your address book.
|
|
|
|
