HPC Engineer/Architect

Job Title: HPC Engineer/Architect

Work Location: New York, NY 10001

Job Type: Contract

Work Type: Hybrid

Duration: 6+ Months

USC/GC/H4/H1

Job Summary:

• You will support day-to-day operations of large-scale parallel file systems, deploy and maintain Linux HPC infrastructure across multiple data centers, and assist HPC engineers and architects with day-to-day operations and tickets.

• Support day-to-day operations of large-scale parallel file systems

• Deploy and Maintain Linux HPC infrastructure across multiple datacenters

• Assist HPC engineers and architects with day-to-day operations and tickets

Experience:

• 16 to 20 years

Required Skills:

• Linux Operating Systems (RHEL/CentOS), Parallel file system (GPFS), Job Scheduler LSF/Slrm

• Anxible, Python, Shell scripting

• GPU-based compute infrastructure (including CUDA)

• CentOS 4.5

• HPCC

Responsibilities:

• Design, architect and oversee implementation of Linux based HPC clusters and storage

• Deploy physical hardware using HPC deployment tools and configuration and orchestration tools (Ansible)

• Parallel file system (GPFS) performance tuning, monitoring and troubleshooting

• Perform systems benchmarking, and developing automated tests for the HPC environment, ensuring the reliability and efficiency of our computational infrastructure

• Infiniband network maintenance and troubleshooting

• Automate and monitor the HPC user lifecycle process

• Slurm installation, configuration, performance tuning and troubleshooting

• Plan, design and implement a transition from the LSF scheduler to Slurm

• Manage the Slurm scheduler and translate Research policies into scheduler configurations

• Consult with faculty and students to develop research pipelines for use on the HPC cluster

• Develop and maintain user lifecycle software suite in Python, implement CI/CD pipeline

• Test and automate upgrades of critical system applications using Ansible and shell scripts.

• The ability to communicate effectively with clinicians, researchers, and other team members to develop technological solutions is key

Qualifications:

• Experience working in a large-scale research based HPC environment

• Proven experience working with distributed file storage solutions (i.e., GPFS)

• Experience with deploying and troubleshooting Linux Operating Systems (RHEL/CentOS)

• Experience with Scripting and Automation (Ansible, Python, Shell Scripting)

• Solid understanding of job schedulers (LSF/SLURM)

• Experience with GPU-based compute infrastructure (including CUDA)