NVIDIA Logo

NVIDIA

Senior Software Engineer, Resilience Engineering - DGX Cloud

Posted 7 Days Ago
In-Office or Remote
Hiring Remotely in Santa Clara, CA
184K-357K Annually
Senior level
In-Office or Remote
Hiring Remotely in Santa Clara, CA
184K-357K Annually
Senior level
Build NVIDIA DGX Cloud’s organization-wide reliability strategy and SLO program. Lead high-severity incident response, develop production code and data-platform tooling, implement chaos engineering and failure-injection practices, and improve operational standards across teams. The role requires extensive experience operating large-scale production systems, understanding complex failure modes, and influencing engineering teams through technical expertise.
The summary above was generated by AI

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.


Are you passionate about building world-class reliability systems? Join NVIDIA as a Senior Software Engineer - Resilience Engineering, DGX Cloud, and be a pivotal part of a team that redefines operational excellence. Our team is at the forefront of redefining how DGX Cloud approaches reliability, making it an outstanding opportunity to develop strategies and drive innovation. We're looking for a seasoned engineer with experience in running large-scale systems and a deep understanding of operational practices.


What you'll be doing:

  • Build org-wide reliability strategy, guiding how NVIDIA matures its operational practices in a 24/7 environment.
  • Stand up a rigorous SLO program, defining and maintaining high standards across teams.
  • Lead incident response for high severity incidents, ensuring low drama and high signal resolution.
  • Build and improve production code daily, enhancing our data platform and related tooling.
  • Implement chaos engineering, failure injection, and resilience testing to elevate our team's standard practices.
  • Improve standards by setting an example with your hands-on experience and leadership.

What we need to see:

  • Deep, hands-on experience running large-scale production systems with a proven track record.
  • A detailed understanding of failure modes in large systems, including cascading dependencies and retry storms.
  • Strong software engineering skills with current, hands-on experience in Go, Python, or similar languages.
  • Proven experience in establishing and maintaining an SLO program with operational rigor.
  • Practical experience in reliability fields such as chaos engineering and failure injection.
  • The ability to influence across team boundaries through credibility and expertise.
  • 10+ years of industry experience with a Bachelor's or Master's degree, or equivalent experience operating systems at scale.

Ways to stand out from the crowd:

  • Experience within a world-class reliability function like Google SRE or Meta production engineering.
  • Expertise in operating GPU, HPC, or AI training infrastructure with outstanding failure modes.
  • A track record of measurable reliability improvements within an organization.
  • Proficiency with modern observability and operational tools like Prometheus, OpenTelemetry, Grafana, PagerDuty, and Rootly.

Widely considered to be one of the technology world’s most desirable employers, NVIDIA offers highly competitive salaries and a comprehensive benefits package. As you plan your future, see what we can offer to you and your family www.nvidiabenefits.com/ 

Your base salary will be determined based on your location, experience, and other objective job-related factors. The expected base salary ranges for this position are:
187,500 USD – 364,000 USD for Level 4; and 232,500 USD – 442,000 USD for Level 5.

You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until November 23, 2026.

This posting is for an existing vacancy. 

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Similar Jobs

An Hour Ago
Remote or Hybrid
California, USA
Senior level
Senior level
Consumer Web • eCommerce • Information Technology • Retail • Software • Analytics • App development
Leads state and local government affairs across Western markets, including policy monitoring, legislative analysis, advocacy strategy, lobbying, stakeholder engagement, and relationships with elected officials, trade associations, NGOs, and community groups. Partners with internal business functions to align public policy initiatives with enterprise priorities, directs lobbyists and consultants, and serves as a subject matter lead on retail policy issues. The role is based in Sacramento and requires travel up to 40%.
2 Hours Ago
Remote or Hybrid
United States
71K-120K Annually
Mid level
71K-120K Annually
Mid level
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
Analyzes operational data across payments, fraud, customer service, compliance, and processing to identify trends, anomalies, risks, control gaps, and process breakdowns. Develops predictive monitoring models, dashboards, and exception reports; leads root cause analysis; recommends corrective actions; and helps standardize control reporting for operations and leadership.
Top Skills: Power BIPython
2 Hours Ago
Remote or Hybrid
United States
59K-99K Annually
Mid level
59K-99K Annually
Mid level
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
Analyzes fraud cases, reviews documentation and account activity, makes audit-ready determinations, monitors fraud and operational risk trends, and develops control reporting. The role identifies control gaps, supports remediation, strengthens fraud oversight and governance, and partners with operations, risk, compliance, privacy, and investigations teams to improve mitigation processes.

What you need to know about the NYC Tech Scene

As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.

Key Facts About NYC Tech

  • Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
  • Key Industries: Artificial intelligence, Fintech
  • Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
  • Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account