Astronomer Logo

Astronomer

Principal Software Engineer

Posted 3 Months Ago
Be an Early Applicant
Hybrid
New York City, NY, USA
277K-390K Annually
Expert/Leader
Hybrid
New York City, NY, USA
277K-390K Annually
Expert/Leader
Lead architecture and implementation of a self-contained control plane and data plane to provision and manage on‑premises Apache Airflow across hybrid and air‑gapped environments. Design extensible APIs, telemetry/observability, Kubernetes operators and deployments, ensure security and backward compatibility, and provide technical leadership, design reviews, and senior mentorship.
The summary above was generated by AI

Astronomer empowers data teams to bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow®. Astro accelerates building reliable data products that unlock insights, unleash AI value, and power data-driven applications. Trusted by more than 900 of the world’s leading enterprises, including Datadog, American Express, Expedia, Adobe, Autodesk, Together AI, and more, Astronomer lets businesses do more with their data. To learn more, visit www.astronomer.io.

About the Role

We are seeking an exceptional Principal Software Engineer to provide technical leadership, architectural vision, and team mentorship for our core engineering group. In this role, you will be the primary architect and public face of a cloud-agnostic software platform that helps enterprise customers provision, scale, and orchestrate Apache Airflow instances completely on-premises within their own environments.

This is a unique infrastructure challenge: our platform is designed to be completely self-contained, operating without reliance on an external SaaS backbone or mandatory outbound communication. Both our control plane and data planes deploy locally inside the customer’s infrastructure. You will lead the engineering team in defining, designing, and building these systems to function reliably across a spectrum of enterprise network setups—ranging from standard customer-managed clouds to strictly isolated, air-gapped environments.

 

What You'll Do

  • Define, Design, and Build Core Architecture: Own the end-to-end definition, design, and implementation of both the control plane and data plane services, creating highly resilient inter-connections between them to power this self-contained infrastructure platform.

  • Architect an Extensible Platform API: Drive the design, data modeling, versioning, and long-term backward compatibility of robust APIs (such as REST or gRPC). These APIs serve as the core platform engine that power the customer-facing UI, and the developer endpoints for customer-facing programmatic integrations.

  • Design Comprehensive Platform Observability: Architect the telemetry framework (metrics, logs, and traces) for the self-contained platform itself, ensuring deep visibility into both the core control plane services and the performance, health, and status of the underlying Airflow instances it provisions.

  • Architect for Varied Isolation Levels: Oversee the architecture of a self-contained software platform, ensuring all services, images, dependencies, and configurations can run smoothly whether deployed in connected private clouds or highly secure, zero-egress networks.

  • Provide Technical Leadership: Act as the technical face of the team, translating complex enterprise infrastructure needs into execution roadmaps while aligning closely with product management and executive stakeholders.

  • Mentor and Elevate: Foster a culture of engineering and operational excellence through comprehensive design reviews, high-leverage code contributions, and direct career mentorship for senior engineers.

 

What You'll Bring

  • Extensive Infrastructure Experience: 10+ years of professional software engineering experience, with a significant track record operating at a Staff, Senior Staff, or Principal level within an enterprise infrastructure, database-as-a-service (DBaaS), or private cloud organization.

  • Control Plane & API Productization: Proven success building production-grade infrastructure control planes, with a deep understanding of API design principles required to support programmatic customer workflows and user interfaces.

  • Cloud-Native Observability Expertise: Practical experience designing and implementing infrastructure observability stacks (e.g., Prometheus, OpenTelemetry, Grafana) to capture, aggregate, and surface system health and workflow data within isolated environments.

  • Strong Kubernetes Foundation: Solid operational knowledge and architectural understanding of container orchestration platforms, specifically Kubernetes. You have practical experience building custom Kubernetes Operators (CRDs), managing Helm deployments, and working with the Kubernetes API to orchestrate application lifecycles.

  • Hybrid Cloud Architecture: Hands-on systems design expertise involving isolated environments across multiple public cloud providers (AWS, GCP, Azure), Red Hat OpenShift enterprise deployments, and bare-metal on-premise infrastructure.

  • Advanced Coding & Systems Design: Expert-level software development skills (e.g., Go, Python, TypeScript) with deep knowledge of distributed state stores (such as Etcd, Consul, or PostgreSQL), concurrency, and network security patterns.

  • Top-Notch Technical Communication: Exceptional oral and written communication skills with a proven ability to author crisp, high-density technical "one-pagers" and actionable RFCs. You can drive fast organizational alignment by distilling complex systems choices into brief, highly readable documents, presenting confidently to executive leadership, and engaging deeply with strategic enterprise clients to unearth their core infrastructure requirements.

The estimated total compensation for this role ranges from $277,000 - $390,000, along with an equity component and a comprehensive benefits package. This range is merely an estimate, and the width of the range reflects willingness to consider candidates with broad prior seniority. Actual compensation may deviate from this range based on skills, experience, and qualifications.

 

#LI-Fulltime

#LI-Hybrid

At Astronomer, we value diversity. We are an equal opportunity employer: we do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

HQ

Astronomer New York, New York, USA Office

54 W 21st St, New York, New York, United States, 10010

Similar Jobs

4 Days Ago
Hybrid
129K-219K Annually
Senior level
129K-219K Annually
Senior level
Aerospace • Hardware • Information Technology • Security • Software • Cybersecurity • Defense
Designs, develops, integrates, and verifies complex embedded real-time software for mission-critical electronic platforms. Leads technical solutions independently, applies software engineering standards, collaborates with systems, hardware, test, and program management teams, and represents engineering decisions to senior internal and external stakeholders. The role requires strong technical judgment, lifecycle expertise, and the ability to obtain and maintain a Secret clearance.
Top Skills: BitbucketContinuous IntegrationDo-178Embedded SystemsGitHardware DebuggersJIRAReal-Time SoftwareSoftware DebuggersStatic Analysis Tools
4 Days Ago
Hybrid
107K-183K Annually
Senior level
107K-183K Annually
Senior level
Aerospace • Hardware • Information Technology • Security • Software • Cybersecurity • Defense
Develop and maintain mission-critical software, including new features, standalone applications, and adaptive maintenance for existing systems. Translate customer requirements into scalable Java-based solutions, collaborate directly with customers, and support modernization roadmaps. Mentor engineers, conduct code reviews, establish development practices, and contribute to cybersecurity and geospatial intelligence capabilities. This is a fully onsite role requiring an active Top Secret clearance.
Top Skills: AgileApacheAutomated Testing FrameworksCi/CdCloud TechnologiesComptia Security+DevOpsDockerGradleHp FortifyJakarta EeJavaJunitLinuxLog4JMavenPostgresPythonRestful Web ServicesSpring BootSQLSteganographyVersion Control
19 Days Ago
Hybrid
2 Locations
Expert/Leader
Expert/Leader
Financial Services
Leads performance engineering across distributed systems, from browser to operating system. Defines SLOs, performance standards, capacity models, automated load testing, CI/CD performance gates, observability, and troubleshooting practices. Builds scalable self-service tooling, mentors engineers, influences senior stakeholders, and governs responsible AI-enabled engineering workflows with security, resiliency, and auditability controls.
Top Skills: Amazon EksBlazemeterCi/CdDynatraceEbpfGrafanaIstioJavaJdbcJmeterJvmK6KafkaKibanaKubernetesLinkerdLinuxMountebankNode.jsOpentelemetryOraclePerfPythonReactRumSpring BootSynthetic MonitoringToxiproxyTypescriptWiremock

What you need to know about the NYC Tech Scene

As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.

Key Facts About NYC Tech

  • Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
  • Key Industries: Artificial intelligence, Fintech
  • Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
  • Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account