Build and operate scalable AI platform infrastructure enabling secure, reliable access to foundation models and AI agents. Responsibilities include developing model gateways, provider integrations, routing, usage controls, observability, governance, and self-service platform capabilities. The role involves architecting distributed systems, improving reliability and security, collaborating across engineering and product teams, mentoring engineers, and establishing high-quality engineering practices.
As a Staff Software Engineer on Aledade's AI Core Platform team, you will help build and operate the shared infrastructure that enables applications and AI agents across Aledade to securely and reliably access foundation models.
You will work on the platform capabilities behind model access, including model gateways, cloud AI services, provider integrations, routing and fallback, usage controls, observability, reliability, and model governance.
We’re looking for engineers who think beyond implementing individual features and consider the full lifecycle of production platform systems: architecture, scalability, failure modes, security, observability, operational readiness, cost, and developer experience. You will help evolve the platform as AI adoption grows across Aledade, building reusable and self-service capabilities rather than one-off solutions.
You will work closely with engineers, product partners, security, infrastructure teams, and teams building AI applications and agents. Successful engineers in this role are comfortable operating in ambiguous problem spaces, making pragmatic technical decisions, and building secure, maintainable, observable, and resilient systems.
Primary Duties:
- Identify and develop scalable and performant solutions.
- Work across discipline to shape product strategy and execution.
- Develop the foundations of code architecture and quality.
- Mentor and coach engineers.
- Set and uphold the standard for engineering processes to support high-quality engineering.
Minimum Qualifications:
- BS/BTech (or higher) in Computer Science, Engineering or a related field required.
- 8+ years of production-level experience as an engineer building highly scalable systems.
- 4+ years of experience acting as a trusted technical decision-maker in a team setting, solving for short-term and long-term business value.
- 4+ years of experience working with SQL or other database querying languages on large multi-table data sets.
- Experience architecting, developing, and deploying large-scale distributed systems at scale.
- Experience with cloud technologies, e.g., AWS, Azure, GCP.
- Experience building continuous integration and continuous development (CI/CD) pipelines.
- Strong familiarity with server-side web technologies (eg: Java, Python, Scala, C#, C++, Go).
Preferred KSAs:
- Experience building or operating production platform infrastructure, distributed systems, developer platforms, or other high-availability shared services.
- Experience with LLM infrastructure, model gateways, foundation model APIs, or managed AI platforms such as AWS Bedrock, Azure OpenAI, Google Vertex AI, or similar technologies.
- Strong experience with cloud infrastructure, preferably AWS, and technologies such as Kubernetes, containers, and infrastructure as a code.
- Experience designing observability and operational practices for production systems, including metrics, distributed tracing, logging, alerting, SLOs, dashboards and failure handling.
- Experience designing APIs and self-service platform capabilities used by other engineering teams.
- Experience with Identity, access control, secrets management, security and governance for shared infrastructure.
- Experience with cost management, usage attribution, quotas, budgets, or capacity management for shared platforms.
- Experience operating systems that handle sensitive or regulated data.
- Ability to evaluate architectural tradeoffs across reliability, performance, security, cost, scalability and developer experience.
AI Infrastructure & Model Platform:
Physical Requirements:
Sitting for prolonged periods of time. Extensive use of computers and keyboard. Occasional walking and lifting may be required.
Base salary range: $176,000 – $238,000 USD annually
The range above reflects the base pay we reasonably expect to offer for this role across all U.S. locations. Where an individual offer lands within the range depends on the candidate's skills, relevant experience, qualifications, scope of the role, and internal equity. This role is eligible for an annual performance bonus/incentive and equity compensation in addition to the benefits listed below.
Similar Jobs
Artificial Intelligence • Fintech • HR Tech • Social Impact • Software • Analytics
Own and evolve Syndio’s AI platform runtime, including agent orchestration, tool calling, memory, context management, RAG, evaluations, observability, and production reliability. Design and ship backend systems in Python on GCP and Kubernetes, establish platform contribution standards, support compliance requirements, collaborate with product teams, and participate in incident response and on-call operations.
Top Skills:
ClaudeDatadog Llm ObservabilityGoogle Cloud PlatformKubernetesMicrosoft TeamsPostgresPythonRagSlackSseVertex Ai
Social Media
Lead the technical strategy, architecture, modernization, and company-wide adoption of Pinterest’s AI/ML evaluation platform. Design reliable evaluation systems for GenAI and machine learning workloads using automated metrics, human feedback, expert judgment, safety assessments, and business outcomes. Drive cross-team workstreams, resolve technical tradeoffs, establish evaluation standards, and improve the quality, reliability, and trustworthiness of AI-powered products.
Top Skills:
AutoevalsData VersioningDistributed SystemsExperimentation PlatformsGenaiMachine LearningModel VersioningObservabilityOpenevalsOrchestrationProduction MonitoringPythonRagasRecommender SystemsRelease Management
Information Technology • Legal Tech • Software
Build and operate LLM-backed pipelines for document extraction, classification, and validation. Develop evaluation datasets, regression suites, metrics, and human review workflows to improve AI accuracy. Own production reliability through prompt versioning, structured output validation, cost and latency optimization, retries, and fallbacks. Collaborate with litigation management stakeholders to identify high-value AI applications and deliver reliable systems for processing complex medical records and claim files.
Top Skills:
AWSElasticsearchGraphQLLlm ApisOcrPostgresScalaTypescript
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
