Design, build, and own ML evaluation systems and production-grade NLP/LLM solutions. Translate research into scalable agentic AI systems, collaborate with Research/Product/Platform, ensure quality, safety, and performance, and mentor engineers.
At ASAPP, our mission is simple: deliver the best AI-powered customer experience—faster than anyone else. To achieve that, we’re guided by principles that shape how we think, build, and execute. We value customer obsession, purposeful speed, ownership, and a relentless focus on outcomes. ASAPP’s AI Engineering team is seeking an enterprising, talented and curious machine learning engineer.
The AI Engineering team is responsible for working closely with the research and modeling teams to create state-of-the-art NLP models for specific tasks, and deploy them in a production setting designed to serve our customers at scale. We are looking for a Machine Learning Engineer to help build and evaluate the core intelligence behind our agentic AI systems. This role will play a key part in designing and owning evaluation frameworks that ensure quality, safety, and performance across complex agentic systems.
This a hybrid role with 10-12 days of in-office presence per month to balance flexibility with collaboration.
What you'll do
- Design and development of ML evaluation systems for agentic and LLM-based architectures
- Translate research ideas into production-grade ML systems with measurable impact
- Partner closely with Research, Product, and Platform teams to productize experiments into robust AI solutions
- Stay current with advancements in ML, NLP, and LLM systems, contributing actively to technical discussions across teams
- Mentor and support other engineers through design reviews, feedback, and knowledge sharing.
What you'll need
- Deep experience in NLP and modern ML systems, including hands-on experience with LLM-based or agentic systems.
- Strong architectural skills, with proven experience designing complex software systems along with production experience with Python, AWS, Kubernetes, and/or Docker.
- Experience implementing technical solutions, tackling tough engineering problems to help build our products and a deep passion for Machine Learning.
- A Bachelor’s Degree in CS or other related fields
- Desire to learn new things, work closely with peers from different teams, and open to teach and learn from others.
- Demonstrate proficiency in the technical mentorship of junior and mid-level engineers, driving the adoption of best practices and ensuring architectural alignment for scalability and extensibility.
What we'd like to see
- Experience building and evaluating agentic systems at scale.
- Production experience with LLM-centric services (e.g., inference, orchestration, evaluation, monitoring)
- Familiarity with large-scale ML experimentation, benchmarking, or simulation framework
- Knowledge of techniques for optimizing model architectures for faster inference
- Experience with AWS, CI/CD, Kafka, Athena
Benefits include:
Competitive compensation with stock options
Comprehensive medical, vision, and dental insurance
401k matching
Fitness and wellness stipend
Mental well-being benefits
Professional learning and development stipend
Parental leave, including adoptive and foster parents
3 weeks paid time off (increases with tenure) along with sick leave, bereavement and jury duty
ASAPP is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, disability, age, or veteran status. If you have a disability and need assistance with our employment application process, please email us at [email protected] to obtain assistance. #LI-AG1 #LI-Hybrid
ASAPP New York, New York, USA Office
New York, NY, United States
Similar Jobs
Financial Services
Build and maintain scalable machine learning infrastructure, including distributed GPU training pipelines, model promotion workflows, monitoring integrations, and ML frameworks. Optimize training throughput, support model deployment and observability, and collaborate with product, architecture, data science, and engineering teams. The role requires strong Python, cloud, ML framework, big data, training and inference system, and enterprise design experience, with preferred expertise in recommendation systems, CUDA, containers, orchestration, and data storage.
Top Skills:
AirflowAmazon EcsSparkCloud ComputingCudaDistributed Model TrainingDockerGpu ClustersKubeflowKubernetesMl Model MonitoringMl ObservabilityPythonPyTorchRaySglangTensorFlowVllm
Artificial Intelligence • Machine Learning • Natural Language Processing • Software
Lead development of an evaluation platform for agentic AI and LLM systems. Design benchmarking, human review, LLM-as-judge, monitoring, annotation, dataset versioning, and data quality workflows. Own technical architecture, production infrastructure, stakeholder reporting, and platform health while partnering with research, product, and platform teams. Mentor engineers and support scalable AI experimentation and deployment.
Top Skills:
AthenaAWSCi/CdDockerKafkaKubernetesLlmsMachine LearningNlpPython
Financial Services
Lead MLOps engineer to build and maintain distributed training pipelines, batch and real-time model serving, LLM quantization and deployment, vector DB optimization, monitoring/observability, and cross-team integration for production ML in AWS.
Top Skills:
Apache AirflowSparkAwqAWSCudaDockerDuckdbEcsGpusKubeflowKubernetesLlmsPtqPythonRaySglangVector DatabasesVllm
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
