Own the architecture and hands-on development of production ML and LLM capabilities across an internal product portfolio. Build and evaluate models, multi-tenant AI services, security controls, monitoring, and reusable engineering patterns. Lead AI architecture decisions, mentor senior engineers, advise executives, and collaborate with product, security, legal, and compliance teams. The role is primarily hands-on, remote within the U.S. or hybrid in New York City, and offered as a 12-month contract.
The Opportunity
We’re partnering with a leading global management consulting firm that advises C-suite leaders on their most complex problems. Its internal platform and product group builds and runs the 30+ products its consultants use, and AI is now going into all of them. Existing products are adding AI features, and new products are being built from AI use cases that are still being defined.
This role owns how AI gets built across that portfolio, and you’ll build much of it yourself. The firm is hiring Principal Engineers first and building teams around them, so you’d come in early and set the patterns everyone else follows. Your users are senior partners who have little patience for AI that demos well and breaks in production. A separate platform team owns infrastructure, so your focus stays on the AI itself.
Expect 70-80% hands-on engineering, with the rest on architecture and mentoring. The role is internal facing with no travel, and it’s open to remote within the U.S. or hybrid in Midtown NYC. It’s a 12-month contract (W2, 1099, or C2C) with a strong likelihood of extension.
What You’ll Own
- Build and ship ML and LLM capabilities into existing products, and help turn early AI use cases into new products. This means training, fine-tuning, and evaluating models, not just prompting them.
- Set the AI architecture: reusable service scaffolding, model gateway and interface patterns, and multi-tenant LLM services with enforced data boundaries between engagements.
- Own AI security from the start. That covers prompt injection defenses, prompt data classification, model supply chain risk, and agent governance (permissions, code execution controls, access to internal systems, human-in-the-loop). You’ll partner with Security, Legal, and Compliance on SOC2/ISO-aligned controls and vendor DPA requirements.
- Build the evaluation and monitoring layer: model evaluation rubrics, regression testing for models and prompts, production-readiness criteria, and quality monitoring that teams adopt without hand-holding.
- Lead major AI decisions through written trade-off docs, advise senior stakeholders on AI risk, mentor 5-7 senior engineers, and work closely with product managers. You’ll also champion AI-assisted development practices across the engineering org.
What You Bring
- 12+ years in software engineering, ML, or data science, including production ML and LLM systems you built, trained, and shipped. Experience applying AI in consulting or professional services preferred.
- Deep Python and ML framework expertise (PyTorch, TensorFlow, Hugging Face), including training and fine-tuning, plus experience deploying models to public cloud. Azure preferred; AWS or GCP is fine.
- Production experience designing and running multi-tenant AI services and LLM integrations: RAG, retrieval, model routing, and managing latency and cost. Familiarity with AI gateway patterns and emerging integration protocols (MCP, etc.) preferred.
- Hands-on model evaluation: offline and online evals, golden datasets, LLM-as-judge with calibration, and production quality monitoring. MLOps tooling experience preferred.
- Substantive AI/LLM security experience: prompt injection, data boundary enforcement, model supply chain risk, AI threat modeling, and sandboxing agents. Experience defining AI controls for SOC2 or ISO 27001 preferred.
- Microservices architecture experience: decomposing monoliths, defining service contracts, and CI/CD for distributed systems. You should be comfortable with Docker, but you won’t own the infrastructure.
- Solid engineering habits around testing, code review, and releases. Go or TypeScript is a plus.
- Strong written and verbal communication, including with executives. You give the answer first and add detail when asked. Open-source contributions, publications, or a track record of internal docs and workshops preferred.
- Comfort working in Agile, cross-functional product teams.
Similar Jobs
Insurance
Serves as a hands-on technical leader for emerging technology and AI experimentation. Designs prototypes, proofs of concept, pilots, integrations, and architecture evaluations; writes and reviews prototype code; troubleshoots complex technical issues; and develops reusable standards and recommendations. Collaborates with architecture, security, data, risk, legal, procurement, vendors, and delivery teams to transition validated concepts into scalable solutions. Mentors engineers and communicates findings, tradeoffs, risks, and adoption recommendations to technical and business stakeholders.
Top Skills:
Agentic AiAIAmazon DynamodbAmazon S3APIsAws LambdaCloud ComputingGenerative AiIntelligent Document ProcessingJ2EeJavaJavaScriptJenkinsMavenNode.jsPythonReactTypescript
.jpg){kind=logo}
Healthtech • Social Impact • Software • Telehealth
Owns the strategy, architecture, and implementation of production AI and machine learning systems for personalization, search ranking, relevance, and patient-provider matching. Builds recommendation engines, retrieval models, LLM applications, and scalable ML infrastructure while establishing AI safety, evaluation, and MLOps standards. The role requires hands-on model development, technical leadership, mentoring, and responsible deployment of AI in a regulated mental healthcare environment.
Top Skills:
A/B TestingAi AgentsAnthropicCollaborative FilteringData PipelinesElasticsearchFaissGdprGeminiGenerative AiGoHipaaJavaLearning-To-RankMapMilvusMlopsNdcgNlpOpenaiOpensearchPineconePythonRagSemantic Vector SearchTypescriptWeaviate
HR Tech • Consulting
Lead development and adoption of AI-driven and traditional test automation, maintain UI automation frameworks, own performance/load testing strategy, align testing to architecture and risk, mentor QA engineers, and collaborate with product and engineering to improve quality and testing efficiency.
Top Skills:
Agent-Driven TestingAi-Assisted TestingApi Testing ToolsGitJunitLoad TestingPerformance TestingSeleniumTestng
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
