About the company
Our client is a fast-growing software company.
The role
Raydar is recruiting for this role on behalf of our client. Own machine learning projects from initial scoping through production release, building the systems that allow automated AI agents to reason, plan and be assessed reliably. The work blends research-style experimentation with hands-on engineering and a high degree of autonomy.
What you'll do
- Design and iterate on multi-agent systems that automate complex professional workflows.
- Define safe operating limits, tool usage and fallback behavior so automated agents remain dependable.
- Manage context and memory across multi-step agent runs, with clear success measures.
- Route, assess and tune models within practical limits on speed, cost and accuracy.
- Build scalable offline and online evaluation pipelines that run large numbers of experiments automatically.
- Define benchmark tasks, labeling approaches and metrics so performance can be compared over time.
- Instrument systems to catch regressions, categorize errors and drive continuous improvement.
- Shape prompt structures and instruction hierarchies, and create search and indexing pipelines that supply the right information to models.
- Convert unstructured documents into structured representations and add validation layers that keep behavior predictable.
- Write concise specs and design documents, test your work end to end, and share progress and learnings with your immediate team.
Requirements
What we're looking for
- Background at a fast-paced startup, a quantitative trading firm, or a leading technology company.
- Based in the US or Canada at the moment, with a preference for the New York area or the east coast.
- Practical experience delivering LLM-powered agent products, including performance benchmarking, coordinating multiple models and testing how reliably agents behave.
- Experience designing and running rigorous ML experiments, from forming hypotheses to building evaluation tooling and acting on measured results.
- A degree in computer science, physics, math or another technical field from a top school.
- Clear communication and the ability to break complex ideas into simple parts.
- Enthusiasm for a startup environment and for working in the office five days a week.
Bonus points
- Strong Python skills and depth with large language model systems.
- Prior work shipping ML-driven products in a fast-moving company.
- Curiosity about how AI is changing finance and the wider economy.
Benefits
Compensation and benefits
- Base salary: USD 175,000 to 350,000 per year
- Equity
- Health, dental and vision coverage
- Life insurance
- Unlimited paid time off and paid holidays
- Retirement plan
- Parental leave
- Commuter benefits
- Mental health and wellness support
Location and work model
- New York, NY, United States
- On-site, 5 days per week in office
- Full-time
Similar Jobs
What you need to know about the NYC Tech Scene
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
