Legora Logo

Legora

Technical Legal Engineer - Evals

Reposted 2 Months Ago
Be an Early Applicant
In-Office
New York City, NY, USA
Entry level
In-Office
New York City, NY, USA
Entry level
Review and evaluate AI-generated legal outputs for accuracy, completeness, reasoning, jurisdictional correctness, usability, and legal risk. Develop evaluation datasets, benchmark cases, gold-standard answers, scoring rubrics, and guidelines. Analyze recurring errors, edge cases, and failure patterns, then translate legal feedback into actionable guidance for product managers, engineers, and legal engineers. The role applies legal judgment at scale without production coding, client advice, or traditional billable work.
The summary above was generated by AI
About Us

Legora is redefining how legal work gets done. Not built for lawyers, built with them. We work alongside the world’s best legal teams, who expect excellence, precision, and speed, and we hold ourselves to the same bar.
Our AI-native workspace lets legal professionals move faster, think more clearly, and operate with sharper precision. By analysing thousands of documents in minutes and powering end-to-end workflows, we cut through complexity, teams can focus on what matters: judgment, strategy, and outcomes.
2,100+ customers across 80+ countries trust us, including Cleary Gottlieb, Goodwin, Linklaters, White & Case, Dentons, and Barclays. We’ve scaled to $200M+ in ARR, with teams across Europe, North America and APAC, and continue to expand through acquisitions including Qura, Walter AI, Graceview, Cadastral, and Wexler.
We partner with world-class performers: including Aaron Judge and the New York Yankees, Ludvig Åberg (and his caddie), and campaigns featuring Jude Law.
Joining Legora means three things.

  • We lean in: ownership over titles, outcomes over intentions.

  • We fight for excellence: high standards, direct, ego-free feedback.

  • We grow together: as a team and with our customers.

Mission before ego. Everyone contributes. No one coasts.

If you’re driven by impact, pace, and raising the bar. This is the place.

The Role

We’re looking for legal professionals with strong analytical skills, technical curiosity and an interest in improving how AI supports legal work.

As a Technical Legal Engineer in the Evals function, you’ll use your legal training and practical experience to evaluate and improve Legora’s products. You’ll define what good legal work looks like, build evaluations that measure it, and help product teams understand where their systems work well and where they need to improve.

You could work across a range of legal workflows, including legal research, drafting and analysis. Depending on your focus, you may join a specific product team and develop deeper expertise in its workflows and evaluation needs.

One example is our Legal Research team, where the role combines hands-on legal evaluation with coordinating contributors who build a growing collection of research evaluations.

What You Will Be Doing

Across the Evals function, you will:

  • Evaluate legal outputs: Review AI-generated work for accuracy, completeness, legal reasoning and practical usability, using the judgment you would apply when reviewing legal work.

  • Build evaluations: Develop cases, reference answers and scoring criteria that enable consistent assessment of product quality.

  • Investigate failures: Identify edge cases, recurring errors and legal ambiguities, and distinguish minor issues from substantive legal risks.

  • Translate findings into improvements: Give product managers and engineers clear, actionable feedback and help assess whether changes improve the experience for lawyers.

  • Maintain evaluation quality: Keep evaluations relevant as products, models and legal requirements evolve.

What You Bring

For all areas of the role, we’re looking for:

  • Legal training and practical experience applying legal judgment to complex work.

  • Strong analytical skills and the ability to explain why an output or source is useful, incomplete or incorrect.

  • An interest in AI products and a willingness to learn how language models and evaluations work.

  • Clear communication skills and the ability to translate legal findings for product managers, engineers and other collaborators.

  • A hands-on, curious approach to investigating problems and improving quality.

We’re also appreciate:

  • Practical experience conducting and reviewing legal research, with sound judgment about authority, relevance and the limits of a conclusion.

  • Experience coordinating projects or contributors, setting clear expectations and keeping work moving through feedback and review.

Experience with evaluation datasets, research supervision or legal technology would be valuable. Production coding is not required; you’ll work closely with engineers on technical implementation.

The Impact You’ll Have

You’ll help Legora build a dependable understanding of legal product quality: where our products perform well, where they fail and whether improvements hold up under testing. If you focus on Legal Research, you’ll also help expand research coverage through reviewed, reusable evaluations and a contributor programme.

Legora is an Equal Opportunity Employer

At Legora, we believe great teams are built on diversity of thought and experience. We’re proud to be an equal opportunity employer and committed to creating an inclusive, high-performance culture where everyone can do their best work. We welcome people of all backgrounds and don’t discriminate based on race, color, religion, national origin, gender, gender identity or expression, sexual orientation, age, disability, veteran status, or any other characteristic protected by law.

HQ

Legora New York, New York, USA Office

Legora New York Headquarters Office

838 Broadway, New York, New York, United States, 10003 4812

Similar Jobs at Legora

2 Days Ago
In-Office
New York City, NY, USA
176K-205K Annually
Senior level
176K-205K Annually
Senior level
Artificial Intelligence • Legal Tech • Software
Own product marketing for Legora’s in-house legal customer segment. Define positioning, messaging, go-to-market strategy, launches, sales enablement, website storytelling, competitive intelligence, and customer insights. Partner with Product, GTM, Growth, Design, and executive leadership to influence roadmap, increase adoption, improve conversion, and drive pipeline and revenue for AI-powered products.
Top Skills: AI
5 Days Ago
In-Office
New York City, NY, USA
175K-206K Annually
Entry level
175K-206K Annually
Entry level
Artificial Intelligence • Legal Tech • Software
Manage Legora’s analyst relations operations, including engagement calendars, briefings, inquiries, events, RFI submissions, research tracking, and communications. Analyze analyst sentiment and coverage, measure share of voice, translate research into recommendations and sales proof points, and identify opportunities to strengthen Legora’s legal tech and AI positioning. Collaborate closely with Product, Engineering, GTM, Sales, Marketing, subject matter experts, and executives.
Top Skills: Artificial IntelligenceB2B SaasLegal Technology
5 Days Ago
In-Office or Remote
New York City, NY, USA
270K-320K Annually
Senior level
270K-320K Annually
Senior level
Artificial Intelligence • Legal Tech • Software
Lead Legora’s legal innovation thought leadership and advise law firms and legal departments on AI transformation, strategy, rollout, pricing, value, policy, and success measures. Represent the company at industry forums, publish market analysis, support executive customer conversations, and partner across sales, onboarding, marketing, product, and value engineering. The role requires firsthand legal-sector transformation experience, strong public presentation skills, and credibility with senior legal executives.
Top Skills: Ai TransformationArtificial Intelligence

What you need to know about the NYC Tech Scene

As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.

Key Facts About NYC Tech

  • Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
  • Key Industries: Artificial intelligence, Fintech
  • Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
  • Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account