About Gramian
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
About the Role
We are looking for a physics-focused professional to contribute to an advanced AI research initiative developing rigorous scientific coding tasks. The role combines physics expertise, Python-based scientific computing, and AI evaluation, requiring you to design complex scientific problems, implement verified solutions, and develop tests that assess the accuracy of AI-generated outputs. You will work on ensuring scientific correctness, determinism, and evaluation quality across structured STEM training tasks.
Responsibilities
Design scientific coding tasks consisting of one main physics problem and at least 3 logically connected sub-problems.
Implement verified golden solutions in Python with complete unit test coverage.
Develop discriminative test cases that distinguish correct from incorrect AI-generated outputs.
Perform quality control checks using the Turing Central Task Platform (CTP), including Tier 1 structure checks and Tier 2 quality rubrics.
Revise scientific tasks and solutions based on QC feedback.
Optimize tasks to meet Pass@K evaluation criteria across multiple LLM judges, including GPT, Gemini, and Nemotron.
Validate scientific correctness, well-posedness, determinism, and expected outputs.
Maintain high first-submission quality and minimize rework.
Participate in reviews, feedback sessions, and project standups during required overlap hours.
CONTRACT: Freelance / Contractor, short-term (8 weeks)
COMMITMENT: 40 hours per week, including 4 hours of overlap with PST working hours
LOCATIONS: Remote — Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Indonesia, Kenya, Nigeria, Pakistan, Turkey, or Vietnam
PROCESS: Not specified
Requirements
- Master's degree or PhD in Physics.
- Strong Python programming skills with experience in scientific computing.
- Experience with scientific Python libraries such as NumPy, SciPy, SymPy, or relevant domain-specific tools.
- Ability to formulate rigorous physics problems with clear constraints and expected outputs.
- Experience implementing scientific solutions with unit tests and deterministic outputs.
- Prior experience in AI data annotation, scientific research, or scientific writing.
- Familiarity with LLM evaluation frameworks or coding benchmarks.
- Published research or academic project experience in physics or another STEM discipline.
Similar Jobs
What you need to know about the NYC Tech Scene
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
