Top Tech Jobs & Startup Jobs in NYC, NY

Design and run red-team evaluations to find failure modes and edge cases in frontier LLMs. Create reproducible, multi-step benchmark tasks, document technical findings, and collaborate with researchers to refine grading and improve model robustness. Commit ~35 hours/week as a remote independent contractor.

Design and author research-oriented, multi-step benchmark tasks for frontier AI models; create reference solutions and evaluation standards using Python and notebooks; review AI outputs to identify methodological and analytical flaws; collaborate with AI researchers to refine evaluation methodologies.

Develop and maintain engine-level systems, game modding tools, and editor extensions. Implement and optimize rendering pipelines and shaders using DirectX/Vulkan/OpenGL/Metal. Profile and optimize CPU/GPU/memory and multithreaded workloads, debug complex graphics issues, and collaborate with engineers, artists, and designers to deliver high-quality, maintainable C++ engine code.

Evaluate and document AI voice assistant interactions in German using a Mac with Apple Silicon. Complete voice-based tasks, transcribe and polish spoken content, assess AI responses for quality, maintain task logs, and upload recordings and files for multilingual voice-data collection.

Evaluate and improve LLM financial reasoning by designing real-world finance tasks, creating reference solutions, scoring AI outputs with structured rubrics, refining evaluation frameworks, and collaborating with researchers to raise data quality and model performance. Must provide detailed feedback and identify gaps to enhance AI across finance use cases.

Design realistic chip architecture and RTL, build verification environments and testbenches, evaluate AI-generated hardware solutions, provide technical feedback, and collaborate to improve AI understanding of digital design, verification, and semiconductor workflows. Commit 20–40 hours/week remotely as an independent contractor.

Remote contractor role to create and evaluate mathematical problems, proofs, and datasets to train and improve AI models. Assess AI-generated solutions for correctness, produce technical documentation, and collaborate with cross-functional teams to develop benchmark datasets and provide actionable feedback to improve model reasoning.

Assess finance tasks and workflows for professional realism, clarity, and completeness. Identify missing assumptions, ambiguities, edge cases, and impractical constraints. Validate workflow logic, evaluate whether tasks measure professional finance capabilities, and provide concise, structured written feedback and improvement recommendations while collaborating with other finance experts during a high-engagement remote sprint.

Validate and harden AI evaluation benchmarks by designing test cases, reviewing tasks and reference solutions, debugging Python evaluation scripts, identifying edge cases and exploits, developing repeatable QA processes, and collaborating with researchers to ensure reliable, reproducible benchmark results.

Design realistic, multi-step data analysis benchmark tasks and create reproducible reference analyses. Evaluate AI-generated analyses for correctness, statistical validity, and interpretation. Collaborate with researchers to improve benchmark quality and document methodologies in Jupyter/Colab notebooks.