Review robot task videos and annotations to ensure high-quality datasets, provide structured feedback to annotators, refine annotation guidelines, observe and coach teleoperation pilots, evaluate pilot readiness, track quality metrics, investigate issues, and collaborate with Autonomy and Operations to improve data collection and pilot performance.
About Ultra
The role
What you’ll do
What we’re looking for
Nice to have
What success looks like
Ultra builds industrial AI robots that are working in the real world today. Our robots automate mission-critical warehouse labor end-to-end, starting with e-commerce order packaging. We deploy fast, learn from real production data, and iterate aggressively. We already have a quickly growing number of revenue-generating robots operating in U.S. warehouses, and are now scaling towards thousands of deployments over the next few years. Our mission is simple: build the world's most useful and deployable robot.
WARNING: Robotics is not for the faint of heart. There are no silver bullets here. It is tiresome, unglamorous, and at times brutal work. The only way out is through, and the only way through is treacherous and unknown. Success will require teamwork - so we are building a world-class team to tackle some of the most exciting problems out there. We hope you join us.
Data Operations AssociateHigh-quality data is foundational to how we improve our robots. Every day, our systems perform a wide range of industrial tasks, generating large volumes of teleoperation, annotation, and performance data. We are looking for a Data Operations Associate to help ensure that this data is accurate, consistent, and useful for training better robot policies.
This role is split between data quality assurance and pilot operations. Roughly half of your time will be spent reviewing robot task data and annotations, identifying quality issues, providing clear feedback to annotators, and helping improve the guidelines and processes used to produce high-quality datasets. The other half will focus on teleoperation operations: observing pilots as they learn new tasks, evaluating their performance, and providing coaching and feedback that helps them operate robots more safely, consistently, and effectively.
This is a hands-on role for someone who is highly attentive, operationally rigorous, and excited to develop a deep understanding of how robot behavior, human performance, and data quality fit together. You will work closely with our Autonomy and Operations teams, and your work will directly affect the quality of the data used to train and evaluate our models.
- Robot data quality assurance. Review videos and associated annotations from a high mix of industrial robot tasks. Identify missing, inaccurate, or inconsistent annotations and determine whether collected data meets established quality standards.
- Annotation feedback and improvement. Provide clear, structured feedback to internal and external annotation teams. Track recurring sources of error and help ensure that feedback leads to measurable improvements in annotation quality.
- Annotation guidelines. Help design, test, and refine annotation instructions, examples, rubrics, and edge-case guidance so that annotators can make consistent decisions at scale.
- Pilot observation and coaching. Observe teleoperation pilots as they train on new tasks, identify performance gaps, and provide timely, actionable coaching.
- Pilot readiness and evaluation. Help evaluate whether pilots are ready to perform tasks independently. Support the development of task-specific training materials, grading criteria, and certification processes.
- Quality monitoring and reporting. Track data-quality and pilot-performance metrics, investigate unexpected changes, and communicate findings to operations and technical stakeholders.
- Process improvement. Identify inefficient or unreliable parts of the data collection and review process, propose improvements, and help implement systems that maintain quality as operations scale.
- Cross-functional collaboration. Partner with the Autonomy team to understand which mistakes matter most for model training and evaluation, then translate those needs into practical guidance for pilots and annotators.
Experience requirements are flexible. We care more about judgment, attention to detail, communication, and demonstrated ability than a specific number of years.
- Exceptional attention to detail and the ability to maintain consistent judgment while reviewing large volumes of complex information.
- Strong written and verbal communication skills, especially the ability to explain errors clearly and provide direct, constructive feedback.
- Comfort reviewing repetitive work without losing focus, while still recognizing unusual edge cases and broader patterns.
- A systems-oriented mindset: you look beyond individual mistakes to identify why they are happening and how the process can be improved.
- Comfort working with technical tools, structured data, spreadsheets, dashboards, and unfamiliar software systems.
- Good judgment and a willingness to make decisions when guidelines do not perfectly cover the situation.
- An interest in robotics, artificial intelligence, industrial operations, or the role high-quality data plays in improving machine-learning systems.
- A hands-on attitude. You are willing to perform a process yourself, understand it deeply, and help improve it before attempting to automate or delegate it.
- Comfort traveling to Mexico on a semi-regular basis, typically once a month or every other month, to work directly with pilot and data-operations teams.
- Experience in data operations, quality assurance, data annotation, technical operations, robotics operations, manufacturing, logistics, or another detail-oriented operational environment.
- Experience reviewing or producing labeled datasets for machine learning.
- Experience in robotics, teleoperation, manufacturing, warehouse operations, or industrial environments.
- Experience training, coaching, or evaluating operators.
- Experience working with external annotation or operations vendors.
- Familiarity with data-quality metrics, sampling methods, inter-annotator agreement, or quality-control workflows.
- Spanish-language proficiency.
Within your first several months, you will develop a strong understanding of our robot tasks, annotation standards, and pilot workflows. You will reliably identify data-quality issues, provide feedback that improves annotation accuracy, and help pilots develop better task-execution habits.
Over time, you will help us move from resolving individual mistakes to building systems that prevent those mistakes from recurring. Your work will result in more consistent annotations, better-trained pilots, clearer operating guidelines, and higher-quality robot data for our AI teams.
Expected Compensation
$25 - $45/hr
Equal Employment Opportunity Statement
We are an Equal Opportunity Employer. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We are committed to creating a diverse and inclusive environment and encourage applicants from all backgrounds to apply.
Ultra (ultra.tech) New York, New York, USA Office
New York, New York, United States, 11232 2405
Similar Jobs
.jpeg)
Automotive • eCommerce • Food • Internet of Things • Logistics
Own day-to-day lead management for Grubhub for Business: source, enrich, upload, and distribute leads in Salesforce, maintain CRM data quality, monitor database health, provide reporting, and partner with Sales, Marketing, and Revenue Operations to improve processes.
Top Skills:
Chili PiperExcelGongGoogle SheetsSalesforceSalesloftZoominfo
Financial Services
Partner with investment, client service, risk, and tech teams to centralize and automate fund data ingestion, ensure data quality and governance, reconcile portfolio accounting (valuations, capital calls, distributions), support client reporting and analytics, and help build APIs and automation tools to support reporting and regulatory needs.
Top Skills:
AddeparAi ToolsAPIsData WarehouseExcelSnowflakeSQL
AdTech • Marketing Tech
Design, build, and maintain BI/reporting infrastructure and visualizations to support campaign and competitive insights. Collect requirements, manage monthly data provisioning, perform ETL, clean and transform large datasets, assist cloud/data lake setup and database builds, develop Tableau/Power BI dashboards, implement automated QA, and translate technical findings for non-technical stakeholders to optimize media investments.
Top Skills:
Adobe AnalyticsAdwordsAlteryxAWSAzureDatoramaDomoGCPGoogle AnalyticsGoogle Campaign ManagerGoogle Data StudioIbm Data Cloud PackLuminaMarinMediaoceanPower BIPrismaPythonSQLTableau
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
