Operate and stress-test LLM-based AI tools, reproduce and document edge cases, file and track tickets, validate fixes, provide prioritized product feedback, triage failures with engineers, design lightweight tests, and support the Head of Engineering with task and calendar management to scale experimentation and improve production reliability.
About the Client
The hiring manager is the Head of Engineering, who partners closely with product team and iterate quickly on emerging LLM workflows. The culture values autonomy, candor, and high standards in a startup-paced environment.
The hiring manager is the Head of Engineering, who partners closely with product team and iterate quickly on emerging LLM workflows. The culture values autonomy, candor, and high standards in a startup-paced environment.
Why does this role exist?
This role exists to serve as a highly autonomous right hand to the Head of Engineering, operating and stress-testing experimental AI tools while closing the feedback loop between users and the engineering/product teams. By proactively finding breakages, documenting issues, and prioritizing work across a 30+ person team, this role accelerates engineering velocity and product quality. It ensures the organization can scale experimentation, reduce failures in production, and maintain tight operational rhythms without relying on rigid SOPs.
The Impact you’ll make
AI Tool Operations & Testing
- Operate, stress-test, and quality-check cutting-edge AI tools and LLM-based systems that may frequently break
- Reproduce edge cases, capture logs/screens, and write clear repro steps; file/track tickets in Shortcut (Clubhouse)
- Validate fixes and run smoke/regression tests before and after deployments
Product Feedback & Collaboration
- Provide structured, actionable feedback to the Head of Engineering and product/engineering teams
- Collect and synthesize feedback from other assistants/users; turn insights into prioritized improvements
- Critique internal software and propose data-backed enhancements to workflows and UX
Technical Troubleshooting & QA
- Triage broken workflows, diagnose likely failure points, and collaborate with engineers on root causes
- Consolidate deployment updates and confirm changes behave as expected in production
- Design lightweight test plans for ambiguous features; track outcomes and follow through to resolution
Operational Support to the Head of Engineering
- Manage and organize the Head of Engineering’s to-do list; drive proactive follow-through across a 30+ person team
- Maintain and optimize work queues; ensure the right priorities are surfaced at the right time
- Handle calendar management via AI tools; coordinate with stakeholders as needed
Systems & Process Optimization
- Build simple, scalable processes in ambiguous environments; streamline task flows and handoffs
- Propose and implement improvements to reduce repeated failure modes and increase throughput
- Leverage AI productivity tools to automate routine steps and create operational leverage
Skills, Knowledge and Expertise
Required:
- Demonstrated technical mindset with strong AI tool fluency (e.g., ChatGPT, LLM workflows) or a programming background
- Proven experience in product operations, QA/Testing, technical support, or similar role working closely with engineers
- Exceptional written communication for clear, structured bug reports and product feedback
- Comfortable operating independently in ambiguous, fast-paced environments with minimal SOPs
- Availability to work 40 hours/week, Monday–Friday, 9:00 am–5:00 pm Eastern Time (may shift slightly earlier after 3–4 weeks)
WFH Set-Up:
- Computer with at least 8GB RAM, an Intel i5 core processor/AMD Ryzen 5 Processor and up.
- Internet speed of at least 40MBPS
- Headset with an extended mic that has noise cancellation and a webcam
- Back-up computer and internet connection
- Quiet, dedicated workspace at home
Your Superpowers:
- Technical: AI tool proficiency; systems thinking; test design and execution; QA methodologies; debugging mindset; clear documentation; comfort with Shortcut (Clubhouse), GSuite, Slack, and experimental internal tools (e.g., Magic AI); familiarity with messaging tools (e.g., Messenger). Bonus: basic scripting/automation, product analytics literacy.
- Operational: Task prioritization; work queue management; lightweight process design; deployment update consolidation; calendar management via AI.
- Behavioral: Proactive ownership; “figure it out” mentality; high attention to detail; resilience when tools break; curiosity and growth mindset; comfort giving candid, constructive feedback; ability to work independently and collaboratively with engineers.
You should apply if...
- You love being an early power user of new AI tools, enjoy breaking things to make them better, and can translate ambiguous symptoms into actionable engineering feedback.
- You thrive in uncertainty, don’t need handholding, and naturally create order and leverage through systems and process improvements.
- You communicate crisply in writing, challenge assumptions respectfully, and partner well with engineers and product managers.
- You want to grow at the intersection of AI operations, product testing, and systems thinking. Bonus: You’re in a LATAM timezone for strong overlap with EST, and/or you have experience in programming, product, or data analytics.
What to expect...
Work Setup:
- Remote position
- Must have a reliable internet connection and a quiet workspace
- Required to provide own computer with Intel Core i5 or something similar or higher operating system
Working Hours:
- 40 hours per week
- M-F 9am to 12 noon, 1 pm to 4 pm US Timezone
Compensation:
- $7 per hour
- No benefits package included
Benefits
About
Magic has connected top remote talent with fast-growing businesses for over 10 years.Founded in San Francisco in 2015, we now have thousands of remote workers around the world. Magic is backed by Sequoia Capital and Y Combinator.
Similar Jobs
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Lead the CFC CRM product for HCP/HCO-facing colleagues, define roadmap, translate stakeholder needs into prioritized enhancements, write requirements and user stories, partner with engineering, UX, vendors and compliance to deliver scalable global CRM solutions and measure business impact.
Top Skills:
Crm PlatformsLife Sciences CloudOceSalesforceVeeva
Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Lead large Oracle Field Service implementation projects as a Solutions Architect senior manager. Design and deploy Oracle Fusion Service and Field Service Cloud solutions, guide assessment and future-state planning, interact with senior clients, coach teams, apply delivery methodologies and accelerators, and drive continuous improvement.
Top Skills:
Oracle Customer ExperienceOracle EpmOracle Field Service CloudOracle FinOracle Fusion ServiceOracle HcmOracle Lead ManagementOracle Marketing AutomationOracle Sales AutomationOracle Scm
Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Lead end-to-end product lifecycle for internal and commercial software products, drive product strategy and commercialization, collaborate with cross-functional teams, apply Agile methods, coach teams, validate client outcomes, and execute roadmaps and go-to-market plans.
Top Skills:
Agile MethodologyProduct Lifecycle Management
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
