Wizard is the top-performing AI Shopping Agent, delivering the best products from across the web with unmatched accuracy, quality, and trust.
The RoleWe’re hiring a Senior Director, Data Platform & Engineering to own the design, evolution and reliability of Wizard’s data platform. Data is foundational to everything we do from ingestion and compilation to personalization and ranking. This is a senior technical leadership role reporting directly to the CTO. This is not purely a strategic role and requires meaningful technical depth. You will be hands-on in architecture and code, setting standards through engineering judgement, not just oversight.
What You’ll Do- Own Wizard’s end-to-end Spark-based data platform, including ingestion, transformation, compilation, quality enforcement, reliability standards, and long-term architectural evolution, ensuring our infrastructure scales to power AI-driven personalization and ranking systems
- Serve as the architectural authority for distributed data systems, setting and enforcing non-negotiable standards for correctness, performance, cost efficiency, and production grade engineering discipline
- Architect, review, and occasionally contribute to production PySpark pipelines (~20% hands-on), bringing recent, real engineering experience from the last 3–5 years.
- Develop proof of concepts and experimental data architectures when exploring new approaches, demonstrating feasibility through hands on builds before operationalizing and handing off to the team
- Incorporate AI assisted coding and modern AI tooling into the development workflow, both personally and across the team to accelerate experimentation, improve engineering velocity and continuously evolve platform capabilities
- Balance near term delivery pressure with long term platform health ensuring speed does not compromise data integrity and model performance
- Oversee data ingestion, normalization, and enrichment of crawled, API and other feed-based datasets to power our AI Search
- Reinvent data operations using the latest AI tooling
- Data pipelines are highly scalable and cost efficient
- Data platform reliability and quality standards are clearly defined and consistently enforced
- Engineering teams rely on your judgement
- Data infrastructure supports AI projects at scale
- Hands on technical role with Pyspark experience within the last 3 years
- 12+ years of experience in data engineering
- Proven experience operating production data platforms at scale in AWS and GCP
- Comfortable contributing technically while leading at a senior level
- Experience with AI/ML workloads
- Experience building data quality and observability frameworks
The expected base salary range for this role is $260,000 – $285,000 USD, and will vary based on skills, experience, role level, and geographic location. Final compensation will be determined by considering these factors alongside overall role scope and responsibilities.
In addition to base salary, Wizard offers:
- Equity in the form of stock options
- Medical, dental, and vision coverage
- 401(k) plan
- Flexible PTO and company holidays
- Fully remote work within the United States
- Periodic company-wide offsites and team gatherings
Wizard is committed to fair, transparent, and competitive compensation practices.
Top Skills
Wizard AI New York, New York, USA Office
New York, New York, United States, 10013
Similar Jobs
What you need to know about the NYC Tech Scene
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory



