Easy Apply
Easy Apply
Seeking a Staff Data Engineer to architect scalable data systems, mentor engineers, and enable analytics for AI initiatives using PySpark.
About Us
Wizard is the top-performing AI Shopping Agent, delivering the best products from across the web with unmatched accuracy, quality, and trust.
The RoleWe are seeking a Staff Data Engineer with deep expertise in PySpark to join our data team. This is a hands-on technical role for someone passionate about building scalable data systems, mentoring engineers and shaping data strategy. You’ll architect systems that power high performance batch and real time data processing, enable advanced analytics and accelerate our AI initiatives.
Key Responsibilities- Design and evolve scalable, distributed data infrastructure across cloud platforms
- Build and maintain real time and batch data processing pipelines supporting analytics and AI/ML workloads
- Develop and manage integrations with third party e-commerce platforms to expand our data ecosystem
- Ensure data availability, reliability and quality through monitoring and automated auditing
- Partner with engineering, AI and product teams on data solutions for business critical needs
- Mentor and support data engineers, establishing best practices and code quality standards
- 5+ years of software development and data engineering experience with demonstrated ownership of production grade data infrastructure
- Bachelor's degree in Computer Science or a related field, or equivalent practical experience.
- Deep expertise scaling Spark in production (Databricks, EMR, etc)
- Strong understanding of distributed computing and modern data modeling for scalable systems
- Proficient in Python with experience implementing software engineering best practices
- Hands-on experience with both relational (MySQL / PostgreSQL) and NoSQL (MongoDB, DynamoDB, Cassandra) databases
- Strong communicator with experience influencing cross functional stakeholders
- Experience working in early-stage, high-growth environments
- Familiarity with MLOps pipelines and integrating ML models into data workflows.
- Passionate about problem-solving with a proactive approach to finding innovative solutions.
The expected base salary range for this role is $208,000 – $235,000 USD, and will vary based on skills, experience, role level, and geographic location. Final compensation will be determined by considering these factors alongside overall role scope and responsibilities.
In addition to base salary, Wizard offers:
- Equity in the form of stock options
- Medical, dental, and vision coverage
- 401(k) plan
- Flexible PTO and company holidays
- Fully remote work within the United States
- Periodic company-wide offsites and team gatherings
Wizard is committed to fair, transparent, and competitive compensation practices.
Top Skills
Cassandra
Databricks
DynamoDB
Emr
MongoDB
MySQL
Postgres
Pyspark
Python
Spark
Wizard AI New York, New York, USA Office
New York, New York, United States, 10013
Similar Jobs
Artificial Intelligence • Cloud • Machine Learning • Mobile • Software • Virtual Reality • App development
The Principal Software Engineer will design and scale engineering components, lead projects, advocate best practices, and influence technical direction across teams.
Top Skills:
DebuggingDistributed SystemsPerformance AnalysisSoftware DesignTest Design
Cloud • Enterprise Web • Logistics • Software
The role involves optimizing business processes, managing Salesforce operations, and ensuring alignment across Sales, Customer Success, Marketing, and Finance. Focus on operational efficiency, community management, and reporting.
Top Skills:
SalesforceSalesforce Experience CloudSalesforce Knowledge BaseSalesforce Service Cloud
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
The Senior Business Process Consultant guides customers in implementing ServiceNow Risk solutions, conducting workshops, and ensuring alignment of business requirements with technical configurations.
Top Skills:
PowerPointServicenowVisioWord
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
