SDET focused on data reconciliation, complex SQL, and ETL testing. Build and automate test frameworks, validate ETL pipelines and data warehouses, write optimized SQL for data validation, perform root-cause analysis of discrepancies, and ensure data integrity between source and target systems.
Job Summary:
Nice to have:
We are seeking a highly skilled and detail-oriented Software Development Engineer in Test (SDET) to join our team with a focus on data reconciliation, complex SQL, and ETL testing. The ideal candidate will have a unique blend of skills in software development, quality assurance, and data engineering. You will be responsible for ensuring data accuracy, validating ETL processes, and building automation frameworks for data-related testing processes. This is a key role to ensure the integrity and quality of our data infrastructure and analytics platforms.
Key Responsibilities:- Complex SQL Queries:
- Write and optimize complex SQL queries to validate data pipelines, transformations, and reporting accuracy.
- Perform root cause analysis of data discrepancies and provide insights into data quality issues.
- Develop automated SQL scripts to streamline data validation and reconciliation processes.
- ETL Testing:
- Validate ETL processes by designing comprehensive test strategies to ensure data accuracy and completeness.
- Test and verify ETL pipelines from various data sources (structured and unstructured) into data warehouses, ensuring data conforms to business rules and logic.
- Implement automated ETL testing frameworks and continuously improve them to optimize the data validation process.
- Data Reconciliation:
- Develop, implement, and automate test scripts to validate data integrity between source and target systems.
- Identify and resolve data inconsistencies, mismatches, and errors in large datasets.
- Design and execute reconciliation processes to ensure accurate data migration and transformation.
Nice to have:
- Experience with HiveQL for Hive Database
- Experience with Python as a scripting language
Photon New York, New York, USA Office
New York, United States
Similar Jobs
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Lead SnackFutures Ventures investments in early and growth-stage companies, manage deal sourcing, screening, negotiation, and portfolio oversight, coordinate cross-functional teams, contribute to investment strategy, and support exits or handovers to Corporate Development.
.png)
eCommerce • Fintech • Hardware • Payments • Software • Financial Services
Provide white-glove technical support and onboarding for high-value resellers: troubleshoot integrations, manage escalations, run onboarding and training, track and drive issue resolution with engineering, and collaborate cross-functionally to improve processes and product experience for enterprise sellers.
Top Skills:
APIsGoogle MeetsJIRASdksThird-Party Integrations
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Lead AI FinOps governance to track and optimize AI spend across multi-cloud providers. Build KPI frameworks, anomaly detection, guardrails, and cost-reduction programs. Coordinate engineering, finance, and cloud vendors, deliver VP-level reporting, and operate as an independent program owner driving measurable savings and contractual correctness.
Top Skills:
AnthropicAWSAws BedrockAws BudgetsAzureAzure Cost ManagementAzure OpenaiBilling ApisGCPGcp Billing ControlsGcp Vertex AiGenai GatewayLlm Proxy PlatformsOpenaiPythonSQLToken Metering
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
