Designs, develops, and supports enterprise-scale ETL/ELT pipelines and data warehouse solutions using Ab Initio, Snowflake, Azure, SQL, Python, and shell scripting. Responsibilities include metadata and lineage management, dimensional and Data Vault modeling, data quality validation, CI/CD, production troubleshooting, and batch, real-time, file-based, and API integrations. The role supports healthcare data domains such as claims, membership, providers, health plans, and benefits.
Senior Data Engineer – Ab Initio / Snowflake
Description:
We are looking for a Senior Data Engineer with strong experience in Ab Initio and Snowflake to design, develop, and support enterprise-scale data integration and data warehouse solutions.
You will work within a large-scale Azure data environment, supporting high-volume ETL/ELT processes, complex data pipelines, metadata management, and cloud modernization initiatives. The role is ideal for a hands-on technical expert comfortable working with complex data ecosystems and hundreds of Ab Initio processes.
Responsibilities:
- Design, develop, and support high-volume ETL/ELT data integration solutions using Ab Initio and Snowflake.
- Develop and maintain Ab Initio graphs, Plans, PSETS, Continuous Flows, and batch processing frameworks.
- Build reusable ETL components, transformations, DML files, XFRs, and PDL scripts.
- Manage and support Ab Initio EME / Metadata Hub, including metadata, lineage, impact analysis, and dataset mapping.
- Develop and optimize Snowflake solutions, including Snowpipe, Streams, Tasks, Time Travel, and Zero-Copy Cloning.
- Perform advanced SQL development, query optimization, data reconciliation, and data quality validation.
- Design and maintain data warehouse solutions using dimensional modeling, Star/Snowflake schemas, and Data Vault concepts.
- Develop automation and operational support tools using Python and Shell scripting (KSH/Bash).
- Support batch, real-time, file-based, and API-driven data integrations.
- Participate in CI/CD, Git-based development, code promotion, branching, and release management processes.
- Troubleshoot production issues and ensure the reliability, scalability, and performance of data pipelines.
- Collaborate with technical and business teams to translate data requirements into scalable solutions.
- Work with healthcare data domains, including claims, membership, providers, health plans, benefits, and employer/group data.
Your Qualifications:
- 6+ years of hands-on experience with Ab Initio development.
- 4+ years of experience with Snowflake development and architecture.
- Advanced SQL skills, including complex joins, subqueries, stored procedures, dynamic SQL, and query tuning.
- Strong experience with Ab Initio EME / Metadata Hub, version control, sandbox management, check-in/check-out, and tag promotion.
- Experience with data warehouse architecture, dimensional modeling, Star/Snowflake schemas, Data Vault, MDM concepts, and source-to-target mapping.
- Proficiency in Python and Shell scripting (KSH/Bash).
- Strong understanding of ETL/ELT patterns, batch and real-time processing, API integrations, and large-scale data movement.
- Experience with DevOps and CI/CD, including Git, branching strategies, automated deployments, and release management.
- Experience working with Azure-based data environments.
- Knowledge of healthcare data, particularly:
- Claims: Professional, institutional, medical, pharmacy, encounter, and adjudication data.
- Membership: Enrollment, eligibility, coverage, benefits, population health, and member attribution.
- Provider: Directories, networks, contracting, NPI, attribution, and provider hierarchies.
- Health Plans: Benefits, products, groups, and employer data.
- Strong analytical, troubleshooting, and problem-solving skills.
- Ability to work independently in a complex enterprise data environment and collaborate effectively with cross-functional teams.
Metova Talent is interested in every qualified candidate who is eligible to work and reside in the United States. However, we will not consider applicants who are not U.S. citizens, U.S. nationals, lawful permanent residents, asylees, or refugees.
Metova Talent is an Affirmative Action and Equal Opportunity Employer.
*Notice to Third Party Recruitment Agencies:
Please note that Metova Talent does not accept unsolicited resumes from recruiters or employment agencies. In the absence of an executed Recruitment Services Agreement, there will be no obligation to any referral compensation or recruiter fee.
In the event a recruiter or agency submits a resume or candidate without an agreement, Metova Talent shall explicitly reserve the right to pursue and hire those candidate(s) without any financial obligation to the recruiter or agency. Any unsolicited resumes, including those submitted to hiring managers, shall be deemed the property of Metova Talent.
Similar Jobs
Fintech • Software • Financial Services
Own data pipelines into and out of the warehouse, dbt models, data integrity, governance, identity resolution, reverse ETL, and quality monitoring across growth, lifecycle, and brokerage data. Partner with product, marketing, and operations teams to establish data contracts, ensure accurate and timely business metrics, and support regulated financial data requirements.
Top Skills:
AirflowArgo WorkflowsChange Data CaptureDatadogDbtFivetranKubernetesPythonReverse EtlSegmentSnowflakeSQL
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Lead product data engineering across architecture, data modeling, pipelines, monitoring, lineage, and data quality. Build reliable analytical datasets, support experimentation and machine learning use cases, and resolve business-critical pipeline issues. Partner with product and engineering stakeholders to translate business needs into automated data solutions. The role requires ownership of end-to-end data engineering projects, on-call participation, and development of AI-assisted monitoring and automation.
Top Skills:
AirflowDatabricksDbtGitOmniPrefectPythonSnowflakeSQLTerraform
Blockchain • Fintech • Mobile • Payments • Software • Financial Services
Lead product data engineering initiatives by designing data models, owning critical pipelines, building data quality and lineage monitoring, and supporting experimentation and ML-ready datasets. Translate business needs into automated data solutions, maintain pipeline SLAs through on-call support, and develop AI-assisted tools for issue detection and resolution. Collaborate with Product, Engineering, Data Science, Machine Learning, and Applied AI teams across Block’s products.
Top Skills:
AirflowDatabricksDbtGitOmniPrefectPythonSnowflakeSQLTerraform
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
