Top Data Engineer Jobs in NYC, NY

Designs and optimizes scalable ETL pipelines, data models, schemas, stored procedures, and SQL queries across data warehouses and lakes. Integrates and transforms data from multiple platforms while embedding validation and monitoring for quality and integrity. Provides production troubleshooting and weekly on-call support, collaborates with cross-functional teams, and develops reliable data solutions using relational databases, Linux, Git, and AWS services.

Develop and optimize SQL Server and Snowflake data solutions, migrate workloads to the cloud, and build ETL/ELT pipelines using SQL, Python, Airflow, and AWS. Responsibilities include performance tuning, data validation, reconciliation, monitoring, production troubleshooting, and support for transaction cost analysis products using large-scale financial, trade, and market data.

Build and support enterprise data ingestion, transformation, modeling, and integration solutions using Microsoft Fabric and Azure data services. Develop batch, API, file, database, and event-driven pipelines; create dimensional and curated data models; write and optimize SQL; resolve data-quality and pipeline issues; and support production workloads. Collaborate in Agile engineering processes using GitHub, Azure DevOps, CI/CD, code reviews, and reusable development standards.

Build and operate production data pipelines, models, identity resolution, canonical data layers, and GTM data products. Integrate internal and external sources, establish data quality and governance, generate intelligence and AI-derived attributes, and deliver trusted data to CRM systems. Partner with Sales Operations, Finance, Data Science, and engineering teams on definitions, planning, segmentation, routing, observability, security, and platform scalability.

Designs and delivers large-scale, governed data pipelines and infrastructure across the full data lifecycle. Responsibilities include coding in Python and SQL, optimizing Airflow, dbt, AWS, Kafka, and Iceberg/Parquet workflows, implementing data quality and observability, contributing to architecture and tool evaluations, mentoring engineers, collaborating with stakeholders, and participating in on-call support.

Design, build, and operate scalable cloud data platforms, pipelines, and applications. Collaborate with data scientists, analysts, product managers, and software engineers while applying distributed data, lakehouse, streaming, database, and security technologies. Establish reusable engineering patterns, ensure performance and resilience, communicate technical outcomes, mentor team members, and deliver secure data solutions with minimal supervision.

Build and operate production data pipelines from clinical, operational, billing, and product systems into trusted warehouse datasets. Responsibilities include ingestion with dlt and Prefect, data modeling in dbt, AI training and evaluation datasets, pipeline testing and monitoring, lineage and alerting, schema changes, backfills, and secure handling of PHI through access controls, de-identification, and audit trails.

Designs and maintains scalable data pipelines and infrastructure, establishes data governance and quality standards, and drives technical decisions across Headway’s data platform. Partners with Product, Engineering, Analytics, Data Science, Infrastructure, Finance, and Insurance Systems to improve architecture, performance, reliability, security, and AI-readiness. Ensures data is complete, traceable, trustworthy, and suitable for powering internal tools and AI-driven insights.

Design, build, and support scalable cloud data platforms, pipelines, and applications. The role involves distributed data processing, data modeling, relational and NoSQL databases, data security, observability, orchestration, and real-time pipelines. It also requires influencing cross-functional teams, mentoring data professionals, establishing engineering standards, and communicating technical outcomes to stakeholders.

Build and architect scalable cloud data platforms, pipelines, and applications using Python, Spark, SQL, Databricks, Snowflake, and relational or NoSQL databases. Design secure, resilient, and reusable data engineering solutions; support distributed and streaming workloads; enforce engineering standards; collaborate with product, software, analytics, and data science teams; communicate technical concepts; and mentor other engineers.

Build and architect scalable cloud data solutions, pipelines, platforms, and data products. Lead large-scale data initiatives, make architecture and platform decisions, ensure performance and operational efficiency, and apply modern technologies such as Python, Spark, Databricks, and Snowflake. Collaborate with Agile teams, product managers, software engineers, analysts, and data scientists while mentoring engineers and communicating technical concepts to stakeholders.

Design and maintain scalable AWS-based data pipelines, architectures, ETL/ELT workflows, databases, and big data solutions. Integrate and transform complex datasets using Python, SQL, Spark, Hadoop, and Kafka. Build data models, dashboards, reports, and analytics solutions while implementing data quality, governance, security, monitoring, and compliance processes. Support migrations and modernization projects, collaborating with data scientists, analysts, and business stakeholders to deliver reliable, decision-ready data.

Build the company’s foundational data platform, including ingestion pipelines, record matching, reconciliation tools, and a flexible data warehouse. Partner with the CTO to establish the core system of record and automate customer onboarding. Translate complex data concepts for nontechnical stakeholders while shaping product direction in an early-stage environment. Bonus responsibilities include integrating AI model services and supporting business intelligence dashboards and visual reporting.

Develops data acquisition strategies, integrations, migrations, workflows, architectures, and data management procedures. Builds and maintains data pipelines, ensures data quality and security, supports cloud and healthcare claims data environments, and prepares data for analytics, machine learning, and predictive modeling. Collaborates with business and technical stakeholders to define data standards, requirements, monitoring capabilities, and process improvements.

Develop and maintain scalable data pipelines, models, datasets, reports, and dashboards for operational reporting. Use dbt, Airflow, Azure Data Factory, SQL, Power BI, and related technologies to build, optimize, test, migrate, and support data delivery solutions. Collaborate with technical and business stakeholders, conduct root cause analysis, automate maintenance, contribute to development standards, and mentor junior engineers. The role requires P&C insurance experience and hybrid work three days per week at a specified office location.

Leads the strategy, architecture, and delivery of complex data platforms supporting investment research and portfolio construction. Designs Databricks and AWS lakehouse pipelines, data models, governance, testing, observability, and AI-ready metadata. Guides AI-agent engineering workflows, ensures security and reliability, manages cross-team initiatives, and leads technical reviews and engineer development. Partners with investment professionals and product managers to deliver trusted data products and scalable AI capabilities.

The Data Engineer will design and implement ETL frameworks, perform data analysis, troubleshoot data issues, and optimize data models to meet business needs.

Design and modify data solutions using Databricks and Palantir Foundry; build data models and reports from ERP and legacy-system datasets; define and execute tests; and translate stakeholder reporting needs into technical specifications. The role requires SQL, Python, PySpark, database architecture, cloud architecture, software engineering practices, independent execution, documentation, and collaboration. It supports an Army client in a hybrid Arlington, Virginia environment and requires an active Secret clearance.

Maintains and monitors Azure Databricks data pipelines, troubleshoots data issues, collaborates with stakeholders on data requirements, documents processes, performs backups and disaster recovery drills, and supports data access and usage. The role requires extensive data engineering experience, Azure expertise, incident management knowledge, and strong problem-solving skills.

Designs scalable data architectures, ELT/ETL pipelines, data models, quality frameworks, and self-service data platforms. Partners with engineering, product, program, leadership, and data science teams to translate business needs into technical solutions. The role includes end-to-end technical ownership, large-scale batch and streaming data processing, mentoring junior engineers, system optimization, and on-call support for platform stability.

Build and maintain reliable data pipelines, computed tables, and scalable data models using SparkSQL and PySpark. Integrate IoT, product, customer, external, unstructured, and ML-related data into a central data lake for analytics, causal inference, model training, and dashboards. Ensure high data quality and uptime while collaborating with data science, analytics, AI/ML, and engineering teams.

Lead the reverse-engineering and migration of legacy ETL pipelines from SAP BusinessObjects Data Services and JD Edwards into dbt and Databricks architectures. Build automated data validation, maintain SQL Server write-back mechanisms, collaborate with client stakeholders, and lead and mentor a small engineering team. The role requires hands-on expertise in SQL, Python, Spark, orchestration, cloud platforms, and legacy ERP and ETL migrations.

Designs, builds, tests, and maintains scalable data infrastructure and ETL/ELT pipelines for mission partners. Uses AWS Glue, Glue Catalog, and Step Functions to integrate, migrate, transform, and validate data. Develops data models, schemas, queries, and production-ready datasets while collaborating with analytics and business teams. The role may also support AI/ML pipelines, data governance, DevOps workflows, Databricks, and real-time streaming architectures.

Architects, builds, tests, and maintains scalable data infrastructure and ELT/ETL pipelines using AWS Glue, Glue Catalog, and Step Functions. Performs data modeling, SQL querying, data migration, schema optimization, validation, and data quality improvements. Partners with analytics and business teams to transform raw data into production-ready datasets. The role may also support AI/ML pipelines, governance, access controls, DevOps workflows, and real-time data architectures.

Design, build, and operate scalable cloud data platforms, pipelines, and applications. Collaborate with Agile teams, product managers, engineers, analysts, and data scientists while applying Python, Spark, SQL, Databricks, Snowflake, relational and NoSQL databases. Establish reusable engineering patterns, data security controls, and resilient architectures. Mentor data professionals, communicate technical concepts to stakeholders, and deliver user-centric data solutions with minimal supervision.