Top Senior Data Engineer Jobs in NYC, NY

Build and operate reliable data systems and products supporting clinical operations, drug evaluation, business development, analytics, machine learning, and AI agents. Responsibilities include designing pipelines, canonical data models, data contracts, warehouse transformations, quality and observability practices, governance for regulated data, and incident response. The role partners closely with Product Engineering and Data Science, uses AI-assisted development, contributes to architecture, and mentors other engineers.

Own and scale the AI research data platform, including ingestion, processing, governance, versioning, lineage, quality, observability, and export for multimodal datasets. Build Databricks and distributed processing infrastructure, synthetic data generation pipelines, and self-service tools for ML researchers. Partner with security and compliance teams, mentor engineers, manage infrastructure automation, and establish the long-term platform roadmap.

Build and optimize scalable data architectures, pipelines, warehouse tables, and data feeds from diverse sources. Ensure data quality, monitoring, CI/CD reliability, security, and efficient compute and storage usage. Collaborate with engineering, product, and cross-functional teams on data analysis, AI, machine learning, experimentation, and ad technology products. Develop automated data integration and cleansing processes, troubleshoot pipeline failures, and contribute to complex projects involving cloud platforms, MLOps, and large datasets.

Design, build, deploy, and maintain AWS-based data and AI/ML infrastructure for enterprise agentic AI platforms. Develop data lake pipelines, observability systems, agent-enablement tooling, and production MLOps/LLMOps capabilities. Own features end-to-end, contribute to architecture and design decisions, optimize reliability, scalability, security, and cost, and collaborate with AWS Professional Services, client teams, and junior engineers.

Build and operate production pipelines for legal records from PACER, court systems, opinions, PDFs, scans, XML, and HTML. Develop LLM-assisted extraction workflows, validate data quality, optimize SQL, and support RAG and semantic search products. Manage AWS S3 and RDS systems within security guardrails while collaborating with engineering teams. The role requires strong Python and SQL skills, legal data experience, document extraction expertise, and autonomous problem-solving with large, messy datasets.

Own datasets for attribution, campaign performance, and customer lifecycle analysis. Design real-time and batch data pipelines, build scalable analytics infrastructure, develop attribution frameworks, optimize Oracle databases and SQL data models, and improve data governance. Partner with analytics and product teams to translate business needs into data solutions.

Build and lead the telemetry and analytics data platform, including collection, transformation, aggregation, data modeling, dashboards, and self-service tooling. Process OpenTelemetry traces, metrics, and logs at scale; analyze system performance, stability, and feature usage; ensure data quality and access; and partner with frontend and backend teams on instrumentation standards. Manage two data engineers, provide technical direction, document tooling, support users, and communicate findings through visualization.

Build and support production-grade Microsoft Fabric data products and pipelines across ingestion, transformation, lakehouse, warehouse, and curated layers. Implement reliable, restartable, and observable data solutions using medallion architecture, automated testing, governance, lineage, monitoring, CI/CD, and DataOps practices. Collaborate with architects, analytics, governance, application, and partner teams while documenting mappings, runbooks, controls, and operational procedures.

Lead enterprise data engineering efforts supporting defense missions. Responsibilities include prioritizing technical tasks, mentoring engineers, conducting code reviews, translating government requirements into scalable data architectures, developing Databricks ETL/ELT pipelines with Python, PySpark, and SQL, troubleshooting pipeline performance and reliability, and documenting systems. The role collaborates with government stakeholders, product owners, data scientists, analysts, and technical teams to deliver secure, high-throughput data platforms and actionable insights.

Designs and maintains scalable Azure data pipelines and ETL/ELT solutions across data lakes, warehouses, lakehouses, streaming platforms, and enterprise sources. Builds integrations using Azure Data Factory, Synapse, Databricks, Spark, Kafka, and related services; applies security, governance, monitoring, and performance optimization practices. Develops Power BI dashboards, reports, data models, and visualizations, while supporting analytics and AI/ML use cases. Collaborates in Agile teams and automates deployments through DevOps and CI/CD practices.

Designs, builds, documents, tests, and maintains large-scale data pipelines across platforms and data sources. Performs ETL and ELT, develops data quality and governance processes, and supports data security and compliance. Collaborates with architects, analysts, and stakeholders to translate business needs into technical solutions. The role also provides mentorship to junior engineers and leads data collection, storage, and process automation initiatives.

Designs and operates data infrastructure for a clinical trial platform. Builds pipelines that extract, normalize, validate, and link information from clinical documents and publications into Aurora and GraphDB. Develops embeddings, vector search, RAG workflows, data-serving APIs, lineage tracking, and regulatory audit trails. The role also requires AWS-based orchestration, clinical data quality controls, biomedical knowledge graph modeling, and deployment automation using infrastructure-as-code and containerization.

Designs and delivers backend systems, APIs, data models, ETL workflows, data interactions, and user interfaces. Optimizes data operations for scalability, performance, integrity, and security while contributing to automated testing and CI/CD. Requires strong Python and Apache Spark experience, distributed computing knowledge, Agile experience, and data analysis skills. The role supports healthcare-focused digital services in a remote-first environment.

Lead enterprise data strategy and delivery across data platforms, master data management, semantic layers, governance, quality, modeling, and pipelines. The role requires hands-on architecture leadership, executive stakeholder engagement, and delivery of complex multi-region platforms using Snowflake, cloud infrastructure, MDM, Collibra, dbt, and batch and streaming architectures.

Architect and lead enterprise data platforms, pipelines, and infrastructure using Spark, Databricks, and cloud technologies. Build scalable data models and dimensional schemas, establish governance and data quality frameworks, optimize performance and reliability, and implement monitoring. Partner with stakeholders to translate requirements into technical solutions, lead design reviews, mentor engineers, contribute to platform strategy, troubleshoot complex failures, and support hiring and team development.

Leads the architecture, development, and maintenance of scalable batch and real-time data pipelines, cloud data platforms, ETL/ELT workflows, and ML data infrastructure. Establishes standards for data quality, observability, lineage, governance, security, and modeling. Partners with data scientists, engineers, DevOps, and business stakeholders; evaluates technologies, drives technical initiatives, mentors engineers, and supports production-grade APIs and machine learning pipelines.

Designs, builds, maintains, and optimizes scalable data pipelines and ETL/ELT workflows using Azure Databricks, SAS, SSIS, SQL, Python, and Spark. Integrates data from internal and external sources, improves performance and reliability, troubleshoots complex processing issues, and supports cloud-based data platform architecture. Collaborates with engineering, analytics, product management, SRE, and other stakeholders to deliver high-quality data solutions.

Own end-to-end production data platforms, writing Python and SQL, designing AWS pipelines, modeling analytics data, and implementing quality and reconciliation controls. Make architecture decisions independently, ensure correctness and auditability, and use AI coding tools effectively. The role also supports applied AI initiatives including entity resolution, LLM-assisted analytics, embeddings, and anomaly scoring.

Owns the end-to-end data platform, including Snowflake infrastructure, ingestion pipelines, Airflow orchestration, dbt transformations, governed gold-layer models, and metric definitions. Ensures reliability, performance, testing, documentation, governance, and CI/CD standards. Builds data systems from scratch, partners with analytics stakeholders, and uses AI-assisted development tools to accelerate engineering work. Requires strong SQL, platform ownership, software engineering practices, and experience with Snowflake, Airflow, and dbt.

Build and optimize data transformation pipelines using SQL, dbt, Airflow, Python, BigQuery, and relational databases. Collaborate with stakeholders and data engineers, manage task scheduling, resolve data issues, perform analyses, present recommendations, support data exploration tool adoption, and mentor team members. The role also contributes to hiring, onboarding, and professional development while ensuring data accuracy and insight quality.

Build and operate large-scale audio data pipelines for AI model training. Responsibilities include discovering data sources, improving data-processing systems for quality, throughput, and cost, collaborating with AI scientists, and owning engineering projects through continuous improvement.

Build and operate large-scale audio data pipelines for AI model training. Responsibilities include discovering data sources, developing reliable processing systems, optimizing data quality, throughput, and cost, and collaborating with AI scientists. The role requires ownership of engineering projects from discovery through continuous improvement in a distributed, ambiguous environment.

Build and operate large-scale data pipelines that acquire, process, and improve audio datasets for AI model training. The role focuses on data quality, throughput, cost optimization, production systems, and collaboration with AI scientists. Responsibilities include discovering data sources, developing reliable processing systems, and owning engineering projects from discovery through continuous improvement.

Build and operate large-scale audio data pipelines for AI model training. Responsibilities include discovering data sources, developing reliable data-processing systems, optimizing data quality, throughput, and cost, and collaborating with AI scientists. The role requires ownership of engineering projects from discovery through continuous improvement in a distributed, autonomous environment.

Build and operate large-scale audio data pipelines for AI model training. Responsibilities include discovering data sources, developing reliable data acquisition and processing systems, optimizing quality, throughput, and cost, and collaborating with AI scientists. The role requires ownership of engineering projects from discovery through continuous improvement, with emphasis on backend development, production systems, and scalable data processing.