Deliberate Software Ltd Logo

Deliberate Software Ltd

Lead Data Engineer

Posted 6 Days Ago
In-Office
New York, NY, USA
160K-220K Annually
Mid level
In-Office
New York, NY, USA
160K-220K Annually
Mid level
Lead the design and implementation of a robust data architecture for clinical datasets, focusing on real-time sensor data processing and API integrations with wearables.
The summary above was generated by AI
Lead Data Engineer

Deliberate AI | Hybrid (NYC or Boston) | Full-Time

About Deliberate AI

We're a venture-backed company at the frontier of precision mental health. In partnerships with some of the world's top ranked medical schools and psychiatric hospitals, we've secured non-dilutive funding from the NIH, ARPA-H, DARPA, the FDA and the Wellcome Trust. We're deploying multimodal AI systems in clinical trials and healthcare settings across four continents — and we're hiring the engineering team to build what comes next.

About the Role

A patient wears an Oura ring to sleep. Their phone picks up a shift in activity patterns overnight. The next morning, a conversational AI agent conducts a brief voice-based check-in — and the vocal features, facial action units, and linguistic markers from that session all flow into the same clinical picture alongside the wearable data. Your job is to make sure every one of those signals — from raw sensor stream to clinically meaningful feature — arrives reliably, on time, and at quality.

You'll architect and own the data infrastructure across all clinical data modalities: audio-visual features from conversational assessments, wearable biometrics, passive mobile sensing, and the feature pipelines that prepare them for fusion in our multimodal ML models. You'll also own the overall data architecture — how data flows into and through Deliberate AI, how it's stored, cataloged, and governed, and how it scales as we deploy across clinical trial sites on four continents. This isn't just a pipeline-building role; it's defining the technical strategy for how clinical data works at a company building the future of precision mental health care.

Who You Are

You've built multi-modal data infrastructure before and you know what it takes to make it reliable at scale. You get equally excited about audio signal processing and wearable API integrations — and you understand that both feed into the same clinical picture. You care about signal quality and data integrity because you know that downstream, a missed feature or a noisy pipeline affects whether a patient's symptoms get caught early or missed entirely. You're serious about privacy, opinionated about architecture, and you want to define the technical strategy for clinical data engineering — not just execute someone else's.

What You'll Do
  • Design and implement the overall data architecture for ingestion, storage, cataloging, and governance of all clinical datasets — audio, video, wearable, mobile sensing, and physiological data from clinical sites worldwide
  • Build and maintain API integrations with commercial wearable devices (e.g. Oura Ring, Fitbit) to collect raw sensor streams (HRV, sleep stages, activity, heart rate) and engineer biometric features
  • Develop systems to capture and process passive mobile signals that trigger adaptive assessments, including real-time streaming and synchronization across modalities
  • Build automated QA systems to detect missing data, sensor failures, and anomalous readings — with data lineage tracking, pipeline observability, monitoring, alerting, and incident triage so problems are caught and resolved before they affect downstream models or clinical decisions
  • Design participant monitoring systems with automated data completeness checks, device health monitoring, and alert mechanisms supporting global deployment
  • Implement reliable incremental load patterns — idempotent runs, backfill strategies, and late-arriving data handling — so the platform stays correct as clinical sites come online across time zones and connectivity conditions
  • Evaluate and select the core data stack — orchestration, warehousing, transformation, and observability tooling — and own those decisions as the foundation the team builds on

Growth trajectory: As Deliberate scales, you'll define the data platform that an expanding team of data engineers builds on — shaping the architecture, standards, and tooling for clinical data engineering in precision mental health.

You may be a good fit if you:
  • Have significant experience in data engineering, including hands-on work in at least two of: audio/video processing, IoT/wearables, or mobile sensing
  • Have expert-level programming skills in Python with experience in performance optimization
  • Have a proven track record architecting and scaling data pipelines for multimedia or sensor data
  • Have deep experience with wearable device APIs (e.g. Fitbit, Oura, Apple Health)
  • Bring strong expertise in time-series data processing, real-time streaming architectures, and feature engineering
  • Have experience with cloud infrastructure (GCP / AWS) and distributed computing
  • Use agentic programming tools (e.g., Claude Code, Codex) as part of your workflow
  • Have a strong understanding of signal processing fundamentals across multiple modalities
  • Hold a Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience; Master's preferred)
Strong candidates may also:
  • Have experience with healthcare or clinical research data (HIPAA compliance, PHI handling)
  • Have knowledge of affective computing, or speech processing
  • Bring background in real-time streaming architectures (Kafka, Pub/Sub, WebSockets) and distributed computing frameworks (Spark, Dask)
  • Have experience with machine learning for audio, video, or sensor applications
  • Have publications or open-source contributions in data engineering or digital health
Compensation & Benefits
  • Base Salary: $160,000 – $220,000 (commensurate with experience, qualifications, and location)
  • Early-stage equity with meaningful ownership — you're joining at a stage where individual grants are substantial
  • Comprehensive health, dental, and vision insurance
  • 401(k) with company match
  • Flexible PTO policy
  • Publication co-authorship on peer-reviewed clinical research — your data architecture shows up in the scientific record, not just the git log

Location: This is a hybrid role. We work in-person roughly 50% of the time in NYC or Boston — this is how we build culture and solve hard problems together as an early, fast-growing team. Candidates should be based in or willing to relocate to one of these cities.

Work authorization: Candidates must be authorized to work in the United States. We welcome applicants who hold US citizenship, permanent residency, or existing work authorization including H-1B (transfer-eligible), OPT/STEM OPT, or TN visa (Canadian and Mexican citizens). If you already hold an H-1B, we will sponsor your green card if desired but we are not currently able to sponsor new H-1B petitions.

Deliberate AI evaluates candidates based on merit, qualifications, and the skills needed to succeed in the role.



Similar Jobs

12 Days Ago
Hybrid
New York, NY, USA
251K-286K Annually
Senior level
251K-286K Annually
Senior level
Fintech • Machine Learning • Payments • Software • Financial Services
As a Senior Lead Data Engineer at Capital One, you'll lead a development team, utilizing various big data and cloud technologies to deliver impactful solutions. Responsibilities include collaborating with Agile teams, mentoring, and ensuring optimal code performance.
Top Skills: AWSEmrGCPGurobiHadoopHiveJavaKafkaMapreduceAzureMySQLNoSQLOpen Source RdbmsPythonRedshiftScalaSnowflakeSparkUnix/Linux
13 Days Ago
Hybrid
New York, NY, USA
251K-286K Annually
Senior level
251K-286K Annually
Senior level
Fintech • Machine Learning • Payments • Software • Financial Services
The Senior Lead Data Engineer will design, develop, and support technical solutions, lead a team, and work with cloud-based services, focusing on data integration and transformation.
Top Skills: AWSEmrGCPGurobiHadoopHiveJavaKafkaMapreduceAzureMySQLNosql DatabasesOpen Source RdbmsPythonRedshiftScalaSnowflakeSparkUnix/Linux
2 Days Ago
In-Office
New York, NY, USA
100K-200K Annually
Senior level
100K-200K Annually
Senior level
Blockchain • Fintech • Financial Services • Cryptocurrency • Quantitative Trading
Lead the technical direction for data platforms, develop high-throughput data processing systems, mentor engineers, and ensure production reliability.
Top Skills: AirflowAWSCi/CdDatabricksDbtJavaPostgresPythonSparkSQL Server

What you need to know about the NYC Tech Scene

As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.

Key Facts About NYC Tech

  • Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
  • Key Industries: Artificial intelligence, Fintech
  • Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
  • Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account