The role involves designing and implementing data pipelines and systems for large datasets, supporting scientific processes and experimentation.
Hiring for multiple roles.
Traditional forecasting breaks under petabytes of features, megabytes of labels, and time-varying noise. Our breakthrough research is foundational in the field: generating industry-beating results (MIT News). Our predictive engines combine a wide variety of data through rigorous scientific process. We leverage alternative datasets (consumer transactions, geolocation, etc) along with private, public, external, and internal data.
The Data Platform is a critical component in our scientific approach to forecasting. The role will require everything from high-level architecture design to performant implementation.
Responsibilities include:
- Build distributed systems supporting complex transformations over petabyte-scale data
- Design, implement, and maintain production cloud data pipelines for complex, disparate datasets
- Work closely with our Quants and Applied Scientists to support rigorous scientific process, rapid experimentation, and new product initiatives
Required qualifications:
- Bachelor’s or advanced degree in Computer Science, math or equivalent
- 3+ years of experience in data engineering or related fields with a track record of building (coding) and scaling a data platform
- 2+ years of industry experience building ETL pipelines in a major cloud provider
- Highly proficient in python and SQL
- Advanced knowledge of databases, design patterns, OOP, and concurrency
- Knowledge of data management fundamentals and data storage principles
- Passion for intuitive data models
- Experience communicating with senior management as well as colleagues from engineering, analytics, and business backgrounds
- Experience with AWS, orchestration platforms, and modern tech stacks
- Strong software engineering fundamentals: system design, algorithms, data structures
Nice-to-haves:
- Experience building systems that integrate LLMs with large-scale data processing
- Working understanding of basic probability and statistics
- Experience implementing ML models
- Experience with DBT
- MS or PhD in STEM field
In-person at our NYC office strongly preferred. Remote candidates who can spend some time in-office (e.g., monthly) will also be considered.
Benefits:
- The salary range for this position is $145,000-$220,000
- Free health, vision, dental insurance
- 401k with match
- Unlimited PTO + 13 company holidays
- Daily Grubhub lunch stipend
- Pre-tax commuter benefits
Covariance.ai New York, New York, USA Office
New York, New York, United States
Similar Jobs
Artificial Intelligence • Cloud • Security • Software • Cybersecurity
Design, build, and operate lakehouse platform components, drive open table format adoption, and collaborate on data architecture management.
Top Skills:
Apache IcebergSparkGoJavaKubernetesPythonScalaTrino
Artificial Intelligence • Information Technology • Machine Learning • Software
Build and scale the data infrastructure for Actively AI's platform, focusing on pipelines, transformations, and operational data management. Ensure data quality and real-time processing.
Top Skills:
AirflowBigQueryDataflowDbtFivetranKafkaPub/SubPythonSnowflakeSQL
Artificial Intelligence • Greentech • Hardware • Internet of Things • Transportation • Cybersecurity • Automation
The Senior Backend Software Engineer will develop and implement a data platform with LLM-enabled capabilities, optimize data processes, and collaborate with cross-functional teams to meet operational needs.
Top Skills:
AWSAzureCloud SqlGoGoogle Cloud PlatformGrafanaGrpcJavaKotlinMySQLPostgresPrometheusRestful Apis
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
