Responsible for maintaining and enhancing data warehouses and pipelines, developing scalable ETL processes, data analysis, and reporting through visualization tools like PowerBI.
Job Overview
We are seeking a skilled Data Engineer to join our team and drive our data infrastructure forward. In this role, you will primarily focus on maintaining and enhancing our data warehouse and pipelines (80%) while also contributing to data analysis and reporting initiatives (20%). You'll work closely with cross-functional stakeholders to build robust data solutions and create actionable insights through compelling visualizations.
Key Responsibilities
Data Engineering
- Infrastructure Management: Maintain, enhance, and optimize existing data warehouse architecture and ETL pipelines.
- Pipeline Development: Design and implement scalable ETL/ELT processes ensuring data quality, integrity, and timeliness.
- Performance Optimization: Monitor and improve pipeline performance, troubleshoot issues, and implement best practices.
- Documentation: Create and maintain comprehensive documentation for data engineering processes, architecture, and configurations.
Data Analysis & Reporting
- Stakeholder Collaboration: Partner with business teams to gather requirements and translate them into technical solutions.
- Report Development: Build and maintain PowerBI dashboards and reports that drive business decisions.
- Data Modeling: Develop new data models and enhance existing ones to support advanced analytics.
- Insight Communication: Transform complex data findings into clear, actionable insights for various departments.
Required Qualifications
Technical Skills
- Programming & Query Languages: Strong proficiency in Python, SQL, and PySpark.
- Big Data Platforms: Experience with cloud data platforms including Snowflake, BigQuery, and Databricks. Databricks experience highly preferred.
- Orchestration Tools: Proven experience with workflow orchestration tools (Airflow preferred).
- Cloud Platforms: Experience with AWS (preferred), Azure, or Google Cloud Platform.
- Data Visualization: Proficiency in PowerBI (preferred) or Tableau.
- Database Systems: Familiarity with relational database management systems (RDBMS).
Development Practices
- Version Control: Proficient with Git for code management and collaboration.
- CI/CD: Hands-on experience implementing and maintaining continuous integration/deployment pipelines.
- Documentation: Strong ability to create clear technical documentation.
Experience & Communication
- Professional Experience: 3+ years in data engineering or closely related roles.
- Language Requirements: Fluent English communication skills for effective collaboration with U.S. based team members.
- Pipeline Expertise: Demonstrated experience building and maintaining production data pipelinesk
Top Skills
AWS
Azure
BigQuery
Databricks
Git
Google Cloud Platform
Power BI
Pyspark
Python
Snowflake
SQL
Similar Jobs
.png){kind=logo}
AdTech • Cloud • Digital Media • Information Technology • News + Entertainment • App development
Lead engineering and automation for a data collaboration ecosystem: design secure, scalable Snowflake/Databricks clean room architectures, build ELT pipelines (Snowpark/PySpark), implement MLOps and AI/LLM integrations, enforce RBAC and privacy controls, drive observability, cost optimization, onboarding, and operational excellence for partner-facing data products.
Top Skills:
Python,Sql,Snowflake,Snowpark,Databricks,Pyspark,Liveramp,Airflow,Dbt,Great Expectations,Langchain,Llamaindex,Vector Databases,Snowflake Cortex,Rag,Llms
Artificial Intelligence • Insurance • Machine Learning • Software • Analytics
Lead design and implementation of scalable, HIPAA-compliant data pipelines and platforms for healthcare ML. Build ETL, orchestration, and tooling for processing EHR, claims, pharmacy, and bioinformatics data; collaborate with data scientists to produce modeling-ready datasets and ensure data quality, reliability, and operational excellence.
Top Skills:
Python,Sql,Apache Spark (Pyspark),Databricks,Snowflake,Airflow,Dagster,Prefect,Terraform,Docker,Kubernetes,Aws,Dbt,Ci/Cd
AdTech • Artificial Intelligence • Marketing Tech • Software • Analytics
The Senior Data Engineer will design, build, and operate data pipelines for Zeta's AdTech platform, focusing on high-scale data processing and analytics-ready datasets.
Top Skills:
AirflowAthenaAWSCassandraDagsterDeltaDynamoDBEmrFlinkGlueGoHudiIcebergJavaKafkaKinesisMySQLParquetPostgresPythonRedisRedshiftS3ScalaSparkSQLStep Functions
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
