Designs and implements scalable data lakehouse platforms on GCP, including BigQuery, GCS, BigLake, and Dataplex. Leads real-time and batch data pipeline development, data governance, metadata management, security, and performance optimization. Partners with clients and stakeholders to translate business needs into architecture, leads technical teams, mentors engineers, and provides hands-on solutioning and technical advisory support.
About Egen:
Egen is a fast-growing and entrepreneurial company with a data-first mindset. We bring together the best engineering talent working with the most advanced technology platforms, including Google Cloud and Salesforce, to help clients drive action and impact through data and insights. We are committed to being a place where the best people choose to work so they can apply their engineering and technology expertise to envision what is next for how data and platforms can change the world for the better. We are dedicated to learning, thrive on solving tough problems, and continually innovate to achieve fast, effective results. If this describes you, we want you on our team.
Want to learn more about life at Egen? Check out these resources in addition to the job description.
Meet Egen
Life at Egen
Culture and Values at Egen
Career Development at Egen
Benefits at Egen
Responsibilities
- Be part of the end-to-end architecture, design, and implementation of scalable Data Lakehouse solutions on Google Cloud Platform (GCP) using BigQuery, GCS, BigLake, and Dataplex
- Collaborate directly with customers to understand business goals, data challenges, and technical requirements; translate them into robust architectural blueprints and actionable plans
- Design and implement data pipelines supporting both real-time and batch ingestion using modern orchestration and streaming frameworks
- Establish and enforce best practices for data cataloging, metadata management, lineage, and data quality across multiple systems
- Define and implement data security, access control, and governance models in compliance with enterprise and regulatory standards
- Serve as the technical lead for project teams - mentoring engineers, reviewing solutions, and ensuring architectural consistency across deliverables
- Balance strategic architecture discussions with hands-on solutioning, POCs, and deep dives into data pipelines or performance tuning
- Partner with stakeholders, cloud architects, and delivery leads to drive solution adoption, scalability, and long-term maintainability
- Represent the company as a trusted technical advisor in client engagements - clearly articulating trade-offs, best practices, and recommendations
Qualifications
- 6+ years of progressive experience in Software Engineering and Data Platform development, with 5+ years architecting data platforms on GCP and/or Databricks
- Proven hands-on experience designing and deploying Data Lakehouse platforms with data products and medallion architectures
- Strong understanding of data ingestion patterns (real-time and batch), ETL/ELT pipeline design, and data orchestration using tools such as Airflow, Pub/Sub, or similar frameworks
- Expertise in data modeling, storage optimization, partitioning, and performance tuning for large-scale analytical workloads
- Experience implementing data governance, security, and cataloging solutions (Dataplex, Data Catalog, IAM, or equivalent)
- Excellent communication and presentation skills - able to confidently engage with technical and non-technical stakeholders and guide clients through solution decisions
- Demonstrated ability to lead by example in mixed teams of engineers, analysts, and architects, balancing architectural vision with hands-on delivery
- Nice to have: Experience with Databricks (Delta Lake, Unity Catalog) and hybrid GCP-Databricks data architectures
- Strong problem-solving mindset, curiosity to explore new technologies, and ability to “zoom out” for architecture discussions and “zoom in” for code-level troubleshooting
Similar Jobs
.png){kind=logo}
AdTech • Cloud • Digital Media • Information Technology • News + Entertainment • App development
Define and lead the architecture of NBCUniversal’s greenfield Microsoft Fabric enterprise data platform. Establish lakehouse, warehouse, integration, real-time streaming, data modeling, governance, security, DevOps, and observability standards. Translate architecture and business objectives into delivery roadmaps, epics, stories, and implementation plans while guiding multidisciplinary engineering teams. Design scalable solutions combining operational, historical, master, and reference data to support reporting, analytics, automation, AI, and reusable data products.
Top Skills:
Amazon KinesisAPIsAzure DevopsAzure Event HubsBicepChange Data CaptureCi/CdData FactoryDelta LakeEventhouseEventstreamGithub ActionsKafkaLakehouseAzureMicrosoft FabricNotebooksOnelakePower BIPysparkPythonReal-Time IntelligenceSemantic ModelsSQLTerraformWarehouse
Artificial Intelligence • Insurance • Software • Automation
Own the architecture and standards for a trustworthy BI analytics platform. Design data models, ingestion and transformation patterns, lineage, PII controls, reliability SLAs, observability, schema governance, and cost management. Partner with engineers, analysts, business leaders, Security, Legal, and Compliance; enforce testing, CI/CD, infrastructure as code, and data quality practices. Mentor data professionals and document architectural decisions.
Top Skills:
AirflowAlationAtlanCi/CdCollibraDagsterDatahubDbtGitPythonSnowflakeSQLTerraform
Database
Leads enterprise data-platform architecture for clients using Microsoft Fabric and Azure. Responsibilities include discovery, target-state design, data modeling, migrations, DevOps and CI/CD, governance, security, AI/ML enablement, consulting, pre-sales, technical delivery leadership, mentoring, and development of reusable architecture assets. The role requires hands-on Fabric production experience, enterprise-scale implementation ownership, advanced SQL and Python/PySpark skills, client communication, and travel up to 25%.
Top Skills:
Adls Gen2AzureAzure Ai SearchAzure Data FactoryAzure DevopsAzure FunctionsAzure Key VaultAzure Machine LearningAzure OpenaiAzure Sql DatabaseAzure Sql Managed InstanceAzure Synapse AnalyticsBicepCosmos DbDacpacDatabricksDatabricks Asset BundlesDataflows Gen2DaxDirect LakeEr/StudioErwin Data ModelerEvent HubsFabric Data FactoryFabric Git IntegrationFabric IqFabric Rest ApisFabric WarehouseFabric-CicdFoundry IqGitGithub ActionsGithub CopilotGraph In FabricKqlLakehouseManaged VnetsMicrosoft Entra IdMicrosoft FabricMicrosoft FoundryMicrosoft PurviewOnelakePower BIPowershellPrivate LinkPysparkPythonSparkSpark SqlSQL ServerSqldbmSqlpackageSsasSsisT-SqlTerraformUnity Catalog
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
