Railroad19
Jobs
Senior Data Engineer (GCP • Python • Iceberg • Delta Lake • Kafka • Snowflake • Databricks)
Railroad19
Senior Data Engineer (GCP • Python • Iceberg • Delta Lake • Kafka • Snowflake • Databricks)
Design and build modern GCP lakehouse architectures, including Python and Spark pipelines, BigQuery ingestion, Kafka CDC, Delta Lake and Iceberg UniForm integrations, Delta Sharing endpoints, Snowflake and Databricks connectivity, data lineage, governed access controls, and semantic-layer alignment. Collaborate cross-functionally, troubleshoot full-stack issues, and deliver reusable data-sharing adapters and end-to-end data features.
Railroad19, Inc is seeking a Senior Data Engineer with deep, hands‑on experience in building modern lakehouse architectures on GCP. This role focuses on designing, developing in Python & Spark, and delivering reusable data‑sharing adapters that connect BigQuery‑backed data products to Snowflake and Databricks using Iceberg UniForm and Delta Sharing.
About Railroad19:
At Railroad19, Inc, we develop customized software solutions and provide software development services. We’re a specialized team of developers and architects. As such, we only bring an “A” team to the table, through hard work and a desire to lead the industry — this is our company culture — this is what sets Railroad19 apart.
As a Railroad19 employee, you will be part of a company that values your work and gives you the tools you need to succeed. Our headquarters is in Saratoga Springs, New York, but this position is 100% remote. Railroad19 provides competitive compensation and excellent benefits, including Medical/Dental/Vision/Pet Insurance, Paid Time Off, and 401 (k).
NO 1099, C2C, Corp-to-Corp; only full-time employment.
NO Agencies.
Core Responsibilities:
- Design and implement the UniForm write layer (Delta + Iceberg dual metadata).
- Build GCS → BigQuery ingestion pipelines for structured operational datasets.
- Develop and implement in Python and Spark.
- Implement Kafka-based CDC patterns for real-time and near-real-time ingestion.
- Develop data lineage, dependency tracking, and modular adapter code.
- Configure Snowflake Horizon external tables for zero-copy reads.
- All data hub tables are to be written once using Delta Lake with Iceberg UniForm enabled… readable by all target consumers without conversion.
- Implement and certify Delta Sharing endpoints for Databricks consumers.
- Build governed access layer components: RBAC, connector registry entries, tenant-scoped authorization.
- Align semantic layer models with LookML and KPI catalog definitions.
- Collaborate with cross‑functional teams to deliver end‑to‑end features.
- Troubleshoot issues across the full stack and contribute to code quality.
Skills/Experience:
- 6+ years of proven enterprise-level experience in Python & Spark
- Advanced experience in GCP BigQuery
- Strong working knowledge of Apache Iceberg, Delta Lake, Iceberg UniForm
- Experience with Delta Sharing; Kafka / CDC pipelines
- Specific work experience in Snowflake Horizon Catalog; Databricks Unity Catalog
- Solid experience with Data lake architecture & ingestion pipeline design
- Excellent Communication skills and the ability to work cohesively with multiple teams.
Preferred Experience – Nice to Have
- Prior delivery in enterprise SaaS, media, or advertising technology.
- Active daily use of AI-assisted development tools (Claude Code preferred).
Similar Jobs
eCommerce • Information Technology • Retail • Industrial
Lead UX strategy and execution for Search and Guidance on Grainger.com. Collaborate with Product, Engineering, ML, Analytics, and Merchandising to design scalable, accessible search and discovery experiences. Conduct research and usability testing, deliver high-fidelity designs and prototypes, evolve the design system, and communicate design rationale to stakeholders to improve discoverability and conversion.
Top Skills:
AccessibilityAi ToolsDesign SystemsFigmaPrototypingResponsive Web DesignWcag 2.1 Aa
Artificial Intelligence • Automotive • Greentech • Information Technology • Machine Learning • Software • Cybersecurity
Processes, codes, audits, and indexes service provider invoices for fleet billing operations. Responsibilities include resolving invoice discrepancies, verifying customer approval limits, completing WACC activities, processing rebills, performing VMRS coding and costing, handling purchase order inquiries, and communicating with service providers and team members.
Top Skills:
TmcareVmrs
Artificial Intelligence • Automotive • Greentech • Information Technology • Machine Learning • Software • Cybersecurity
Processes, codes, audits, indexes, and rebills invoices for fleet services. Contacts service providers to resolve discrepancies, verifies customer approval limits, completes billing workflows, handles purchase order inquiries, and performs VMRS coding and invoice costing. Requires strong attention to detail, accurate high-volume processing, and effective communication.
Top Skills:
TmcareVmrs
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
