KANINI Logo

KANINI

Sr. GCP Data Engineer (Job-ID 3301)

Posted 3 Months Ago
In-Office or Remote
Hiring Remotely in Nashville, TN
Senior level
In-Office or Remote
Hiring Remotely in Nashville, TN
Senior level
Design, develop, and optimize scalable batch and real-time data pipelines on GCP. Implement Medallion Architecture, build Python and PySpark transformations, optimize BigQuery and SQL workloads, deploy Dataflow pipelines, and orchestrate workflows with Cloud Composer and Airflow. Manage GCS storage, CI/CD, version control, IAM, data governance, security, reliability, scalability, and cost efficiency across enterprise data platforms.
The summary above was generated by AI

This is a remote position.

KANINI is seeking a highly skilled Senior Data Engineer with deep expertise in Google Cloud Platform (GCP) and modern data architecture. The ideal candidate will have hands-on experience designing scalable data pipelines, implementing Medallion Architecture, and building robust enterprise-grade data solutions.

This role requires strong technical proficiency in BigQuery, PySpark, Dataflow, and Airflow, along with a solid understanding of cloud data governance, performance optimization, and CI/CD practices.


Key Responsibilities
  • Design, develop, and maintain scalable batch and real-time data pipelines on GCP
  • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data processing
  • Build high-performance data transformations using Python and PySpark
  • Develop and optimize complex SQL queries for analytical workloads
  • Work extensively with BigQuery for large-scale data processing and performance tuning
  • Develop and deploy pipelines using Cloud Dataflow
  • Orchestrate workflows using Cloud Composer (Apache Airflow)
  • Manage data storage and lifecycle using Google Cloud Storage (GCS)
  • Implement version control and CI/CD pipelines using Git-based tools
  • Ensure data security, governance, and access control using GCP IAM
  • Optimize data solutions for performance, scalability, reliability, and cost-efficiency


Required Skills & Experience
  • Strong hands-on experience with Google Cloud Platform (GCP)
  • Expertise in BigQuery (partitioning, clustering, query optimization)
  • Proven experience implementing Medallion Data Architecture
  • Strong programming skills in Python and PySpark
  • Hands-on exposure on Java
  • Advanced proficiency in SQL (complex joins, window functions, performance tuning)
  • Hands-on experience with Cloud Dataflow
  • Experience with Cloud Composer (Airflow) for orchestration
  • Experience working with Google Cloud Storage (GCS)
  • Knowledge of version control systems (Git) and CI/CD practices
  • Strong understanding of GCP IAM and cloud security best practices

Preferred Qualifications

  • Experience working with large-scale enterprise data platforms
  • Knowledge of data warehousing and data lake concepts
  • Familiarity with real-time streaming frameworks
  • Experience in data governance and data quality frameworks
  • Exposure to

Similar Jobs

2 Hours Ago
Remote or Hybrid
United States
142K-195K Annually
Senior level
142K-195K Annually
Senior level
Cloud • Fintech • Software • Business Intelligence • Consulting • Financial Services
Lead CFO-level client advisory engagements, manage change initiatives, provide financial analysis, develop client budgets, and mentor staff.
Top Skills: Bill.ComIntaactNetSuiteQuickbooks Online
2 Hours Ago
Remote or Hybrid
United States
142K-195K Annually
Senior level
142K-195K Annually
Senior level
Cloud • Fintech • Software • Business Intelligence • Consulting • Financial Services
The CFO will oversee month-end close, financial reporting, audit readiness, client relationship management, and mentoring staff for casino industry clients.
Top Skills: Accounting Software
2 Hours Ago
Remote or Hybrid
United States
107K-144K Annually
Senior level
107K-144K Annually
Senior level
Cloud • Fintech • Software • Business Intelligence • Consulting • Financial Services
Manage complex tax returns and consulting for partnerships, mentor staff, and develop tax planning strategies while ensuring compliance with tax regulations.
Top Skills: Internal Revenue CodeIrs RegulationsTax-Related Software

What you need to know about the NYC Tech Scene

As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.

Key Facts About NYC Tech

  • Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
  • Key Industries: Artificial intelligence, Fintech
  • Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
  • Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account