The role involves building and operating a multi-tenant analytics platform on AWS with a focus on streaming and batch data pipelines, managing workflows, and ensuring reliability and performance in production systems.
CTERA is looking for a Senior Data Engineer to build and operate a multi-tenant analytics platform on AWS + Kubernetes (EKS), delivering streaming and batch pipelines via GitOps as a Platform-as-a-Service (PaaS).
Responsibilities:
- Ingestion pipelines: Build and operate Flink / Spark streaming and batch jobs ingesting from Kafka, S3, APIs, and RDBMS into OpenSearch and other data stores.
- Platform delivery: Provide reusable, multi-tenant pipelines as a self-service PaaS.
- Workflow orchestration: Manage pipeline runs using Argo Workflows.
- GitOps delivery: Deploy and operate pipelines via ArgoCD across environments.
- IaC & AWS: Provision infrastructure with Terraform and secure access using IAM / IRSA.
- Reliability: Own monitoring, stability, and troubleshooting of production pipelines.
- Collaboration: Work with product, analytics, and infra on schemas and data contracts.
Requirements:
- Software skills: Senior-level, hands-on data engineering experience building and operating production systems with ownership of reliability and scale.
- Processing: Strong experience with Flink and Spark (streaming + batch).
- Data sources & sinks: Experience integrating with Kafka, S3, REST APIs, and RDBMS, and publishing to OpenSearch / Elasticsearch, data warehouses, or NoSQL databases.
- Big Data: Familiarity with big-data systems; Iceberg / PyIceberg a plus.
- Cloud & DevOps: Hands-on experience with EKS, RBAC, ArgoCD, and Terraform for infrastructure and delivery workflows.
- Datastores: Hands-on experience with OpenSearch / Elasticsearch including indexing strategies, templates/mappings, and operational troubleshooting.
- AI tools: Experience with AI-assisted development tools. (e.g., CursorAI, GitHub Copilot, or similar).
Top Skills
APIs
Argo Workflows
Argocd
AWS
Eks
Elasticsearch
Flink
Gitops
Kafka
Kubernetes
Opensearch
Rdbms
S3
Spark
Terraform
CTERA New York, New York, USA Office
205 E. 42nd Street, New York, New York, United States, 10017
Similar Jobs
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
The Senior Software Engineer will develop clean, scalable code, work with product owners, enhance products, and mentor colleagues.
Top Skills:
AngularJavaJavaScriptReactVue
Artificial Intelligence • Hardware • Information Technology • Security • Software • Cybersecurity • Big Data Analytics
Lead and execute sales strategy in Israel, develop relationships with key stakeholders, and oversee the entire sales lifecycle for Silvus Technologies.
Top Skills:
ManetRf NetworkingTactical Radio TechnologiesWireless Communications
Productivity • Sales • Software
As an RTS, you will handle complex technical issues, improve product quality, and drive initiatives to enhance customer experience while collaborating with R&D and other stakeholders.
Top Skills:
APIsChrome DevtoolsGraphQLOauthRedashSAMLSQL
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
