About the company Tensormesh is building the next generation of AI inference infrastructure. Our mission is to make large language models faster, cheaper, and easier to deploy across any environment — cloud, on-prem, or hybrid. We help enterprises and AI teams optimize GPU utilization and scale inference workloads with up to 10× better performance.
About the role We are seeking a Kubernetes Engineer to build the core orchestration logic of the Tensormesh platform. Unlike a standard DevOps role, this position involves deep software development within the Kubernetes ecosystem. You will extend Kubernetes capabilities by writing custom operators and controllers that manage complex AI inference workloads, ensuring high availability and seamless auto-scaling.
What you’ll do
Develop Custom Operators: Design and implement Kubernetes Custom Resource Definitions (CRDs) and Controllers (using Golang/Kubebuilder) to manage the lifecycle of Tensormesh products.
Traffic Management: Architect and build high-performance ingress and load-balancing systems capable of handling high-throughput inference requests.
Resilience & Scaling: Develop logic for intelligent auto-scaling (HPA/VPA) based on GPU metrics and ensure High Availability (HA) for critical components.
K8s Optimization: Tune the scheduler and resource management configurations to maximize GPU utilization for inference tasks.
Ideal candidate credentials
0-3 years of software engineering experience, with a focus on distributed systems or container orchestration.
Strong proficiency in Go (Golang); experience with the Kubernetes client-go library and Kubebuilder/Operator SDK is highly preferred.
Deep understanding of Kubernetes internals (API machinery, Controller runtime, Networking, CNI).
Experience with service meshes (Istio, Linkerd) or ingress controllers (Nginx, Traefik) is a plus.
Understanding of distributed consensus and state management.
Compensation & Benefits
Competitive base salary
Performance-based bonus
Equity options
Medical, dental, and vision insurance
401(k) retirement plan
Paid time off
Why Join Tensormesh
Build infrastructure powering next-generation AI applications
Work alongside a highly technical and experienced engineering team
Make a direct impact in a fast-growing startup environment
Take ownership of challenging technical problems at scale
Similar Jobs
What you need to know about the NYC Tech Scene
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
