Lead GPU-edge inference systems, ensuring compliance and performance through design and monitoring. Manage global infrastructure for AI deployment.
Full-time | Remote | Infrastructure | Reports to CTO
About Elloe
Elloe is the trust layer for AI.
We sit between the world’s most powerful language models and the institutions that can't afford to get it wrong — hospitals, banks, regulators. We trace and block failures in real time. That’s not marketing — we’re deployed at the European Commission, with NIH clinical trials, and inside a Top-5 EU bank catching GDPR violations live.
This is the enforcement layer GenAI has been missing. We're not visualizing problems — we're fixing them.
About the Role
You’ll lead our GPU-edge inference systems. From chaos-resilient deployment to SHAP-driven compliance metrics, you’ll own global infra that makes AI safe and performant.
What You’ll Own
1. Global Edge Routing
- Design zone-routing that ensures <50ms SLA in 10+ regions
- Build fallback orchestration to handle compliance-aware rollbacks
2. GPU Infra Ops
- Maximize utilization across 100K+ GPUs via mesh & load prediction
- Integrate compliance overlays with VaultChain and SHAP triggers
3. Reliability Telemetry
- Ship `/vault/audit`, `/inference/predict`, `/compliance/log` endpoints
- Trace every edge request across governance and model layers
Who You Are
- Senior systems engineer with GPU fleet experience (KubeRay, Istio, Envoy)
- Operated real-time AI infra with 10M+ QPS loads
- Comfortable with compliance observability and infra governance
Why This Matters
Our competitive edge isn’t just AI — it’s defensible enforcement. This role turns that into product.
You’ll Leave This Role With
- Referenceable contributions to enforcement infra that’s live in EU and US institutions
- First-hand product work across legal, engineering, and GTM teams
- Influence over how regulatory primitives become systems people trust
Logistics & Application
- Start Date: Flexible (Q3–Q4 ideal)
- Location: Remote-first; timezone overlap with NY or EU preferred
- Compensation: Top of market salary + equity
- To Apply: Send your resume and a sentence on the hardest infra problem you'd want to own at scale.
Similar Jobs
Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
Owns complex Packet Core customer service requests from initiation through resolution, leading network investigations, root-cause analysis, incident response, and service improvements. Provides technical leadership during crises, coordinates global support teams, maintains technical knowledge, and develops stable customer solutions. The role requires extensive Ericsson Packet Core expertise, cloud and virtualization knowledge, Linux and IP networking skills, and strong customer-facing communication.
Top Skills:
Dns/IpworksDockerEpcEpgEricsson Packet CoreImsIp NetworkingKubernetesOpenstackPcc/PcgPcrfProps-CRanSapc/CcpcSgsn-MmeUnix/Linux
Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
Leads techno-commercial strategy for RAN engagements across East and North Africa. Translates radio technology into ROI, TCO, pricing, offer scope, negotiation, and value narratives. Partners with sales, pricing, contract management, presales, and technical domain teams to improve win rates, protect margins, and support profitable growth. Requires strong RAN expertise, complex telecom deal structuring, commercial modelling, stakeholder management, financial fluency, and negotiation skills.
Top Skills:
2G3G4G5GCloud ComputingCloud RanCore NetworksMassive MimoOpen RanOss/BssRanSoftware LicensingTransport Networks
Energy
Supervises daily production operations, coordinates cross-functional activities, assigns work, and provides technical guidance. Ensures standardized processes, quality, safety, and contractual compliance while adapting workflows to client needs. Participates in production planning, resource allocation, reporting, and continuous improvement by analyzing operational data, identifying bottlenecks, and optimizing performance.
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
