Build and operate production LLM systems using Ollama and llama.cpp, develop RAG pipelines and MCP integrations, and create Python backend services for AI features. Deploy and maintain workloads across Azure and AWS using Docker and CI/CD. Automate operations, process structured data, support Kafka-based architectures, troubleshoot across the stack, and progressively develop expertise in model quantization, GPU tuning, and secure offline deployments.
Salvo Software is a global technology company specializing in custom software development and advanced engineering solutions. With distributed teams across the US, LATAM, and India, we partner with clients to build high-performance, scalable systems that solve complex technical challenges. Our culture values innovation, ownership, and engineering excellence. We're growing our AI department and are looking for a hands-on AI Developer to help build it.
Role OverviewWe are looking for an AI Developer to join and strengthen our AI department. Your core work will be building and operating LLM-powered systems: serving open-source models with Ollama and llama.cpp, building RAG pipelines, and developing MCP integrations that connect large language models to real tools and data. Solid DevOps fundamentals - Docker, CI/CD, Azure - support this work, but AI engineering is the heart of the role.
You don't need to be a deep ML researcher. What matters is production-grade Python, strong fundamentals, and the aptitude to learn fast. You'll work closely with our engineering and product teams to take LLM-powered features from prototype to reliably deployed systems, with mentorship available as you ramp up in areas like RAG architecture, Kafka, and advanced MCP work.
Key ResponsibilitiesAI / LLM Engineering (core focus)
- Serve and operate open-source LLMs using Ollama and llama.cpp, locally and in on-prem environments.
- Build and maintain RAG pipelines: embeddings, vector databases, chunking strategies, and retrieval quality.
- Develop MCP (Model Context Protocol) integrations connecting LLMs to internal tools and data sources.
- Build backend services in Python that power ML inference and AI-driven product features.
- Parse and process structured and semi-structured data (XML/XSD, Office document formats) as pipeline inputs.
- Grow into model optimization over time: quantization (GGUF), GPU/CUDA tuning, and offline/air-gapped deployments.
DevOps & Infrastructure (supporting)
- Build and maintain CI/CD pipelines for AI services (Azure DevOps preferred; GitHub Actions / GitLab CI also used).
- Deploy AI workloads to Microsoft Azure,AWS and containerize services with Docker.
- Automate operational tasks with Python, Bash and/or PowerShell scripting.
- Troubleshoot across the stack - dig into root causes rather than patching symptoms.
- Support event-driven architectures using Apache Kafka (producers/consumers).
Requirements
- 3–5+ years engineering experience with production-level Python for building real services, LLM inference, RAG pipelines, and automated tooling.
- Hands-on LLM & AI integration serving models via Ollama/llama.cpp, implementing vector retrieval/RAG, and developing MCP integrations (GPU/CUDA tuning & model quantization nice to have).
- Cloud & Infrastructure expertise covering Azure/AWS deployments, Docker containerization, Linux/Windows admin, and CI/CD automation (Azure DevOps, GitHub Actions, or GitLab CI), with Terraform, Kubernetes, Kafka, or systems languages (Go, Rust, C++) as a plus.
- DevOps, Security & Operations awareness, including Git workflows, scripting (Python, Bash, PowerShell), observability tools (Prometheus/Grafana), DevSecOps, and compliance frameworks (SOC 2, ISO 27001, Zero Trust).
- Strong problem-solving & communication with a root-cause troubleshooting mindset, self-directed learning, strong ownership, and seamless remote/distributed team collaboration.
Similar Jobs
Artificial Intelligence • Information Technology • Machine Learning • Natural Language Processing • Productivity • Software • Generative AI
Build and evolve Superhuman Mail’s Model Context Protocol platform, agentic workflows, personalization features, and LLM evaluation systems. Select and integrate models, develop AI building blocks for cross-functional teams, and help shape modeling approaches. The role focuses on backend engineering and applied AI, requiring production experience with LLM-powered features, strong ownership, and comfort with ambiguity and rapid change.
Top Skills:
AnthropicAWSGoKubernetesMcpOpenaiPostgresPythonRedisTurbopufferTypescript
Fintech • Financial Services
Design, build, and scale production AI agents for CX and operations workflows. Own agent architecture, tool calling, state management, observability, evaluation pipelines, integrations, deployment, cost optimization, and human-in-the-loop controls. Partner with operations, AI, engineering, and product teams to identify opportunities, validate business impact, and deliver safe, reliable automation. Mentor engineers and establish best practices for production-grade generative AI systems.
Top Skills:
Arize PhoenixCi/CdCrewaiDockerGitGitGoogle AdkGoogle Ai SdkKubernetesLangchainLanggraphLangsmithOpenai SdkPydanticPythonRagVector DatabasesWeights & Biases
Fintech • Financial Services
Design, build, and scale production AI agents for customer operations. Responsibilities include agent architecture, tool calling, state management, LLMOps observability, evaluation pipelines, RAG optimization, deployment, cost engineering, human-in-the-loop controls, and operational integrations. The role partners with operations and engineering leaders to translate business problems into reliable AI workflows, measure outcomes, and mentor engineers.
Top Skills:
Arize PhoenixCi/CdCrewaiDockerGitGitGoogle AdkGoogle Ai SdkKubernetesLangchainLanggraphLangsmithOpenai SdkPydanticPythonRagVector DatabasesWeights & Biases
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
