Define the architectural foundation for processing unstructured data for LLM applications. Oversee the core platform's development and mentor engineering teams.
Unstructured is defining the standard for enterprise data transformation in the age of LLMs and generative AI. In just two years, we’ve raised over $65M from world-class investors, including Menlo Ventures, Bain Capital, Databricks, NVIDIA, Microsoft, and IBM.
Our open-source toolkit has been downloaded 52M+ times and is used by 66,000+ companies, including nearly half of the Fortune 500. We power production AI workflows across commercial and federal sectors—transforming PDFs, HTML, Word docs, images, emails, and more into AI-ready data pipelines that scale.
We’re not just building tools, we’re building the backbone of generative AI and the infrastructure that unlocks intelligence across industries.
Unstructured is looking for Staff and Principal Software Engineers to define the architectural foundation for how enterprises process and transform unstructured data for large-language-model (LLM) applications.
This is a high-impact, hands-on role for someone who thrives in deep systems work—someone who could write the book on Kubernetes, lives and breathes Python, and understands Postgres at the index and transaction-log level. You’ll own the technical direction for our core platform, guiding how data moves through every layer—from ingestion to transformation to retrieval.
You’ll partner closely with the Head of Engineering, Head of Infrastructure, and product leaders to ensure Unstructured’s systems are performant, resilient, and ready to support the next generation of AI-driven workloads.
Location(s) Preferred: San Francisco, Bay Area, San Jose, Seattle, Sacramento
What You’ll Own & Drive:
Define and evolve the end-to-end architecture for Unstructured’s data transformation and retrieval platform.
Build and scale distributed systems that process massive volumes of unstructured data across diverse formats and sources.
Serve as the company-wide authority on Kubernetes orchestration, cluster design, performance tuning, and reliability.
Lead Python architecture and best practices—ensuring performance, modularity, and maintainability across services.
Design and optimize Postgres schemas, queries, and indexing strategies to support large-scale metadata and retrieval pipelines.
Mentor senior engineers through design reviews and code guidance, raising the bar for technical excellence across the org.
Partner with the infrastructure and product teams to translate research prototypes into production-grade systems.
Evaluate emerging technologies and open-source tools in LLM infrastructure, retrieval, and orchestration—deciding where and how to integrate them.
What You Bring:
Have 10+ years of software engineering experience with a focus on distributed systems, infrastructure, or data architecture.
Are a Python expert—capable of building frameworks and performance-critical services from scratch.
Have deep Kubernetes expertise; you can design, deploy, and debug at scale and could teach others how to productionize it securely.
Are fluent in Postgres—you understand query planning, partitioning, and tuning for high-throughput environments.
Are obsessed with clean, scalable architecture and can lead design reviews that shape how entire systems evolve.
Have experience in high-performance data or AI/ML systems—especially those involving retrieval pipelines, embeddings, or hybrid workloads.
Thrive in fast-moving, ambiguous environments where technical depth and judgment matter more than process.
Bonus Points
Experience building or scaling LLM-powered or RAG systems in production.
Familiarity with open-source orchestration frameworks, vector databases, or hybrid cloud infrastructure.
Contributions to open-source projects in Python, Kubernetes, or distributed systems.
Why You’ll Love It Here
You’ll be surrounded by smart, kind, low-ego people who genuinely enjoy building together. We invest in our team, with company offsites, best-in-tech swag, and the tools you need to do your best work, wherever you’re based.
The work itself is deeply meaningful. You’ll be at the center of some of the most exciting problems in AI today - tackling meaty, novel technical challenges that directly power real production systems used by thousands of companies, including nearly half of the Fortune 500.
We support you holistically, not just at work. From day-one medical, dental, and vision coverage (plus life and disability insurance), to unlimited PTO, flexible parental leave, 401(k) options, and competitive referral incentives, we want you focused on building, growing, and staying energized for the long haul.
Most importantly, we’re scaling fast. This is a chance to join early enough to build foundational technology, influence how we operate, and help shape the future of a company defining the next era of AI infrastructure.
If that sounds like your kind of team, we’d love to meet you.
Similar Jobs
AdTech • Agency • Artificial Intelligence • Digital Media • Marketing Tech • Social Media • PropTech
Senior individual-contributor full-stack engineer who designs, builds, and ships end-to-end product features. Leads technical direction, modernizes legacy systems, debugs distributed systems, owns production reliability and on-call, and mentors engineers. Drives AI-native development adoption while collaborating with Product and Design and improving team practices, observability, testing, and deployment workflows.
Top Skills:
Ai ToolsAWSGCPNode.jsReactSQLTypescript
Security • Cybersecurity
Designs and delivers leadership and manager development programs, owns the LMS, creates multi-format learning content, partners with People Business Partners, manages vendors, measures program effectiveness, and leverages AI to scale learning.
Top Skills:
Absorb LmsAi-Enabled Learning ToolsLearning Management System
Fintech • Financial Services
Own and architect Forward's security technology stack across AppSec, SecOps, and GRC. Design integrations with Engineering/IT, lead tool evaluations and PoCs, automate controls with infrastructure-as-code and detection-as-code, build centralized logging/SIEM and identity governance, align architecture to security frameworks, mentor security engineers, and support incident response and remediation.
Top Skills:
AWSAzureChronicleCi/CdCis ControlsCloudtrailGCPGoGuarddutyIamIso 27001KubernetesMitre Att&CkNistPantherPythonRubySIEMSoc 2SplunkTerraformWaf
What you need to know about the NYC Tech Scene
As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.
Key Facts About NYC Tech
- Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
- Key Industries: Artificial intelligence, Fintech
- Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
- Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory
