Staff Data Engineer

Blackbird.AI helps organizations discover emergent threats and stay one step ahead of real-world harm through our AI-powered Narrative and Risk Intelligence Platform. Our commitment is to prioritize safety and security, providing the tools to identify potential risks and ensure a safer environment proactively. No matter the job or where it's located, we're all connected by a shared vision: To lead and enhance the landscape of risk intelligence.
As a Staff Data Engineer, you will play a critical role in architecting and scaling our data platform and AI/ML processing infrastructure. You'll be a technical leader responsible for our entire data ecosystem—from ingestion pipelines that process diverse data sources to the lakehouse architecture that powers our narrative analysis capabilities. You'll architect systems that seamlessly support batch and streaming data patterns while building real time alerting on generated insights.

You'll work at the intersection of data engineering, AI-powered data transformation, and platform engineering, making architectural decisions that will shape our ability to detect misinformation, disinformation, and narrative attacks at scale while managing costs effectively. A key aspect of this role involves building intelligent pipelines that use traditional AI and generative AI to cluster, enrich, classify, and extract insights from data as it flows through our system.

As a Staff Data Engineer you will:

• Design and implement scalable data platform architecture on Databricks, supporting both batch and streaming ingestion
• Build robust, fault-tolerant data ingestion pipelines that integrate with multiple third-party APIs and data providers
• Design and implement AI-powered enrichment stages within pipelines—applying ML clustering, generative AI summarization, classification, and entity extraction to transform raw data into actionable intelligence
• Build analytical systems with full-text search capabilities using Elasticsearch for rapid querying and analysis of enriched data
• Work with AI/ML researchers to implement, integrate and scaling AI processing
• Expose data platform capabilities as APIs and other interfaces for downstream consumption by applications and services
• Optimize data lake and lakehouse architecture for performance, cost-efficiency, and scalability
• Design and implement data quality frameworks, monitoring, and alerting systems
• Design efficient architectures for calling external AI APIs and managing rate limits, costs, and reliability
• Architect solutions with cost-efficiency as a first-class concern, implementing monitoring and optimization strategies for compute and storage
• Make critical build-vs-buy decisions and establish architectural standards for the data organization
• Mentor engineers and elevate the team's technical capabilities through code reviews, design discussions, and knowledge sharing

• 8+ years of software engineering experience with 5+ years focused on data platforms or data engineering
• Deep expertise with Databricks, Apache Spark, and data lakehouse architectures
• Strong experience building and operating data pipelines at scale (handling TBs+ of data)
• Experience integrating AI/ML capabilities into data pipelines (clustering, LLM APIs, classification, summarization)
• Proficiency in Python, DBT, and SQL for data processing and pipeline development
• Experience with both batch and streaming large scale data processing patterns
• Strong understanding of cloud platforms (AWS, Azure)
• Excellent communication skills and ability to mentor engineers

Preferred Qualifications:

• Experience designing both batch and streaming/near real-time data architectures
• Proficiency with Elasticsearch for building analytical systems with full-text search capabilities
• Hands-on experience with LLM APIs and understanding of rate limiting and cost optimization
• Experience with Agentic AI, context engineering, and evaluation
• Background in trust & safety, security, or content moderation domains
• Experience with data observability tools and building comprehensive monitoring systems
• Prior experience at a startup or fast-paced environment
• Apply agentic coding tools for day to day development
• Familiarity with Databricks' Lakeflow, Agent Bricks, and vector databases

What’s in it for you:

Blackbird.AI is embarking on an exciting growth journey with numerous opportunities for career development within the company. You will join a nurturing, inclusive, and experienced team. 

Join us as we soar to new heights!

Values:

At Blackbird.AI, our core values shape how we work and make decisions. Our values inspire us to be authentic and continue improving. 

We embrace a strong sense of responsibility to society, recognizing the vital role our services play in empowering governments, communities, and individuals to foster critical thinking and empowerment. We believe in integrating personal and professional lives with societal needs, emphasizing the importance of creating an environment that attracts top talent and provides substantial growth opportunities. We are motivated by the potential of science and technology to impact humanity positively. 

• Competitive compensation package, 401(k), and equity - everyone has a stake in our growth!
• Comprehensive health benefits for you and your loved ones, including wellness days and monthly wellness reimbursements - an apple a day doesn't always keep the doctor away!
• Generous vacation policy, encouraging you to take the time you need - we trust you to strike the right work/life balance!
• A flexible work environment with opportunities to collaborate with your team in person - you can have it all!
• Inclusion and Impact - soar to new heights!
• Professional development stipend - never stop learning!

Location & Work Eligibility:

We are only able to hire candidates currently residing in the U.S. Unfortunately, we cannot offer visa sponsorship for this role. Applicants must be legally authorized to work in the U.S. without future sponsorship. Candidates applying for this position should meet the residency requirement and be able to provide proof of U.S. work authorization. 

Pay Transparency:  [NEW YORK ONLY]

For individuals assigned and/or hired to work in New York, Blackbird.AI is required by law to include a reasonable estimate of the compensation range for this role. This compensation range is specific to New York. It takes into account the wide range of factors that are considered in making compensation decisions, including, but not limited to, skill sets, experience and training, licensure and certifications, and other business and organizational needs. At Blackbird.AI, it is not typical for an individual to be hired at or near the top of the range for their role, and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current compensation range for this position is expected to be $160,000 - $190,000. This range may vary for positions outside of New York and as it has not been adjusted for the applicable geographic differential associated with the location where the position may be filled. 

Regardless of location, candidates can expect during the first few conversations with Blackbird.AI’s Talent Team and Hiring Managers to share any approved budget. 

Apply Today

Equal Opportunity Employer