540 Logo

540

Data Engineer

Posted 2 Months Ago
Remote
Hiring Remotely in USA
Mid level
Remote
Hiring Remotely in USA
Mid level
Design, build, and maintain scalable production data pipelines and APIs in a Databricks/Spark environment. Integrate and transform diverse health, financial, and operational datasets into reusable data products, implement data validation and governance, optimize pipeline performance, and collaborate with engineers, architects, and stakeholders to deliver reliable analytics datasets and platform capabilities.
The summary above was generated by AI

540 is seeking a Data Engineer to support a mission-critical federal health data modernization initiative. The team is bringing together data from diverse defense and federal systems to build scalable pipelines, APIs, and data products that improve how critical information is integrated, accessed, and used.

In this role, you’ll work hands-on with complex health and mission data to develop scalable, production-ready capabilities while helping establish reusable patterns for data ingestion, integration, and delivery. Your work will contribute to the technical foundation of a growing data platform ecosystem supporting multiple teams, data products, and mission areas.

Location: Remote within the continental United States. Periodic travel may be required
Citizenship & Clearance Requirement: per client requirements, candidates must be U.S. Citizens with an active DoW Secret (or higher) clearance
Education Requirement: Bachelor’s Degree in Computer Science, Engineering, Data Science, or a related technical field (preferred)
540 Internal Thrive Level: Data Engineer II or III

WHY 540?

540 is a forward-thinking company that the government turns to in order to #getshitdone. We don’t just talk about innovation – we deliver it. We break down barriers, build impactful technology, and solve mission-critical problems.

HOW YOU’LL DRIVE IMPACT

  • Design, develop, and maintain scalable, production-ready data pipelines using Spark, Python, and SQL in a Databricks environment
  • Integrate and transform diverse health, financial, and operational datasets into reliable, reusable data products
  • Develop data models and analytics datasets that support enterprise reporting and operational decision-making
  • Build and maintain API-based integrations for ingesting and exchanging data across systems
  • Monitor, troubleshoot, and optimize scheduled workflows and production pipelines for performance, reliability, and scalability
  • Implement data validation, quality checks, lineage, and governance practices that improve trust in data products
  • Contribute to automated testing, code reviews, technical documentation, and reusable engineering patterns
  • Collaborate with engineers, architects, analysts, and customer stakeholders to translate data needs into practical technical solutions

REQUIRED SKILLS & EXPERIENCE

  • 3+ years of data engineering, software engineering, or related technical experience
  • Strong proficiency with Python and SQL
  • Experience with Apache Spark or similar distributed data-processing frameworks
  • Experience working with Databricks or a similar modern data platform
  • Experience building and maintaining production data pipelines and ETL/ELT processes
  • Working knowledge of data modeling and transforming raw data into reusable analytics datasets or data products
  • Experience integrating or consuming data through REST APIs
  • Experience troubleshooting and optimizing data pipelines for performance and reliability
  • Familiarity with data validation, quality, governance, security, privacy, and compliance considerations
  • Experience with Git-based development workflows and modern software engineering practices
  • Comfort working in terminal and command-line environments
  • Experience using AI-assisted development tools to accelerate routine engineering tasks, testing, documentation, and debugging
  • Strong communication skills and experience working with customers or stakeholders to understand and resolve data needs
  • Ability to navigate ambiguity, learn unfamiliar systems, and independently drive assigned work toward resolution

NICE TO HAVE SKILLS & EXPERIENCE

  • AWS cloud experience
  • Experience working with very large datasets, including datasets containing billions of records
  • Experience with Palantir Foundry
  • Experience with Advana or similar DoW data environments
  • Experience with GitLab, CI/CD pipelines, or automated deployment workflows
  • Experience working with federal health, financial, or other regulated and sensitive data

BENEFITS & PERKS

  • Flexible PTO + all Federal holidays off
  • Health, dental and vision insurance plans
  • Flexible Spending Account (FSA)
  • 401k with employer match
  • Company-sponsored life insurance, short- and long-term disability 
  • Professional development (training, certifications, conferences)
  • Paid cloud developer accounts
  • Referral bonuses
  • HQ office perks (parking / metro reimbursement, nitro coffee & lunches) 
  • Annual social events (540 Week, hackathon, charity golf tournament, etc.)
  • Access to 540’s Washington Capitals & Nationals tickets

EQUAL EMPLOYMENT OPPORTUNITY (EEO)

540's policy is to provide equal employment opportunity to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.

Similar Jobs

7 Hours Ago
Remote or Hybrid
USA
140K-215K Annually
Senior level
140K-215K Annually
Senior level
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Architects, deploys, and operates scalable data platform infrastructure supporting analytics. Responsibilities include administering Airflow and Superset, managing Terraform infrastructure, building CI/CD pipelines, implementing security and compliance controls, developing dbt data models, cataloging assets in OpenMetadata, automating with Python, and mentoring engineers. The role requires extensive infrastructure and data engineering experience, cloud expertise, container orchestration, and eligibility for CJIS clearance.
Top Skills: Ai TechnologiesApache AirflowApache SupersetAWSAzureCi/CdDbtDockerGCPIamKubernetesMachine Learning PipelinesOciOpenmetadataPythonSQLTerraform
Yesterday
In-Office or Remote
120K-215K Annually
Senior level
120K-215K Annually
Senior level
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Build and lead production AI/ML, Generative AI, and data engineering solutions for healthcare analytics and patient safety platforms. Develop full-stack applications, microservices, APIs, data pipelines, dashboards, self-service analytics, RAG systems, and event-driven architectures using cloud-native technologies. Apply engineering best practices including testing, CI/CD, observability, security, and performance optimization. Mentor engineers, lead technical initiatives, and collaborate with product, analytics, business, and platform teams.
Top Skills: Amazon KinesisAWSAzureAzure Event HubsCi/CdDatabricksDelta LakeGCPGenerative AiJavaJavaScriptKafkaLarge Language Models (Llms)Microsoft FabricMlopsPysparkPythonReactRest ApisRetrieval-Augmented Generation (Rag)SnowflakeSparkSpring BootSQLTypescriptVector Search
Yesterday
Remote or Hybrid
US
105K-150K Annually
Senior level
105K-150K Annually
Senior level
Information Technology
Designs, develops, validates, and documents analytics data solutions for diverse client environments. Builds Snowflake, SQL, dbt, and AWS ingestion and transformation pipelines; monitors data quality and performance; troubleshoots complex technology issues; and supports reporting and business insights. Manages project scope, budgets, documentation, governance, client communication, implementation planning, and quality assurance while collaborating across teams and identifying opportunities for additional solutions.
Top Skills: Analytics Data MartsAWSData GovernanceData IngestionData PipelinesData PrivacyData TransformationDbtMedallion ArchitectureQa TestingRelational DatabasesSnowflakeSQL

What you need to know about the NYC Tech Scene

As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.

Key Facts About NYC Tech

  • Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
  • Key Industries: Artificial intelligence, Fintech
  • Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
  • Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account