540 Logo

540

Data Engineer

Posted 9 Days Ago
Remote
Hiring Remotely in USA
Junior
Remote
Hiring Remotely in USA
Junior
Build and maintain scalable Python-based data pipelines and cloud-native solutions supporting U.S. Army financial-system modernization. Develop ETL/ELT workflows using AWS Glue, Lambda, S3, and CloudWatch; integrate APIs and diverse file formats; improve data reliability, performance, security, and governance; implement testing and monitoring; and collaborate with engineers and stakeholders in an Agile environment.
The summary above was generated by AI

540 is looking for a Data Engineer to develop data transformation and workflow solutions supporting an impactful U.S. Army modernization initiative. You will help deploy and operate a cloud-based replacement for a legacy financial system while developing innovative, cloud-first solutions that strengthen the Army's data management ecosystem.

In this role, you'll design, build, and maintain scalable data pipelines and cloud-native data solutions, with a focus on data reliability, security, governance, and performance. You'll work with large, complex datasets, contribute to technical design decisions, and collaborate across engineering teams to transform business and mission requirements into working software.

Location: Remote within the continental United States, East Coast working hours with some flexibility afforded. Occasional travel may be required (3-4x annually)
Citizenship & Clearance Requirement: per client requirements, must be a U.S. Citizen with the ability to obtain a DoW Secret clearance
Education Requirement: Bachelor’s Degree in Computer Science or related engineering field (preferred)
540 Internal Thrive Level: Data Engineer II

WHY 540?

540 is a forward-thinking company that the government turns to in order to #GetShitDone. We do more than talk about it, we break down barriers to innovation and build impactful tech. We’re looking for someone with big ideas, who thrives on innovation and is a self-starter. If you love to learn about and work with new technologies, we want to talk to you.

HOW YOU’LL DRIVE IMPACT

  • Design, build, maintain, and troubleshoot reliable data pipelines throughout the software development lifecycle, supporting the modernization of legacy U.S. Army financial systems and migration to AWS cloud environments
  • Develop Python-based workflows to ingest, transform, validate, and deliver structured and semi-structured data
  • Build and support cloud-native data solutions using AWS services such as Glue, Lambda, S3, and CloudWatch
  • Integrate data from multiple systems and sources, including APIs and various file formats, while supporting data accuracy, consistency, and reliability
  • Work with large datasets, identify performance bottlenecks, and help improve pipeline efficiency and scalability
  • Participate in technical design discussions and contribute to decisions around data integration, storage, performance, and scalability
  • Implement automated testing, logging, monitoring, and error handling to support data quality and reliable system performance
  • Write clean, reusable, and maintainable code while following established software engineering and DataOps practices
  • Collaborate with engineers, architects, and stakeholders to translate business and technical requirements into working software
  • Support secure data management practices, including access controls, data governance, and compliance with DoW requirements
  • Participate in code reviews and Agile development activities, sharing ideas and contributing to engineering improvements
  • Document data transformations, pipeline processes, and technical decisions to support maintainability and downstream users
  • Identify opportunities to improve existing workflows, reduce manual processes, and strengthen the reliability of the Army's data ecosystem

REQUIRED SKILLS & EXPERIENCE

  • 2+ years of related experience in data engineering, software development, data integration, or a closely related field
  • Experience developing Python-based data transformations, scripts, or workflow automation
  • Hands-on experience building, maintaining, or troubleshooting data pipelines and ETL/ELT processes
  • Experience consuming data from APIs and working with file formats such as X12, JSON, flat files, or fixed-width files
  • Experience working with AWS cloud services, particularly AWS Glue, AWS Lambda, Amazon S3, and CloudWatch Logs
  • Familiarity with working with large datasets, including identifying and troubleshooting data processing or performance issues
  • Understanding of data structures, data modeling concepts, and common approaches to organizing and transforming data
  • Experience writing and maintaining automated tests using Pytest or similar testing frameworks
  • Familiarity with data validation, error handling, logging, and debugging practices
  • Understanding of data security, access controls, and data quality principles
  • Experience using GitLab for version control and collaborative development
  • Comfortable working through the terminal or command-line interface
  • Experience using Jira and Confluence in an Agile development environment
  • Ability to interpret ambiguous requirements, file specifications, and JSON structures to develop functional data solutions
  • Strong problem-solving skills and ability to collaborate effectively with technical and non-technical stakeholders
  • Must currently possess or have the ability to obtain an AWS Cloud Practitioner certification (or higher) within 30 days of hire
  • Must currently possess or have the ability to obtain a CompTIA Security+ certification (or higher) within 30 days of hire

NICE TO HAVE

  • Experience working with DoW systems, data, or other regulated environments
  • Familiarity with X12/MILS processing
  • Experience with Apache Spark, Kafka, or other distributed data processing frameworks
  • Knowledge of data orchestration tools such as Apache Airflow or AWS Step Functions
  • Exposure to Infrastructure-as-Code tools, including Terraform or CloudFormation
  • Understanding of DevOps and DataOps practices, including CI/CD pipelines
  • AWS certifications such as Solutions Architect, Data Analytics, or similar
  • Experience developing reusable pipeline components or improving established engineering patterns
  • Experience contributing to data architecture decisions, schema design, or pipeline performance optimization
  • Familiarity with database performance optimization, indexing, or partitioning strategies
  • Experience with data lineage, metadata management, or data cataloging tools
  • Experience providing technical guidance, mentoring teammates, or contributing to engineering best practices
  • Familiarity with AI-assisted development tools for coding, testing, documentation, or workflow automation

BENEFITS & PERKS

  • Health, dental and vision insurance plans
  • Flexible Spending Account (FSA)
  • 401k with employer match
  • Company Sponsored Life Insurance, Short and Long Term Disability coverage
  • Flexible PTO + all Federal holidays off
  • Professional development opportunities (training, certifications, conferences)
  • Paid cloud developer accounts
  • Referral Bonus - good people know good people
  • HQ Office Perks (parking / metro reimbursement, nitro coffee & lunches) 
  • Annual Social Events (540 week, hackathon, charity golf tournament, etc.)
  • Access to 540’s Washington Capitals & Nationals tickets

EQUAL EMPLOYMENT OPPORTUNITY (EEO)

540's policy is to provide equal employment opportunity to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.

Similar Jobs

6 Hours Ago
Remote or Hybrid
USA
140K-215K Annually
Senior level
140K-215K Annually
Senior level
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Architects, deploys, and operates scalable data platform infrastructure supporting analytics. Responsibilities include administering Airflow and Superset, managing Terraform infrastructure, building CI/CD pipelines, implementing security and compliance controls, developing dbt data models, cataloging assets in OpenMetadata, automating with Python, and mentoring engineers. The role requires extensive infrastructure and data engineering experience, cloud expertise, container orchestration, and eligibility for CJIS clearance.
Top Skills: Ai TechnologiesApache AirflowApache SupersetAWSAzureCi/CdDbtDockerGCPIamKubernetesMachine Learning PipelinesOciOpenmetadataPythonSQLTerraform
Yesterday
In-Office or Remote
120K-215K Annually
Senior level
120K-215K Annually
Senior level
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Build and lead production AI/ML, Generative AI, and data engineering solutions for healthcare analytics and patient safety platforms. Develop full-stack applications, microservices, APIs, data pipelines, dashboards, self-service analytics, RAG systems, and event-driven architectures using cloud-native technologies. Apply engineering best practices including testing, CI/CD, observability, security, and performance optimization. Mentor engineers, lead technical initiatives, and collaborate with product, analytics, business, and platform teams.
Top Skills: Amazon KinesisAWSAzureAzure Event HubsCi/CdDatabricksDelta LakeGCPGenerative AiJavaJavaScriptKafkaLarge Language Models (Llms)Microsoft FabricMlopsPysparkPythonReactRest ApisRetrieval-Augmented Generation (Rag)SnowflakeSparkSpring BootSQLTypescriptVector Search
Yesterday
Remote or Hybrid
US
105K-150K Annually
Senior level
105K-150K Annually
Senior level
Information Technology
Designs, develops, validates, and documents analytics data solutions for diverse client environments. Builds Snowflake, SQL, dbt, and AWS ingestion and transformation pipelines; monitors data quality and performance; troubleshoots complex technology issues; and supports reporting and business insights. Manages project scope, budgets, documentation, governance, client communication, implementation planning, and quality assurance while collaborating across teams and identifying opportunities for additional solutions.
Top Skills: Analytics Data MartsAWSData GovernanceData IngestionData PipelinesData PrivacyData TransformationDbtMedallion ArchitectureQa TestingRelational DatabasesSnowflakeSQL

What you need to know about the NYC Tech Scene

As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.

Key Facts About NYC Tech

  • Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
  • Key Industries: Artificial intelligence, Fintech
  • Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
  • Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account