Microsoft Logo

Microsoft

Data Engineer

Posted 7 Days Ago
Remote
Hiring Remotely in United States
102K-219K Annually
Junior
Remote
Hiring Remotely in United States
102K-219K Annually
Junior
Designs, develops, and maintains secure, scalable data pipelines, curated datasets, and data products supporting analytics and AI. Uses PySpark, SQL, Microsoft Fabric, Azure Synapse, dimensional modeling, and lakehouse architectures. Ensures data quality, governance, security, monitoring, performance, and production reliability while applying software engineering and DevOps practices. Partners with analysts, engineers, data scientists, and business stakeholders to translate requirements into trusted data solutions and improve platform efficiency.
The summary above was generated by AI
Overview

MCAPS-Core accelerates customer outcomes and business growth. By uniting product, engineering, marketing, sales, customer success, and partners around a common customer mission, we help customers realize value faster and turn innovation into measurable business impact. Through deep technical expertise and differentiated go-to-market execution, we win competitively, capture market share, and scale repeatable growth motions. We differentiate through value creation - delivering world-class, AI-powered customer experiences that accelerate adoption, strengthen loyalty, and create sustainable competitive advantage, driving growth that outpaces the market.

MCAPS-Core Data & Applied AI (DAAI) is a centralized data and AI organization driving Microsoft's transformation through enterprise data platforms, advanced analytics, AI-powered solutions, automation, and governance. Our mission is to turn data into intelligence and intelligence into action, empowering teams across Microsoft to accelerate decision-making, increase productivity, and deliver exceptional customer outcomes through innovative customer-zero solutions.

MCAPS-Core DAAI is seeking a Data Engineer to design and build the scalable data foundations, pipelines, and data products that power analytics, AI, and business insights across Microsoft. This role combines strong data engineering and software engineering skills to deliver reliable, high-quality solutions that integrate enterprise data and enable trusted, actionable insights. The ideal candidate is a hands-on engineer who enjoys solving complex data challenges, building reusable and automated solutions, and partnering across engineering, analytics, data science, and product teams to deliver measurable business impact.

Candidates who live within 50 miles of Microsoft's Redmond campus will be required to work in the office 3 days per week. 

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.


Responsibilities
  • Design, develop, test, and maintain secure, scalable data pipelines and processing solutions supporting UDP and enterprise analytics workloads.
  • Develop data engineering solutions using PySpark, SQL, Spark notebooks, and distributed data processing technologies across Microsoft Fabric and Azure Synapse environments.
  • Build and maintain curated data models and reusable datasets, applying sound dimensional modeling, data transformation, and data architecture practices.
  • Implement automated data quality, validation, monitoring, and production support capabilities to ensure reliable and trustworthy data products.
  • Optimize data pipelines and processing workloads for performance, scalability, reliability, and efficient resource utilization.
  • Apply engineering best practices including source control, code reviews, testing, CI/CD, documentation, and maintainable development patterns.
  • Implement appropriate security, privacy, access controls, and data governance requirements when processing sensitive enterprise and customer data.
  • Troubleshoot data pipeline, transformation, and production issues, performing root-cause analysis and implementing durable fixes.
  • Partner with analysts, engineering teams, and business stakeholders to translate reporting and analytics requirements into well-designed data solutions, communicate progress and technical tradeoffs, and resolve data issues.
  • Contribute to the continuous improvement of the UDP engineering platform by identifying opportunities to simplify pipelines, improve data architecture, increase reuse, and reduce operational complexity.

Other

  • Embody our culture and values.

Qualifications
Required/minimum qualifications 
  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 1+ year(s) experience in business analytics, data science, software development, data modeling, or data engineering
    • OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 2+ years experience in business analytics, data science, software development, data modeling, or data engineering
    • OR equivalent experience.
Other Requirements
  • Ability to meet Microsoft, customer and / or government security screening requirements are required for this role.  These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire / transfer and every two years thereafter.
Additional or preferred qualifications
  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ years experience in business analytics, data science, software development, data modeling, or data engineering
    • OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 5+ years experience in business analytics, data science, software development, data modeling, or data engineering
    • OR equivalent experience.
  • 1+ year(s) experience with data governance, data compliance and/or data security.
  • 3+ years of experience in data or analytics engineering, building and supporting production data solutions.
  • Experience developing Apache Spark/PySpark notebooks for large-scale data ingestion, transformation, enrichment, and processing.
  • Experience with Azure Synapse Analytics or similar distributed data platforms, including secure ETL and data-processing workloads.Proficiency in SQL and dimensional data modeling, transforming complex source data into curated datasets optimized for analytics and Power BI.
  • Experience building reliable production data pipelines, including validation, error handling, performance optimization, monitoring, and troubleshooting.
  • Ability to translate business and reporting requirements into scalable data solutions and collaborate across data engineering, BI, analytics, and architecture teams.
  • Hands-on experience with Microsoft Fabric Data Engineering, including Lakehouses, notebooks, Delta tables, OneLake, and SQL analytics endpoints.
  • Experience designing medallion/lakehouse architectures across raw, curated, and consumption-ready data layers.Understanding of enterprise data security and governance, including identity-based access, environment separation, and handling of sensitive data.
  • Experience with software engineering and DevOps practices for data solutions, including Git, code reviews, testing, CI/CD, and deployment across development, UAT, and production.
  • Experience using AI-assisted development tools and reusable AI skills to improve data engineering productivity, code quality, testing, and troubleshooting.

Data Engineering IC3 - The typical base pay range for this role across the U.S. is USD $102,100 - $202,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $133,800 - $219,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay


This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.



Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

Similar Jobs

Yesterday
In-Office or Remote
92K-164K Annually
Senior level
92K-164K Annually
Senior level
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Lead migration of on-premises ETL and database workloads to Azure and Snowflake. Design scalable data platforms using ADF, Databricks, ADLS, Logic Apps, and Python; modernize SSIS and T-SQL solutions; and develop AI-powered applications using LLMs, semantic search, vector databases, and RAG architectures. Collaborate independently across teams, optimize data operations, and support governance, quality, and advanced analytics initiatives.
Top Skills: Anthropic ClaudeAWSAzure Ai SearchAzure Ai StudioAzure Blob StorageAzure CloudAzure Data FactoryAzure Data Lake Storage Gen2Azure Machine LearningAzure OpenaiDatabricksDocument IntelligenceGoogle Cloud PlatformLarge Language ModelsLogic AppsMlflowMlopsOpenaiPrompt EngineeringPythonRetrieval-Augmented GenerationSelf-Hosted Integration RuntimeSemantic SearchSnowflakeSparkSsisT-SqlVector Databases
Yesterday
Remote or Hybrid
CA, USA
123K-223K Annually
Senior level
123K-223K Annually
Senior level
Blockchain • Fintech • Mobile • Payments • Software • Financial Services
Build and maintain sales compensation systems, data pipelines, integrations, data models, and commission calculation logic. Translate compensation plans into tested technical configurations, ensure reconciliation and historical traceability, and implement access controls, audit trails, monitoring, and release practices. Partner with Sales Compensation Operations, Finance, Payroll, and technical teams to deliver accurate, explainable commission results.
Top Skills: APIsCaptivateiqCloud Data WarehousesLookerExcelPigmentSalesforceSpiffSQLXactly
2 Days Ago
In-Office or Remote
CA, USA
123K-223K Annually
Senior level
123K-223K Annually
Senior level
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Build and maintain the technical foundation for sales compensation, including commission systems, integrations, data pipelines, calculation logic, data models, testing, access controls, audit trails, reconciliation, and release practices. Partner with Sales Compensation Operations, Finance, Payroll, and technical teams to implement approved compensation plans and ensure accurate, explainable, reproducible commission results.
Top Skills: Access ControlsAPIsAudit TrailsAutomated TestingData ModelingData PipelinesExcelLookerPigmentSalesforceSQLVersion Control

What you need to know about the NYC Tech Scene

As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.

Key Facts About NYC Tech

  • Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
  • Key Industries: Artificial intelligence, Fintech
  • Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
  • Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account