Scala - Spark Developer

Overview

We are looking for a Scala - Spark Developer to join our regulatory reporting data engineering team. The application processes very large datasets, with business logic implemented using Scala/Spark workloads on Databricks and Snowflake as the core data platform. The selected candidate will work closely with senior engineers to develop data pipelines, implement controls, and support testing and operational activities.

What you'll do in the role:

• Assist in developing and enhancing Spark-based dat pipelines written in Scala on Databricks

• Implement and maintain business logic for regulatory reporting under guidance of senior team members

• Develop automation utilities and operational controls using Python

• Work with datasets stored in Snowflake including data transformations and validations

• Support batch workflow setup and maintenance using Autosys scheduler

• Write and maintain BDD test cases and Python unit tests

• Participate in code reviews, design discussions, and Agile ceremonies

• Provide basic support during production issues and assist in root-cause analysis

• Ensure good documentation, coding standards, and quality practices

What you'll bring to the role:

• Bachelor’s degree in Computer Science, Engineering, MIS, or a related field

• 3+ years of experience in software development or data engineering

• Strong foundational knowledge of Scala and familiarity with Apache Spark

• Understanding of cloud data platforms (Snowflake preferred)

• Database knowledge and core database concepts(SQL, indexing, joins, transaction, query optimization)

• Ability to understand and debug spark application(DAGS, Job, Stages, Failure)

• Understanding of spark job optimization techniques

• Working knowledge of Python for scripting, automation, and testing

• Exposure to BDD frameworks and writing unit tests

• Strong understanding of SDLC, coding practices, and Agile methodology

• Good communication skills and willingness to learn complex financial datasets

Desired Skills:

• Basic understanding of Trade Lifecycle concepts

• Awareness of Equities and Options asset classes

• Exposure to Front Office (FO) trade/order data

• Experience with Unix/Linux commands and shell scripting
  Familiarity with batch job schedulers (Autosys is a plus)

• Basic exposure to Generative AI use cases

• Experience with large-scale data environments

Why Join Us?

• Hands-on learning with modern tech stack: Scala, Spark, Databricks, Snowflake

• Opportunity to work on large-scale, high-impact regulatory reporting systems

• Strong mentoring and skill-building environment with senior engineers

• Collaborative & respectful culture focused on growth, learning, and continuous improvement

WHAT YOU CAN EXPECT FROM MORGAN STANLEY:

At Morgan Stanley, we raise, manage and allocate capital for our clients – helping them reach their goals. We do it in a way that’s differentiated – and we’ve done that for 90 years.  Our values - putting clients first, doing the right thing, leading with exceptional ideas, committing to diversity and inclusion, and giving back - aren’t just beliefs, they guide the decisions we make every day to do what's best for our clients, communities and more than 80,000 employees in 1,200 offices across 42 countries. At Morgan Stanley, you’ll find an opportunity to work alongside the best and the brightest, in an environment where you are supported and empowered. Our teams are relentless collaborators and creative thinkers, fueled by their diverse backgrounds and experiences. We are proud to support our employees and their families at every point along their work-life journey, offering some of the most attractive and comprehensive employee benefits and perks in the industry. There’s also ample opportunity to move about the business for those who show passion and grit in their work.

To learn more about our offices across the globe, please copy and paste https://www.morganstanley.com/about-us/global-offices​ into your browser.

Expected base pay rates for the role will be between $90,000 and $150,000 per year at the commencement of employment. However, base pay if hired will be determined on an individualized basis and is only part of the total compensation package, which, depending on the position, may also include commission earnings, incentive compensation, discretionary bonuses, other short and long-term incentive packages, and other Morgan Stanley sponsored benefit programs. 

Morgan Stanley's goal is to build and maintain a workforce that is diverse in experience and background but uniform in reflecting our standards of integrity and excellence. Consequently, our recruiting efforts reflect our desire to attract and retain the best and brightest from all talent pools. We want to be the first choice for prospective employees.

It is the policy of the Firm to ensure equal employment opportunity without discrimination or harassment on the basis of race, color, religion, creed, age, sex, sex stereotype, gender, gender identity or expression, transgender, sexual orientation, national origin, citizenship, disability, marital and civil partnership/union status, pregnancy, veteran or military service status, genetic information, or any other characteristic protected by law.

Morgan Stanley is an equal opportunity employer committed to diversifying its workforce (M/F/Disability/Vet).