Data Engineer
Bloomberg Second Measure is a leading provider of data analytics that deliver valuable insights into company performance and consumer behavior. Using data from billions of anonymized transactions, we have created self-service analytics products for daily tracking and real-time exploration of 5,200+ public and private companies. Clients use our product to discover new markets, gain an advantage in financial investments, and inform their competitive strategies. Second Measure was acquired by Bloomberg in 2020.
As a Data Engineer, your goal is to ingest, transform, and productionize alternative datasets to yield value for our clients. You will extract and process raw datasets using tools such as Spark or Flink and expose it for use by our data scientists. You will partner with the product team, to produce and evolve high quality insights using alternative datasets and deliver these insights to our clients. You will collaborate with data platform engineers to craft and evolve the tools and systems in our platform.
We’ll trust you to:
- Emphasize and ensure reliable and timely delivery of data for our 200+ institutional and corporate clients
- Build and run data pipelines using an orchestration framework such as Argo Workflows, Apache Airflow, Luigi
- Build and maintain data processing systems using data processing frameworks including Apache Spark, Apache Flink, Dask
You’ll need to have:
- 3+ years experience in software development using Python or Scala
- Knowledge and experience with statistics
- Experience processing structured and unstructured data
- Distributed query systems experience with tools such as Presto/Trino, AWS Redshift, Google BigQuery, Snowflake
- Software engineering fundamentals, including participating in Agile, knowledge, and use distributed version control systems such as Git/Mercurial, code reviews, emphasis on quality, and dedication to documentation
- BA, BS, MS, or PhD in Computer Science, Engineering or related technology field, or equivalent experience
We’d love to see:
- Container orchestration systems experience (Kubernetes, Nomad, AWS ECS) and the deployment of distributed data processing frameworks onto these systems
- Experience with building systems in event-driven or streaming architectures
- Experience building data quality checks using tools like great_expectations, dbt, and property-based testing in general
- Highly competitive compensation
- Comprehensive medical, dental, and vision benefits for employees and their families
- Competitive retirement savings plan with a generous company match
- Wide-ranging support for physical and mental wellbeing
- Generous gender-neutral parental leave
- Pre-tax FSAs and commuter benefits
We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.