Senior Hadoop Infrastructure Engineer - Data & Analytics Platform Team

| Greater NYC Area
Sorry, this job was removed at 11:33 a.m. (EST) on Monday, May 14, 2018
Find out who's hiring in Greater NYC Area.
See all Developer + Engineer jobs in Greater NYC Area
Apply
By clicking Apply Now you agree to share your profile information with the hiring company.

Many of the most prevalent applications in Bloomberg need real-time data analysis and business intelligence. Instead of using off-the-shelf applications, we use Hadoop and its ecosystem to provide a large-scale data platform with low-latency SLAs and large storage capabilities. This platform has revolutionized the way we manage and analyze data in a distributed environment.

We design software and hardware systems to support low-latency/high volume requests, security, fault tolerance/high availability and easy customization. Our Hadoop Infrastructure Platform is built to fully automate deployment and operations using Chef; developed and open sourced at https://github.com/bloomberg/chef-bach. With hundreds of applications depending on our platform, we are looking to grow our Hadoop Infrastructure team. That's where you come in.

We'll trust you to:

  •  Evaluate Hadoop projects across the ecosystem and extend and deploy them to exacting standards (high availability, big data clusters, elastic load tolerance)
  •  Develop automation, installation and monitoring of Hadoop ecosystem components in our open source infrastructure stack, specifically HBase, HDFS, Map/Reduce, Yarn, Oozie, Pig, Hive, Tez, Spark and Kafka
  •  Dig deep into performance, scalability, capacity and reliability problems to resolve them
  •  Create application patterns for integrating internal application teams and external vendors into our infrastructure
  •  Troubleshoot and debug Hadoop ecosystem run-time issues
  •  Provide developer and operations documentation to educate peer teams

You'll need to have most of the following:

  •  Experience building out and scaling a Hadoop-based or UNIX-hosted database infrastructure for an enterprise
  •  2+ years of experience with Hadoop infrastructure or a strong and diverse background of distributed cluster management and operations experience
  •  Experience writing software in a continuous build and automated deployment environment

We'd love to see:

  •  2+ years of DevOps or System Administration experience using Chef/Puppet/Ansible for system configuration, or quality shell scripting for systems management (error handling, idempotency, configuration management)
  •  In-depth knowledge of low-level Linux, UNIX networking and C system calls for high performance computing
  •  Experience with Java, Python or Ruby development (including testing with standard test frameworks and dependency management systems, knowledge of Java garbage collection fundamentals)
  •  Experience or exposure to the open source community (a well-curated blog, upstream accepted contribution or community presence)

We want to work with others who are passionate about community-driven development both within the company and with the wider open source community. If this sounds like you, submit an application, and learn more about the work we do from Clay and Amit's interview at ChefConf2015: https://www.youtube.com/watch?v=LnMCFxXgDEw

Read Full Job Description
Apply Now
By clicking Apply Now you agree to share your profile information with the hiring company.

Location

731 Lexington Avenue, New York, NY 10022

Similar Jobs

Apply Now
By clicking Apply Now you agree to share your profile information with the hiring company.
Learn more about BloombergFind similar jobs