ConsultAdd Inc Logo

ConsultAdd Inc

Hadoop Developer

Posted An Hour Ago
Be an Early Applicant
In-Office
New York, NY, USA
Mid level
In-Office
New York, NY, USA
Mid level
Design, develop, and productionize Hadoop-based applications and ETL pipelines. Build and populate data lakes, implement MapReduce/Hive/Pig jobs, manage large-scale datasets, collaborate with business partners to translate requirements into use cases, and mentor users.
The summary above was generated by AI
Company Description

ConsultAdd Inc , is leading provider of Information Technology,Consulting and Business process services. Our dedication employees offer strategic insights, technological expertise and industry experience .Work supported by a global delivery network that helps large organisation responds to challenges of future of work with flexible and efficient operations.

Website:- www.consultadd.com

Job Description

Job Description 

This position sits within our Enterprise Data and Analytics (EDA) Division focusing on Big Data. The EDA Big Data platform supports Hadoop and Aster technology in a multi-tenant environment supporting multiple LOBs.

The EDA Big data team is looking for an Application System Engineer to work in a fast-paced agile development environment to act as a developer on the Hadoop development team; the resource should be able to design and build applications using Big data ecosystem and help with data management (tracking of sources in the big data environment), system documentation, metadata capture quickly analyze, develop, and test potential use cases for the business; take valid use cases from ideation through development to production; write efficient code to extract, transform, load, and query very large datasets, including unstructured data; develop standards and new design patterns for Big Data applications; build out and populate a data lake; understand MapReduce architecture and master the tools and technology components within the Hadoop and Aster environments; help formulate use cases from potentially ambiguous business requirements; provide timely communication to business partners on use cases and project status; mentor and assist users in accessing the data; contribute significantly toward developing a development framework on Big Data.

Required Qualifications

  • 2+ years of application development and implementation experience
  • 1+ years of Hadoop experience
  • 3+ years of UNIX experience
  • 3+ years of Java experience
Desired Qualifications
  • Good verbal, written, and interpersonal communication skills
  • Basic knowledge in data warehouse, Teradata, Oracle, DB2, Agile & ETL (Extract, Transform, Load) programming.

Other Desired Qualifications

  • Bachelor degree in Computer Science or related technological degree.
  • Experience with MapReduce, HDFS, Hive or Pig on a major distribution such as Hortonworks or Cloudera or MapR
  • Experience in large scale data warehousing, distributed computing, and parallel processing environments
  • Experience working with business partners on analytical solutions
  • Experience building HBase, Spark, Kafka solutions
  • Hadoop developer certification
  • Experience with unstructured datasets, such as log files, email, text

Additional Information

We have started the process for sponsorship.

HQ

ConsultAdd Inc New York, New York, USA Office

175 Greenwich St, 38th floor c/o Hana by Industrious, New York, NY, United States, 10007

Similar Jobs

53 Minutes Ago
In-Office
New York, NY, USA
Senior level
Senior level
Information Technology • Professional Services • Consulting
Design, develop, debug and tune Hadoop-based ETL using HDFS, Pig, Hive, HBase, Sqoop and Java. Provide technical oversight to the Hadoop team, write/modify shell or Python scripts, liaise with application teams, perform performance tuning and troubleshooting, and provide on-call weekend support.
Top Skills: HadoopHbaseHdfsHiveJavaPigPythonShellSqoop
53 Minutes Ago
In-Office
New York, NY, USA
Mid level
Mid level
Information Technology • Professional Services • Consulting
Develop, debug, and optimize Hadoop-based data solutions using Pig, Hive, Sqoop, HBase and Java. Write and maintain shell/Python scripts, perform performance tuning, apply Big Data architecture/design patterns, guide the Hadoop team, coordinate with application teams for issue resolution, and provide on-call weekend support.
Top Skills: HadoopHbaseHdfsHiveJavaPigPythonShell ScriptSqoop
3 Hours Ago
In-Office
100K-150K Annually
Senior level
100K-150K Annually
Senior level
Artificial Intelligence • Information Technology • Software • Consulting
Design, build, and operate large-scale Hadoop-based data pipelines and streaming systems. Implement ETL/ELT workflows, optimize Spark/MapReduce jobs, manage data storage and lakehouse formats, enforce data governance and monitoring, automate orchestration, and mentor engineers to deliver reliable, production-grade analytics platforms.
Top Skills: AirflowApache AtlasSparkAws EmrBigQueryCi/CdCollibraDatabricksDeltaFlinkHadoopHbaseHdfsHiveHudiIcebergInfrastructure-As-CodeJavaKafkaKubernetesMapreduceNoSQLOozieOrcParquetPigPythonScalaShellSnowflakeSpark StreamingSQLSqoopTrino

What you need to know about the NYC Tech Scene

As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.

Key Facts About NYC Tech

  • Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
  • Key Industries: Artificial intelligence, Fintech
  • Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
  • Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account