STEM Xpert Logo

STEM Xpert

Big Data Engineer

Reposted 21 Days Ago
Be an Early Applicant
In-Office
New York, NY, USA
Senior level
In-Office
New York, NY, USA
Senior level
Design and implement large-scale data processing solutions (GB/PB) using Hadoop, NoSQL, MPP and related open-source tech. Participate in architecture/design, provide implementation details to offshore teams, produce whitepapers/case studies, ensure timely quality deliveries, and share knowledge through training sessions.
The summary above was generated by AI
Company Description

TekWissen provides a unique portfolio of innovative capabilities that seamlessly combines clients insights, strategy, design, software engineering and systems integration. Our tightly integrated offerings are tailored to each clients requirements and span the services spectrum from Application Development/Maintenance testing, IT Consulting & staffing for IT Infrastructure Management through strategic consulting and industry-oriented business process.

Job Description

Primary Skills:

1. Very strong server-side Java experience, especially in an open source, data-intensive, distributed environments.

2. Should have worked on open source products and contribution towards it would be an added advantage.

3. Implemented and in-depth knowledge of various java, J2EE and EAI patterns.

4. Implemented complex projects dealing with the considerable data size (GB/ PB) and with high complexity

5. Well aware of various Architectural concepts (Multi-tenancy, SOA, SCA etc) and NFR’s (performance, scalability, monitoring etc)

6. Good understanding of algorithms, data structure, and performance optimization techniques. 

7. Knowledge of database principles, SQL, and experience working with large databases beyond just data access.

8. Exposure to complete SDLC and PDLC.

9. Capable of working as an individual contributor and within team too.

10. Self-starter & resourceful personality with ability to manage pressure situations

11. Should have experience/ knowledge on working with batch processing/ Real time systems using various Open source technologies like Solr, hadoop, NoSQL DB’s, Storm, kafka etc.

Role & Responsibilities

• Implementation of various solutions arising out of the large data processing (GB’s/ PB’s) over various NoSQL, Hadoop and MPP based products

• Active participation in the various Architecture and design calls with bigdata customers. 

• Working with Sr. Architects and providing implementation details to offshore. 

• Conducting sessions/ writing whitepapers/ Case Studies pertaining to BigData

• Responsible for Timely and quality deliveries.

• Fulfill organization responsibilities – Sharing knowledge and experience within the other groups in the org., conducting various technical sessions and trainings

Additional Information

Thanks & Regards,

Aravind Jakku

aravind.j-tekwissen.com

734-619-8931

Similar Jobs

2 Days Ago
Hybrid
New York, NY, USA
170K-180K Annually
Senior level
170K-180K Annually
Senior level
AdTech • Big Data • Digital Media • Software
Design, deploy, administer, and optimize large-scale, highly available Big Data platforms (Kafka, Hadoop/MapR, Aerospike). Perform capacity planning, performance tuning, monitoring, automation, production troubleshooting, and participate in on-call rotations while collaborating with SRE, Infrastructure, Database, and Engineering teams.
Top Skills: AerospikeAnsibleApache KafkaBashCentosCephClouderaDnsGrafanaGraphiteHadoopKubernetesLoad BalancingMaprMySQLPostgresPrometheusPuppetPythonRhelRocky LinuxScylladbSparkTcp/IpTerraform
7 Days Ago
Hybrid
New York, NY, USA
105K-190K Annually
Senior level
105K-190K Annually
Senior level
AdTech • Digital Media • Marketing Tech • Software • Analytics
Design, implement, and maintain scalable big-data pipelines and ETL on cloud (GCP). Build production data solutions using Python, Spark, Scala, Airflow, Kafka, and BigQuery; collaborate with data scientists and product, estimate and scope projects, drive data engineering best practices, and mentor teammates.
Top Skills: AirflowGCPGisGoogle BigqueryGraph DatabasesKafkaNumpyPandasPythonScalaScikit-LearnScipySparkSQL
15 Days Ago
In-Office or Remote
United States
Senior level
Senior level
Agency • Information Technology
Design and implement large-scale data applications using Java, Spark, and MapReduce within Cloudera/Hadoop ecosystems. Integrate Hive, Impala, YARN, and Kafka, perform data analysis, write tests (JUnit), use scripting, and manage global technology teams following industry best practices.
Top Skills: SparkClouderaHadoopHiveImpalaJava 1.8JunitKafkaMapreducePythonUnix ShellYarn

What you need to know about the NYC Tech Scene

As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.

Key Facts About NYC Tech

  • Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
  • Key Industries: Artificial intelligence, Fintech
  • Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
  • Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account