12 Oct
Manuh Technologies
Tumakuru
Position - Big Data Engineer
Location - Pune/Hyderabad/Any
Experience - 5+
Job Summary
- As a Big data engineer responsibilities include - review and analysis of requirements, design workshop participation and facilitation, design creation and documentation, data pipeline implementation, testing, bug fixing, release support and post release aftercare.
- You will be part of a team responsible for delivering the highest quality data processing and analytical systems, drawing upon your engineering and coding expertise, whilst being open minded to the opportunities the cloud provides. You will deliver code, facilitate solution designs and identify potential technical impediments then work to resolve them.
Roles & Responsibilities
- Devise technical solution options aligned to the strategic architecture.
- Design data models / structures to support capital modelling and reporting processes.
- Influence the design and implementation of batched processes to create model inputs and execute models
- Liaise with other projects to achieve synergies through mutually beneficial design / delivery decisions.
- Act as a technical mentor/coach to developers to help them deliver solutions that meet customer requirements and architectural standards.
- Be available to discuss and resolve technical issues with individuals within and without the team.
- Help to embed the use of new technologies to deliver project goals.
- Help to ensure work carried out is in line with quality standards
- Must Have:
·Strong knowledge of Hadoop/Big Data concepts
·Experience in working with Cloudera clusters is a plus
·Experience in Analysing data sets and creating data models using SQL
·Good working knowledge of Apache Spark with Java/Scala
·Knowledge of Spark Streaming with Kafka
·Must have hands on experience in building ETL pipelines using Apache NiFi
·Must have hands on experience in Impala with Kudu
·Experience in writing transformations using SQL
·Experience in monitoring, debugging and troubleshooting Spark Jobs/Impala Queries in the cluster
·Basic knowledge of Unix shell scripting
·Good to have knowledge of Spring boot, Spring MVC, REST APIs and Microservices.
·Good to have knowledge of AWS Cloud (EC2, IAM, S3, Lambda, CloudWatch etc.)
Skills: Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scripting
▶️ Big Data Engineer
🖊️ Manuh Technologies
📍 Tumakuru