IRIS Software Pvt Ltd
Big Data Developer - ( 20000059 )
We are looking for a Spark developer who knows how to fully exploit the potential of our Spark cluster.
You will clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data to our feature developers and business analysts.
This involves both ad-hoc requests as well as data pipelines that are embedded in our production environment.
Knowledge of Java and Python is Preferred.
Create Scala/Spark jobs for data transformation and aggregation
Produce unit tests for Spark transformations and helper methods
Write Scaladoc-style documentation with all code
Design data processing pipelines
Scala ,Scalatest, JUnit, Mockito
Apache Spark 2.x
Apache Spark RDD API
Apache Spark SQL DataFrame API
Apache Spark MLlib API
Apache Spark GraphX API
Apache Spark Streaming API
Spark query tuning and performance optimization
SQL database integration Microsoft, Oracle, Postgres, and/or MySQL
Experience working with HDFS, S3, and/or MangoDB, middlewares like MQ solace
Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)
Primary Location : INDIA
Organization : Iris Software
Schedule : Regular
Shift : Standard
Job Type : Full-time