28 Oct
Sourcebae
Thiruvananthapuram
Job Title: Solution Architect Lead Data Engineer
Experience: 10 -15 Years
Contract/ Remote
About Us:
We are leading Solution Service company that provide Services the field of Data
Science, Big Data, Enterprise Cloud & Automation. We are at the forefront of leveraging cutting-
edge technologies to drive innovation and enhance our business processes. As part of our
commitment to staying ahead in the industry, we are seeking a talented and experienced Data
& AI Engineer with strong Azure cloud competencies to join our dynamic team.
Role Overview:
Deliver successful projects in customer environments to bring use cases into production,
machine learning projects and large migrations, in order to deliver on value proposition.
Key Responsibilities
* ARCHITECTURE AND DESIGN FOR DATA ENGINEERING AND MACHINE LEARNING PROJECTS
Establishing architecture and target design for data engineering and machine learning projects.
* REQUIREMENT ANALYSIS, PLANNING, EFFORT AND RESOURCE NEEDS ESTIMATION
Current inventory analysis, review and formalize requirements, project planning and execution
plan.
* ADVISORY SERVICES AND BEST PRACTICES
Troubleshooting, Performance Tuning, Cost Optimization, Operational Runbooks and Mentoring
* LARGE MIGRATIONS
Assist customers with large migrations to Databricks from Hadoop ecosystems, Data
Warehouses (Teradata, DataStage, Netezza, Ab Initio), ETL engines (Informatica), SAS, SQL, DW,
Cloud-based Data platforms like Redshift, Snowflake, EMR, etc
* DESIGN, BUILD AND OPTIMIZE DATA PIPELINES
The Databricks implementation will be best in class, with flexibility for future iterations.
* PRODUCTION READINESS
Assisting with production readiness for customers,
including exception handling, production
cutover, capture analysis, alert scheduling and monitoring
* MACHINE LEARNING (ML) – MODEL REVIEW, TUNING, ML OPERATIONS AND OPTIMIZATION
Build and review ML models, ML best practices, model lifecycle, ML frameworks and deploying
of models in production.
Must Have:
▪ Hands on experience with distributed computing framework like DataBricks, Spark-
Ecosystem (Spark Core, PySpark, Spark Streaming, SparkSQL)
▪ Willing to work with product teams to best optimize product features/functions.
▪ Experience on Batch workloads and real time streaming with high volume data
frequency.
▪ Performance optimization on Spark workloads
▪ Environment setup, user management,
Authentication and cluster management on
Databricks
▪ Professional curiosity and the ability to enable yourself in new technologies and tasks.
▪ Good understanding of SQL and a good grasp of relational and analytical database
management theory and practice.
Key Skills:
* Python, SQL and Pyspark
* Big Data Ecosystem (Hadoop, Hive, Sqoop, HDFS, Hbase)
* Spark Ecosystem (Spark Core, Spark Streaming, Spark SQL) / Databricks
* Azure (ADF, ADB, Logic Apps, Azure SQL database, Azure Key Vaults, ADLS, Synapse)
* AWS (Lambda,AWS Glue, S3, Redshift)
* Data Modelling, ETL Methodology.
Basically they are shifting there data from old data warehouse to Redshift . The candidate they're looking for must have strong skills in implementation, data governance,
data modeling, presales, AWS, and Databricks. They should also be able to lead a team at an Architect level.
If Intrested. Please submit your CV to or share it via WhatsApp at 91094 36045
Stay updated with our latest job opportunities and company news by following us on : :
▶️ Solutions Architect
🖊️ Sourcebae
📍 Thiruvananthapuram