21 Oct
Delphic (South Asia)
Nadiad
Job Summary:
We are seeking an experienced Azure Data Engineer with 3-5 years of expertise in designing, building, and maintaining scalable data pipelines and architectures using Microsoft Azure's cloud platform. The candidate will work closely with data scientists, data analysts, and software developers to ensure high-quality data solutions. The ideal candidate should have a solid background in Azure cloud services, data warehousing, ETL processes, and strong programming skills.
Key Responsibilities:
1. Design and Implementation of Data Solutions:
- Develop, optimize, and maintain data pipelines using Azure Data Factory (ADF), Databricks, and other Azure data services.
- Create scalable and high-performance data architectures to support business needs.
- Design, build, and maintain efficient ETL/ELT processes to ensure reliable data flow from various data sources to target destinations.
1. Data Integration and Processing:
- Integrate data from multiple sources such as APIs, databases, flat files, and third-party applications into Azure-based solutions.
- Develop data models, schemas, and strategies to ensure data quality and integrity across all stages of processing.
- Collaborate with data teams to deploy and manage data lake, data warehouse, and data mart solutions in Azure Synapse, Azure SQL Database, and/or Azure Data Lake.
1. Azure Services Management:
- Utilize Azure Data Factory, Azure Databricks, Azure SQL, Azure Synapse Analytics, and Azure Data Lake Storage to build robust data solutions.
- Implement data security measures and ensure that Azure data services adhere to compliance and governance standards.
1. Monitoring & Optimization:
- Monitor, troubleshoot, and optimize data pipelines, ensuring reliability, performance,
and efficiency.
- Automate data-related tasks where possible to improve pipeline efficiency.
1. Collaboration & Support:
- Work closely with cross-functional teams including data scientists, analysts, and business stakeholders to provide data support and recommendations.
- Collaborate with DevOps teams to integrate data solutions into continuous integration/continuous delivery (CI/CD) pipelines.
1. Documentation and Best Practices:
- Maintain comprehensive documentation of data engineering processes and pipelines.
- Promote best practices for data engineering, architecture, and security within the team.
Required Skills and Experience:
- Education: Bachelor's degree in Computer Science, Information Technology, or related fields.
- Experience:
3-5 years of hands-on experience in Azure Data Engineering roles.
Technical Skills:
1. Azure Data Services: Strong experience with Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Synapse Analytics, and Azure Data Lake Storage.
2. Programming: Proficient in Python, SQL, and/or Scala for data manipulation and transformation.
3. Data Warehousing & ETL: Experience with building data lakes and data warehouses, and managing ETL/ELT pipelines.
4. Database Technologies: Knowledge of relational and NoSQL databases (e.g., SQL Server, Cosmos DB, etc.).
5. Big Data Processing: Familiarity with Apache Spark or Hadoop for large-scale data processing.
6. Version Control & CI/CD: Knowledge of version control systems like Git and working experience with CI/CD pipelines (Azure DevOps).
7.
Data Security: Understanding of Azure data security, including encryption, access control, and data governance.
8. Monitoring & Performance Tuning: Experience with monitoring and tuning Azure resources for performance and cost-efficiency.
Preferred Qualifications:
- Certifications: Azure Data Engineer Associate, Azure Fundamentals, or any relevant Azure certifications.
- Experience with BI tools like Power BI or Tableau.
- Exposure to Agile/Scrum development methodologies.
- Familiarity with data governance frameworks and policies.
Soft Skills:
- Strong problem-solving skills with attention to detail.
- Ability to collaborate with cross-functional teams and communicate technical solutions to non-technical stakeholders.
- Eagerness to learn new technologies and apply them to solve business problems.
- Ability to manage multiple priorities in a fast-paced environment.
Why Join Us:
- Opportunity to work on cutting-edge technologies.
- Collaborative and growth-focused work environment.
- Competitive salary and benefits.
▶️ Azure Data Engineer
🖊️ Delphic (South Asia)
📍 Nadiad