Job Description

:Data Engineer - Ingestion::We are seeking a highly skilled and experienced Data Engineer with strong proficiency in Azure and ETL (Extract, Transform, Load) skills to join our team. As a Senior Data Engineer, Team needs to play a critical role in designing, building, and maintaining scalable data pipelines and data solutions in Azure. Expertise in Azure and ETL will be essential in ensuring the efficient and accurate processing of large volumes of data. Team member should collaborate closely with cross-functional teams, including data scientists, data analysts, and software engineers, to support data-driven decision-making and enable advanced analytics.Qualifications:Responsibilities: * Design scalable, performant, and cost-effective data storage systems to meet organizational needs.

• Good experience in Azure services such as Azure Data Factory, Azure Databricks(PySpark or Scala), Azure SQL Data Warehouse, and CosmosDB.
• Experience in Snowflake (Azure)
• Extract, transform, and load (ETL) data from various sources such as databases, APIs, and flat files into Modern Cloud data storage systems(Azure) .
• Sound knowledge ETL/ELT concepts
• Optimize data extraction, transformation, and loading processes to improve performance and reduce latency.
• Monitor and troubleshoot data pipeline issues, and proactively identify and resolve performance bottlenecks and scalability challenges.
• Good in Relational & NO-SQL Concepts
• Experience in Relational & NO-SQL databases
• Collaborate with data scientists and analysts to understand data requirements and transform raw data into usable formats for analysis and reporting.
• Ensure data quality and consistency by implementing data validation and cleansing processes within the ETL workflows.
• Implement data governance practices, including data security, privacy, and compliance measures, in line with organizational policies and industry best practices.
• Stay up-to-date with the latest Azure technologies, tools, and best practices for data engineering and ETL, and recommend and implement improvements and enhancements to existing processes.
• Collaborate with software engineers and database administrators to ensure the smooth integration of data pipelines with other systems and databases.
• Mentor and provide guidance to junior data engineers, promoting knowledge sharing and fostering a culture of continuous learning and improvement.

Chubb