Job Description

5-8 years of overall experience mainly in the data engineering space building data ingestion and transformation pipelines.


• Design and implement data pipelines using best practices and industry-leading tools like Databricks and Azure Data Factory (ADF)
• Extract, transform, and load large datasets from various sources, ensuring data quality and integrity
• Utilize Python and Spark to perform complex data manipulations and aggregations
• Write optimized SQL queries to interact with relational databases and Cosmos DB
• Integrate data pipelines with APIs and external systems using efficient methods
• Monitor and maintain data pipelines, ensuring smooth operation and identifying potential issues
• Collaborate with data scientists, analysts, and engineers to understand data needs and deliver valuable insights
• 4+ years of Hands-on experience in Azure cloud data implementation.
• Must have implementation experience of Azure based cloud Data project/program as solution architect.
• Expert level knowledge on Spark. Extensive hands-on experience working with data using SQL, Python, Java, Scala
• Strong experience and understanding of very large-scale data architecture, solutioning, and operationalization of data warehouses, data lakes, and analytics platforms. (Both Cloud and On-Premise)
• Excellent communications skills with the ability to clearly present ideas, concepts, and solutions
• Bonus points if you have experience on Insurance Domain for data engineering project.