Job Description

Position: Senior Engineer Azure databricksOverall IT experience: 6 to 9 yearsPrimary Skills: Azure Databricks, PySpark, SQLSecondary Skills: ADF/Azure Data factoryWork Location: Bengaluru/HyderabadMode of work: HybridNotice Period: 0 to 30 DaysExperience & Expertise:

• Total Experience: 6 to 9 years in Cloud Data Engineering.
• Azure Databricks & PySpark: Minimum 3 to 5 years of hands-on experience.

Technical Skills & Responsibilities:

• SQL Expertise:
• Proficient in SQL with 5 to 6 years of experience in complex joins, subqueries, functions, and procedures.
• Strong ability to perform SQL tuning and query optimization independently.
• ETL & Data Processing:
• Design, develop, and maintain ETL pipelines using Databricks, PySpark, and Azure Data Factory (ADF).
• Extract, transform, and load (ETL) data from various sources.
• Optimize and fine-tune existing ETL workflows for performance and scalability.
• Azure Cloud Expertise:
• Hands-on experience in implementing large-scale hosted solutions using Azure ADF, Azure Data Lake Storage, and Databricks.
• Big Data & Streaming:
• Experience working with large datasets.
• Exposure to batch and streaming data processing (good to have).
• Security & Compliance:
• Ability to develop reliable, secure, and compliant data processing systems.
• Collaboration & Independence:
• Work closely with cross-functional teams to support data analytics, reporting, and business intelligence initiatives.
• Self-driven and capable of working independently without support.

Expertia AI Technologies