Job Description

####

About the job Azure Databricks

###

: Data Engineer (Azure Databricks & PySpark)

Position:

Data Engineer

Experience:

3 to 5 years

Primary Skills:

Azure Databricks, PySpark, SQL

Secondary Skills:

ADF (Azure Data Factory)

Location:

Bengaluru/Hyderabad

Mode of Work:

Hybrid

Notice Period:

0 to 30 Days


###

Key Responsibilities:

ETL Development & Data Processing:

Design, develop, and maintain

ETL pipelines

using

Azure Databricks, PySpark, and ADF

. Extract, transform, and load data from multiple sources into

Azure Data Lake Storage

. Optimize and fine-tune

ETL workflows for performance and scalability

.

SQL & Query Optimization:

Write

complex SQL queries, joins, subqueries, and functions

for data processing. Perform

SQL tuning and query optimization

for improved efficiency. Work with

large datasets

and ensure optimal data handling.

Azure Cloud Integration:

Implement

data solutions using Azure technologies

(Databricks, ADF, ADLS). Work with

batch and streaming data processing

for real-time analytics (preferred). Ensure

data security, compliance, and reliability

in cloud-based systems.

Collaboration & Problem-Solving:

Work closely with cross-functional teams to support

data analytics and business intelligence initiatives

. Identify and resolve

data-related issues proactively

. Provide insights and recommendations for

improving ETL performance

.

###

Required Skills & Qualifications:

3 to 5 years of experience

in

Azure Databricks, PySpark, and SQL

.

Strong SQL skills

complex joins, functions, procedures, query tuning.

Hands-on experience in

ADF (Azure Data Factory) and Azure Data Lake Storage

.

Ability to handle

large datasets efficiently

.

Experience in

performance optimization of ETL workflows

.

Good understanding of

batch and streaming data processing

(preferred).

Strong analytical and problem-solving skills.

Ability to

work independently

and take ownership of tasks.