Job Description

Vidpro Consultancy Services is seeking a Senior Databricks Engineer / Databricks Technical Lead / Data Architect with 5-10 years of experience.Location: Bangalore, Chennai, Delhi, Pune, Kolkata and GurgaonWork Type: HybridMandatory Skills: Python/ PySpark / Spark with Azure/ Azure Databricks / Azure DataFactoryPrimary Roles and Responsibilities:- Creating Modern Data Warehouse solutions using Databricks and AWS/Azure Stack- Providing forward-thinking solutions in data engineering and analytics- Collaborating with DW/BI leads for new ETL pipeline development requirements- Troubleshooting issues in existing pipelines and addressing gaps- Understanding reporting layer needs and developing data models- Assisting team members with problem resolution and technical challenges- Leading technical discussions with client architects and team members- Orchestrating data pipelines in a scheduler via AirflowSkills and Qualifications:- Bachelor\'s and/or master\'s degree in computer science or equivalent experience- Minimum 3 years of experience in Data warehouse/ETL projects- Deep understanding of Star and Snowflake dimensional modelling- Strong knowledge of Data Management principles- Proficiency in Databricks Data & AI platform and Databricks Delta Lake Architecture- Hands-on experience in SQL, Python and Spark (PySpark)- Experience with AWS/Azure stack and ETL with batch and streaming (Kinesis)- Building ETL/data warehouse transformation processes- Familiarity with Apache Kafka for use with streaming data/event-based data- Working knowledge of Hadoop (incl. Hive, Pig, Impala) and non-relational/NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J)- Experience with structured and unstructured data including imaging & geospatial data- Proficiency in RDBMS, complex SQL, PL/SQL, Unix Shell Scripting, performance tuning and troubleshooting- Databricks Certified Data Engineer Associate/Professional Certification (Desirable)- Comfortable working in a dynamic, fast-paced, innovative environment with several ongoing concurrent projects- Experience working in Agile methodology- Strong verbal and written communication skills- Strong analytical and problem-solving skills with a high attention to detail

FullThrottle Labs