Job Description

As a Data Engineer you will work on implementing complex data projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into value using multiple platforms.You will work with business analysts, architects and data scientists to understand customer business problems and needs, secure the data supply chain, implement analysis solutions and visualize outcomes that support improved decision making for a customer. You will understand how to apply technologies to solve big data problems and to develop innovative big data solutions.Key Accountabilitiesxefx82xb7 Working with colleagues to understand and implement requirements both Functional and Non-Functionalxefx82xb7 Securing the data supply chain, understanding how data is ingested from different sources and combined / transformed into a single data set.xefx82xb7 Understanding how to analyse, cleanse, join and transform data.xefx82xb7 Implementing designed / specified solutions into the chosen platform.xefx82xb7 Working with colleagues to ensure that the on-prem/cloud infrastructure available is capable of meeting the solution requirements.xefx82xb7 Planning, designing and conducting tests of the implementations, correcting errors and re-testing to achieve an acceptable result.xefx82xb7 Appreciate how to manage the data including; security, archiving, structure and storage.Key Experience and Qualificationsxefx82xb7 Degree level education in Mathematics, Scientific, Computing or Engineering discipline or equivalent experiencexefx82xb7 5-8 years of experience at various levels of Data Engineering roles including 3 years in technical lead role managing end to end solution.xefx82xb7 Hands on Experience with Azure Databricks using PYSPARK with ability to do cluster capacity planning and workload optimization.xefx82xb7 Experience building and optimizing Big Data pipelines, architectures and data sets using MS Azure data management and processing components through IaaS/PaaS/SaaS implementation models implemented through customxefx82xb7 Experience in Azure Data factory.xefx82xb7 Proficiency in Python scripting. Experienced in using different python modules used for data munging.xefx82xb7 Experience in designing solutions using databases and data storage technology using RDBMS (MS SQL Server)xefx82xb7 Experience in Data Modeling for optimizing the solution performance.xefx82xb7 Experienced in Azure DevOps with exposure to CI/CD.xefx82xb7 Good understanding of infrastructure components and their fit in different types of data solutionsxefx82xb7 Experience of designing solutions deployed on Microsoft and Linux operating systemsxefx82xb7 Experience of working in an agile environment, within a self-organising team.

Deloitte