Job Description

Company DescriptionExperian is a global data and technology company, powering opportunities for people and businesses around the world. We help to redefine lending practices, uncover and prevent fraud, simplify healthcare, create marketing solutions, all using our unique combination of data, analytics and software. We also assist millions of people to realise their financial goals and help them save time and money.We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more industry segments.We invest in people and new advanced technologies to unlock the power of data. As a FTSE 100 Index company listed on the London Stock Exchange (EXPN), we have a team of 22,500 people across 32 countries. Our corporate headquarters are in Dublin, Ireland. Learn more atWe are seeking an experienced and experienced Lead Data Engineer to join our dynamic team. You have a strong background in designing and implementing both batch and streaming data pipelines, migrating on-premise data solutions to the cloud, and applying AI/ML techniques to solve complex business problems. We are looking for proficiency in multiple programming languages, cloud technologies, and big data tools, and mentor a team of engineers.Key Responsibilities:

• You will be #LI-hybrid based in Hyderabad and reporting to manager.
• This is individual contributor (Non-Managerial) role
• Design and Implement Data Pipelines: Lead the development of batch and streaming data pipelines using scalable data structures and algorithms to handle large-scale data processing workloads.
• Cloud Migration: Provide technical guidance to the team for the migration of on-premise data solutions to AWS, ensuring adherence to cloud architecture best practices. Utilize AWS services such as EMR, S3, RDS, MSK, Lambda, CloudWatch, EKS, and IAM.
• AI/ML Integration: Apply AI/ML and Generative AI techniques to create innovative solutions that meet business requirements. Ensure these solutions are scalable, efficient, and aligned with industry best practices. Mentor team members in AI/ML concepts and techniques.
• Software Development: Develop software modules using Java, Scala, and Python. Perform unit and automated integration testing using CI/CD pipelines before deployment on AWS EC2 and EMR instances.
• Technical Collaboration: Engage in collaborative technical discussions with the team to review designs and code, and suggest improvements.
• Customer Support: Work closely with customers to provide ongoing maintenance and support for software solutions, ensuring high levels of customer satisfaction.

Qualifications

• Education: Master's degree in Computer Science, Engineering, or a related STEM field with 5 years of work experience in Data Engineering, or a Bachelor's degree with 7 years of relevant work experience. Experience in Data Science.
• Programming Skills: Proficiency in Python, Java, Scala, and SQL. Solid understanding of relational and non-relational datastores.
• Big Data Technologies: Expertise in Hadoop, Spark, and Kafka. Experience with AWS services such as EMR, S3, RDS, MSK, Lambda, CloudWatch, EKS, and IAM.
• AI/ML Knowledge: Foundational understanding of AI/ML, including Generative AI.
• Experience in leading a team of engineers.

Additional InformationOur uniqueness is that we celebrate yours. Experian's culture and people are important differentiators. We take our people agenda very seriously and focus on what matters; DEI, work/life balance, development, authenticity, collaboration, wellness, reward & recognition, volunteering... the list goes on. Experian's people first approach is award-winning; World's Best Workplacesxe2x84xa2 2024 (Fortune Global Top 25), Great Place To Workxe2x84xa2 in 24 countries, and Glassdoor Best Places to Work 2024 to name a few. Check out Experian Life on social or our Careers Site and Glassdoor to understand why.Experian is proud to be an Equal Opportunity and Affirmative Action employer. Innovation is a critical part of Experian's DNA and practices, and our diverse workforce drives our success. Everyone can succeed at Experian and bring their whole self to work, irrespective of their gender, ethnicity, religion, color, sexuality, physical ability or age. If you have a disability or special need that requires accommodation, please let us know at the earliest opportunity.BenefitsExperian care for employee's work life balance, health, safety and wellbeing. In support of this endeavor, we offer best-in-class family well-being benefits, enhanced medical benefits and paid time off.Experian Careers - Creating a better tomorrow together

Experian