Job Description

Data Engineering

:


We are seeking a Senior Lead Data Engineer with comprehensive Data System development expertise. This role involves strategic and tactical planning and execution to advance our Subscription and Data platform's vision and mission. As a passionate and skilled Data engineering expert and system architect, you will work in a dynamic environment as part of Agile Scrum teams.


You will play a crucial role in the Data Engineering and Solutioning team, tasked with re-architecting traditional Full Stack applications into modern, cloud-centric solutions. Responsibilities include leading the data engineering team, introducing best practices and the latest technology, collaborating with development managers and teams to resolve technical challenges, and building data pipelines (both batch and streaming), data products, AI, and platform capabilities.

Responsibilities:

• Collaborate with the legacy product team to re-architect legacy full stack applications into modern, cloud-centric, and Big Data-oriented solutions, including those previously built in Java and other full stack languages.
• Partner with the data & analytics team to design data products, such as data marts, data semantic layers, analytics, and third-party system integrations. Establish technical standards and best practices, enhancing the subscription and data platform's engineering and scalability.
• Architect and design data platform and product capabilities, including data processing batch and streaming engines, Data Lake, In-Memory Computing Systems, and Distributed Data Systems, with a focus on Data Governance & Observability.
• Implement the best architecture practices to build highly available and scalable applications. Participate in design and code reviews to ensure quality work aligning with development standards.
• Provide thought leadership on the latest trends and best practices in software development. Collaborate with Product Engineering, Centre of Excellence, and Operations teams to evaluate new and mature technologies, helping shape the organization's technology roadmap.
• Identify opportunities for future enhancements to standards, best practices, and development methodologies.
• Contribute to product innovation development.
• Coach and mentor junior team members, fostering team talent development.

Knowledge and Skills

• Strong understanding of data structures and OOPS.
• In-depth knowledge of cloud technologies (e.g., AWS, Azure) and experience re-architecting traditional full stack applications to modern, cloud-centric, container-based, and big data-oriented applications.
• Proven experience as a Software Architect with a complete software lifecycle expertise (10+ years). Demonstrated ability in specifying, developing, and delivering software applications across various standards and languages.
• Experience with Big Data and analytics, leveraging technologies like Databricks, Spark, and MapReduce.
• Proficiency in several programming languages, such as Scala and Python.
• Expertise with OLAP databases such as Delta Lake, Snowflake, and Redshift.
• Strong analytical and problem-solving skills.
• Experience with ETL and Databricks notebooks development.
• Proficiency in at least one full stack language (Java/.NET).