Job Description

Overview:

As a data engineering manager, you will be the key technical expert developing and overseeing PepsiCo's data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business. You'll be an empowered member of a team of data engineers who build data pipelines into various source systems, rest data on the PepsiCo Data Lake, and enable exploration and access for analytics, visualization, machine learning, and product development efforts across the company. As a member of the data engineering team, you will help lead the development of very large and complex data applications into cloud environments directly impacting the design, architecture, and implementation of PepsiCo's flagship data products around finance. You will work closely with process owners, product owners and business users. You'll be working in a hybrid environment with in-house, on-premise data sources as well as cloud and remote systems.
Responsibilities:
• Provide leadership and management to a team of data engineers, managing processes and their flow of work, vetting their designs, and mentoring them to realize their full potential.
• Engages with team members, uses informal and structured approaches to career development to focus on individual improvement/capabilities, and to provide balanced feedback.
• Act as a subject matter expert across different streams.
• Manage and scale data pipelines from internal and external data sources to support new product launches and drive data quality across data products.
• Build and own the automation and monitoring frameworks that captures metrics and operational KPIs for data pipeline quality and performance.
• Responsible for implementing best practices around systems integration, security, performance and data management.
• Empower the business by creating value through the increased adoption of data, data science and business intelligence landscape.
• Collaborate with internal clients (data science and product teams) to drive solutioning and POC discussions.
• Evolve the architectural capabilities and maturity of the data platform by engaging with enterprise architects and strategic internal and external partners.
• Ability to anticipate data bottlenecks (latency, quality, speed) and recommend appropriate remediation strategies
• Ensure on time and on budget delivery which satisfies project requirements, while adhering to enterprise architecture standards.
• Facilitate work intake, prioritization and release timing, balancing demand and available resources. Ensure tactical initiatives are aligned with the strategic vision and business needs.
• Ensure sustainability of live pipelines in production environment
• Hands on experience of implementing and designing data engineering workloads using Spark, Databricks , or similar modern data processing technology
• Work with product owners, scrum masters and technical committee to define the 3 months road-map for each program increment (sprint wise) and yearly road-map
• Manage and scale data pipelines responsible for ingestion and data transformation.
• Evolve the architectural capabilities and maturity of the data platform by engaging with enterprise architects and strategic internal and external partners.
• Prototype new approaches and build solutions at scale.
• Research in state-of-the-art methodologies.
• Create documentation for learnings and knowledge transfer.




Qualifications:
Required skill set and experience: • Bachelor's degree in Computer Science, MIS, Business Management, or related field
• 12 + years' experience in Information Technology
• 8+ years of experience with Data Lake Infrastructure, Data Warehousing, and Data Analytics tools.
• 8+ years of experience in SQL optimization and performance tuning, and development experience in programming languages like Python, PySpark, Scala etc.).
• 5+ years in cloud data engineering experience in at least one cloud (Azure, AWS, GCP).
• Experience scaling and managing a team of 5-10 engineers.
• Experience of building frameworks for different processes such as data ingestion and dataops
• Well versed with Spark optimization techniques
• Experience dealing with multiple vendors as necessary.
• Hands on experience of writing complex SQL queries
• Big Data (Hadoop, HBase, MapReduce, Hive, HDFS etc.), Spark/PySpark
• Sound skills and hands on experience with Azure Data Lake, Azure Data Factory, Azure Data Bricks ,Azure Synapse Analytics, Azure Storage Explorer
• Proficient in creating Data Factory pipelines for on-cloud ETL processing; copy activity, custom Azure development etc.
• Experience with data modeling, data warehousing, and building high-volume ETL/ELT pipelines.
• Experience with data profiling and data quality tools
• Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.
• Experience with at least one MPP database technology such as Redshift, Synapse or SnowFlake.
• Working knowledge of agile development, including DevOps and DataOps concepts.
• Familiarity with business intelligence tools (such as PowerBI).

Mandatory "Non-Technical" Skills • Excellent remote collaboration skills
• Experience working in a matrix organization with diverse priorities.
• Enthusiast for learning functional knowledge specific to finance business
• Ability to work with virtual teams (remote work locations); within team of technical resources (employees and contractors) based in multiple global locations.
• Participate in technical discussions, driving clarity of complex issues/requirements to build robust solutions
• Strong communication skills to meet with delivery teams and business-facing teams, understand sometimes ambiguous, needs, and translate to clear, aligned requirements.