Sr Data Engineer Gcp Pyspark

Year    KA, IN, India

Job Description

About Lowe's


Lowe's Companies, Inc. (NYSE: LOW) is a FORTUNE 50 home improvement company serving approximately 16 million customer transactions a week in the United States. With total fiscal year 2023 sales of more than $86 billion, Lowe's operates over 1,700 home improvement stores and employs approximately 300,000 associates. Based in Bengaluru, Lowe's India develops innovative technology products and solutions and delivers business capabilities to provide the best omnichannel experience for Lowe's customers. Lowe's India employs over 4,200 associates across technology, analytics, merchandising, supply chain, marketing, finance and accounting, product management and shared services. Lowe's India actively supports the communities it serves through programs focused on skill-building, sustainability and safe homes. For more information, visit, www.lowes.co.in.

About the Team


In Price and Promotions (PPCI), we develop comprehensive systems that encompass the entire spectrum of pricing and promotions capabilities and functionalities. We create data products that integrate various customer-facing applications, leveraging data and machine learning models to provide insights that inform decision-making and streamline workflows.

Job Summary:


The primary purpose of this role is to translate business requirements and functional specifications into logical program designs and to deliver modules, stable application systems, and Data Engineering solutions.

This includes developing, configuring, or modifying complex integrated business and/or enterprise infrastructure or application solutions within various computing environments.

This role facilitates the implementation and maintenance of complex business and enterprise Big Data Platform (Hadoop on-Prem and GCP Cloud) solutions to ensure successful deployment of released applications.

Roles & Responsibilities:

Core Responsibilities:

• In this role you will design, develop, and maintain scalable data pipelines and systems using Pyspark, Spark SQL on GCP platform. You will implement and optimize data engineering processes with a focus on performance, cost-efficiency, and reliability.
• You will be leading and executing data engineering projects from inception to completion, ensuring timely delivery and high quality while you continuously monitor, troubleshoot, and improve data pipelines and workflows to ensure optimal performance and cost-effectiveness
• You will collaborate with cross-functional teams to understand data needs and deliver solutions that meet business requirements, mentor, and educate junior engineers on best practices, emerging technologies, and efficient data engineering techniques

Years of Experience:

• 5+ years of Experience in data engineering.

Qualification & Certifications

Required Minimum Qualifications :

• Bachelor's degree in engineering, Computer Science, CIS, or related field (or equivalent work experience in a related field)

Skill Set Required

Primary Skills (must have)

• Hands on GCP experience preferably in the following services


+ DataProc
+ GCS (Google Cloud Storage)
+ Big Query Tables
+ Cloud Composer
• Pyspark (Spark with Python)
• Python scripting and programming
• Advanced SQL
• Airflowany other orchestration tool
• CICD
• Git or any other VCS knowledge
• Data modellingData warehouse knowledge

Secondary Skills (desired)

• NoSQL and relational Databases
• ETL and ELT process
• Data mesh architecture
• Kafka or any other messaging queue experience
• Spark streaming (lambda and kappa architecture)
• Spark optimization

•Lowe's is an equal opportunity employer and administers all personnel practices without regard to race, color, religious creed, sex, gender, age, ancestry, national origin, mental or physical disability or medical condition, sexual orientation, gender identity or expression, marital status, military or veteran status, genetic information, or any other category protected under federal, state, or local law.•

Beware of fraud agents! do not pay money to get a job

MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.


Related Jobs

Job Detail

  • Job Id
    JD3423376
  • Industry
    Not mentioned
  • Total Positions
    1
  • Job Type:
    Contract
  • Salary:
    Not mentioned
  • Employment Status
    Permanent
  • Job Location
    KA, IN, India
  • Education
    Not mentioned
  • Experience
    Year