Develop data pipelines/data frameworks/applications/apis using industry best practices. Make adjustments to adapt to new methodologies that provide the business with increased flexibility and agility.
Drive Proof of Concept (POC) and Proof of Technology(POT) evaluations.
Build data system and data quality monitoring process and ensure health of the big data system.
Review KPIs, Monitoring and Alarms for successful production system.
Work with the stakeholders to understand the business & reporting needs xe2x80x93 Propose, design and develop creative data solutions that provides required insight, improve/automate processes and adds value.
Influence and educate the team with your experience, idea and learning.
Stay current with the latest development tools, technology ideas, patterns and methodologies; share knowledge by clearly articulating results and ideas to key stakeholders.
Publish and update technical architecture and user/process documentation.
What you'll bring:
Bachelorxe2x80x99s degree in Computer Science or related technical field.
4+ years of relevant experience on Scala/Python (PySpark), Distributed Databases, Kafka with solid hands-on multi-threading, functional programing etc.
A good understanding of CS Fundamentals, Data Structures, Algorithms and Problem Solving.
Professional hand-on experience in Sql and Query Optimization.
Experience in building frameworks for data ingestions and consumptions patterns.
Experience with Orchestration tools like Airflow ( perfered) , Automic , Autosys etc.
Hand-on experience in Data processing and Data manipulation.
Expertise with GCP cloud and GCP data processing tools, platforms and technologies like GCS, DataProc, DPaaS, BigQuery, Hive etc.
Experience with Streaming data via Kafka or structured streaming etc.
Exposure to lambda Architecture.
Exposure to visualization tools for data reporting such as Tableau, PowerBI, Looker etc.
Excellent communication skills for collaborating with teams.
Appreciate productivity and care deeply about helping others work more effectively and efficiently.
Mandatory Skills Scala/Python (Pyspark), BSkills:Bigdata, Sql, Hadoop, SparkAbout Company:UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the worldxe2x80x99s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clientsxe2x80x99 organizations. With over 30,000 employees in 30 countries, UST builds for boundless impactxe2x80x94touching billions of lives in the process.