Job Description

Job SummaryData Platform EngineerResponsible to procure and maintain required infra capabilities like Kube storage(Hadoop/object store)Responsible to maintain and support job scheduler required for running jobResponsible to support for ADT data table creation modification replication or cleanup as needed for Data product or Data scienceResponsible to create scripts and ensure onboard/offboard data products to access modules like ADAM/PapyrusResponsible for namespace administrationResponsibilitiesData Platform EngineerResponsible to procure and maintain required infra capabilities like Kube storage(Hadoop/object store)Responsible to maintain and support job scheduler required for running jobResponsible to support for ADT data table creation modification replication or cleanup as needed for Data product or Data scienceResponsible to create scripts and ensure onboard/offboard data products to access modules like ADAM/PapyrusResponsible for namespace administrationResponsible for CICDDefines and maintains Github processes like branching and PR processes to mainDefines and maintains deployment pipeline using Rio for new data products and implements checks for dev/stage/prodSetup necessary access controls create deployment config by environment to ensure data pipelines can runCreates job workflow on job scheduler and enables alerts to slack email etcResponsible for enabling notebooks and compute needed for data scienceEnable Git processes for exploratory data analysis for sql scripts/python codeEnable processes for running pipelines from notebooks for adhoc analysisConfigure and maintain notebook plugins as needed for specific usecaseSupport for procuring data access for DS following governance practices as prescribed * Works with Amp Platform to leverage platform capabilities example: Lakehouse Snowflake etcStrong understanding ofHDFSObjectStoreContainersKubernetesJenkinsBash / Python scriptingAirflow Job SchedulerGIT and Github enterpriseworking knowledge of Iceberg tables and Parquet fileTrinoAccess control principlesHive Metastores

Cognizant