Job Description

Req ID: 248428 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Sr Data Engineer to join our team in Bangalore, Karn taka (IN-KA), India (IN). Position\'s General Duties and Tasks . Build data streams to ingest, load, transform, group, logically join and assemble data ready for data analysis / analytics/ reporting , build data streams to ingest, load, transform, group, logically join and assemble data ready for data analysis / analytics/ reporting. . Pipeline data using Cloud: Databricks , AWS Big Data Services etc. . Responsible to write Pyspark code using databricks to connect databases, AWS services to transform data. . Design/Implement QA framework within the Data-Lake. Design and implement test strategies, write test cases, design/implement test automation . Responsible for maintaining integrity between Data Lake, databases. . Work on Data Lake/Delta Lake Data Pipeline to take data across from the source all the way to the consumption layer. . Maintain knowledge and proficiency of current and upcoming hardware/software technologies. Mentor junior staff in ramping up analytical and technical skills. Requirements for this role include: . A bachelor\'s degree from an accredited college in Computer Science or equivalent. . Strong knowledge of Databricks Data Lake/Delta Lake developments . Strong knowledge of AWS data related services (DMS, Glue, EMR, S3, Athena, Lambda, Redshift, DynamoDB, KMS). . Strong knowledge of Python and Pyspark (Hive). . Strong database/relational/non-relational concepts required. . Strong analytical and problem solving skills. . Databricks, oracle, SQL , Hadoop, Kafka, Spark, Scala (preferred). . Must have 5+ years of Data Sytems/Warehouse/Lakes/Equivalent sytem experience with multiple OS platforms . Must have 3 + years of Python and Pyspark. . Must have 3 + years of AWS or alternative cloud systems. . Must possess or develop business knowledge of how customer transactions reflect the business logic that drive the existing or future code. . Must possess or develop ability to converse with the business, development, operations, carriers, vendors, etc. . Strong experience of different architectural components comprising the middle-ware is required. . 1+ year of Matallion/IBM Data Stage or equivalent ETL tool. #LI-INPAS

foundit