Job Description

Designation: Lead Data Engineer (Individual Contributor)Location: HyderabadWork Mode: OfficeReporting to: Sr. Tech LeadAbout US:Foundation AI automatically ingests incoming documents, emails, and attachments from across your firm. It profiles matches, classifies, and saves each to your DMS, and then automates document-dependent workflows according to your rules. Read more about us atJob Overview:We are seeking a highly skilled Senior Data Engineer with 5-7 years of experience in designing and developing scalable data systems. The ideal candidate will have a strong technical background, be proficient in both relational and NoSQL databases, and have hands-on experience with cloud platforms. This role demands expertise in building robust data pipelines, optimizing data models, and ensuring data quality and integrity.Responsibilities:

• Architect and Design Data Systems: Design and develop scalable data pipelines and systems to manage large volumes of structured and unstructured data.
• Data Pipeline Development: Develop scalable end-to-end ETL (Extract, Transform, Load) data pipelines to ingest data from various diverse sources, perform transformations, and load data into target systems to meet both functional and non-functional requirements.
• Data Modeling and Optimization: Develop and optimize data models (including relational Data) for efficient storage and retrieval, ensuring data quality and integrity.
• Technical Oversight: Manage technical scope, ensuring alignment with business objectives and delivering high-quality solutions. Provided required technical guidance and oversight to various cross-functional teams and team members on databases like PostgreSQL/ Vector DB and various queries, functions, and datasets.
• Configuration Management: Manage the configurations for various tools and databases like postgresDB, vectorDB, etc. for both on-premises and cloud platforms for optimized performance and redundancy
• Performance Tuning: Monitor and optimize data pipelines, SQL/queries, functions, procedures, and triggers for performance and scalability.
• Stakeholder Collaboration: Work closely with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and deliver solutions ensuring alignment with organizational goals.
• Promoting Best Practices: Define and promote reusable, scalable, and maintainable solutions, emphasizing software engineering best practices and continuous improvement.
• Communication: Communicate effectively at all levels about the importance of solution design, conducting technical walk-throughs to ensure a clear understanding of system architecture.
• Continuous Improvement: Work with data and analytics experts to enhance functionality in data systems and products, driving improvements and growth within the data team.
• Cloud Experience: Experience of working with AWS RDS databases is preferred

Skills and Tools:

• Experience: 5-7 years in High-Performance Data Products or Data Systems as a Data Architect/Engineer.
• Technical Skills: Proficiency in orchestration tools like Airflow. Proficiency in SQL and relational databases (e.g., PostgreSQL, Aurora) and NoSQL databases (e.g., MongoDB, Cassandra).
• Good knowledge of Vector databases like pgvector etc.
• Hands-on experience with cloud platforms (e.g., AWS, Azure, GCP) and containerization technologies (e.g., Docker, Kubernetes).
• Experience with agile development methodologies (e.g., Scrum, Kanban).
• Software Engineering: Proficient in software engineering best practices, unit testing, integration testing, and tools like Git and Docker.
• Security and Compliance: Familiarity with security compliances and design practices.
• Communication: Exceptional interpersonal, analytical, and communication skills. Ability to explain and discuss concepts with colleagues and teams effectively.
• CI/CD Pipeline: Fully adhere to and promote an entire CI/CD pipeline.
• API Development: Familiarity with API development and data formats like JSON/XML.
• Domain Experience: Experience in the Insurance/Legal domain is a plus.
• Regulatory Knowledge: Familiarity with security and privacy regulations such as GDPR and HIPAA.
• Collaboration: Proven ability to collaborate effectively with cross-functional teams in a fast-paced environment.
• Problem Solving: Demonstrated ability to conduct root cause analyses and drive continuous improvement initiatives.

Certifications: AWS Certified Data Engineer, Cloudera certified professional Data Engineer, Data Bricks certified professional Data Engineer.Education:A Bachelor\'s or Master\'s degree in Computer Science, Data Science, or a related fieldOur Commitment:At Foundation AI, we\'re committed to creating an inclusive and diverse workplace. We value equal opportunity and affirmative action principles, giving everyone an equal chance to succeed. We\'re dedicated to offering equal employment opportunities regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, or veteran status. Upholding these values and adhering to applicable laws is paramount to us.
For any feedback or inquiries, please contact us atPowered by JazzHR

Foundation AI