Job Description

RemoteStar is looking to hire a SQL Developer on behalf of our client based in Bangalore with Hybrid setup.

About Client

Our Client is a technology partner dedicated to driving business transformation. With deep expertise in Cloud, Data, MLOPs, AI, Infrastructure services, Application modernization services, Blockchain, and Big Data, Our Client help organizations tackle complex challenges and seize growth opportunities in today's fast-paced digital landscape. they are more than just a tech service provider; they are a catalyst for meaningful change. Responsibilities

Migration consultants will play a critical role in ensuring the successful migration of data science and reporting assets from BigQuery to MaxCompute. Their primary responsibilities include:

1. Dependency Assessment
Verify upstream dependencies have been migrated and are functional before initiating migration tasks.

2. Schema and Data Type Migration
Migrate schema definitions and data types, ensuring compatibility with MaxCompute standards.

3. SQL Translation
Convert SQL queries from BigQuery dialect to MaxCompute, maintaining functionality and optimizing for efficiency.

4. Pipeline Cloning and Materialization
Clone and configure data pipelines in MaxCompute, replicating workflows and materializing datasets as needed.

5. Historical Data Migration
Manage the transfer of historical data, ensuring minimal downtime and no data loss.

6. Data Validation and Quality Assurance
Perform thorough validation checks to guarantee data accuracy, completeness, and consistency post-migration.

7. Documentation and Reporting
Maintain clear and detailed documentation of the migration process, including challenges, solutions, and progress updates.

8. Collaboration
Work closely with internal teams to resolve dependencies and align with overall project goals and timelines. Competencies - To successfully execute the migration tasks outlined above, a suitable consultant should possess the following skills and experience: Strong SQL Proficiency : 4 -5 years of experience of working with SQL queries. Experience with Cloud Data Warehouses: Hands-on experience with BigQuery, MaxCompute, and an understanding of schema design and data modeling principles. Data Pipeline Management : Familiarity with cloning pipelines and using ETL/ELT tools to ensure seamless migration. Data Validation Expertise: Proven ability to execute validation checks and maintain data integrity. Attention to Detail: Precision in following templates, guidelines, and systematic processes. Communication and Collaboration Skills: Strong ability to coordinate with teams, document progress, and report findings effectively.

Bonus skills: BigQuery Experience. Historical Data Migration : Experience with large-scale data transfers across cloud platforms. dbt Expertise : Knowledge of DBT workflows and integration with cloud data warehouses. Flyte Experience : Familiarity with Flyte for workflow orchestration and scheduling, including design, management, and troubleshooting.