Job Description

*Assessment required - Hadoop, Cloudera, AWS migration (CO - Lead Cloudera/Hadoop Migration Engineer)

(Cloudera Hadoop Stack - HDFS, Hive, Spark, Oozie. Other skills like Amazon EMR, Amazon Glue, Apache Airflow)

• Migration project where we are looking for somebody who has experience working on initiatives
• Cloudera architecture (AWS)
• Experience migrating from on prem to AWS
• Experience with Hadoop

-

• Bachelor's Degree in Computer Science, Information Technology, Information Systems, or related field.
• 10+ years of experience with enterprise and application design patterns that are applied on Hadoop and Big data platforms in aws.
• The ideal candidate will possess deep expertise in both on-premises Hadoop ecosystems and modern cloud-native big data solutions on AWS.
• Hands-on experience with Cloudera Hadoop stack (HDFS, Hive, Spark, Oozie, Python)
• Proficiency in writing, maintaining, and troubleshooting shell scripts (Bash or similar) for automation and operational support.
• Migration to AWS will need
  • Analyze existing Cloudera Hadoop architecture, workflows, and data pipelines.
  • Develop a comprehensive migration strategy to transfer data, processing logic, and applications to AWS services.
  • Design target architecture using AWS big data services such as Amazon EMR, Airflow, Glue, Lambda.
  • Evaluate and select appropriate AWS tools for data ingestion, storage, processing, and analytics.
  • Oversee the migration process, including data transfer, application re-platforming, and validation.
  • Ensure the new architecture meets performance, scalability, reliability, and security requirements.
  • Collaborate with developers, data engineers, DevOps, and stakeholders to ensure a seamless migration.
  • Create detailed technical documentation and migration runbooks.
  • Mentor team members on AWS big data best practices.
  • Strong understanding of data security, compliance, and governance in the cloud.
  • Proven track record of migrating or modernizing big data workloads to AWS.
  • Experience with cloud automation and Infrastructure as Code (e.g. Terraform).
  • AWS certifications (e.g., AWS Certified Solutions Architect, AWS Certified Big Data Specialty) are highly desirable.

Job Tags

Similar Jobs