Mid-Senior Level

Manager Data Engineer | CRA015

Chennai, Tamil Nadu   |   Full Time

Hiring for our leading software product client


Role: Director/Sr/Manager Data Engineering

Location: Chennai

Experience: 10 - 18 years


About the role

As the leader of the Data Engineering org, you are tasked to Grow, lead, and mentor a team of data engineers. Own data strategy for the company, work with stakeholders to build out state the art data platform. Work with data science and business stakeholders to deploy scalable data pipelines and machine learning models in production in a timely manner


Must-Have

  • Bachelor’s/master’s degree in computer science, Engineering, Data Science, or related field. 
  • 10+ years of data platform/engineering experience with 7+ years of experience building big data pipelines using Python, SQL, and Apache Spark at scale.
  • Proven ability to hire, mentor, and manage a high-quality team of data engineers, 2+ years of experience in managing managers.
  • Ability to translate data and technical concepts into requirements documents and user stories for Data engineering as well as Data Science.
  • 5+ years deploying and maintaining ML models in production with demonstrable business impact.
  • Experience building streaming pipelines using Kafka and/or Spark.
  • Experience with cloud architecture services (AWS) including Cloud Computing Capacity (EC2), Data Lakes (S3), Data Warehouses, Serverless Computing (Lambda / GCFs, Containers, Kubernetes).
  • Familiarity with open-source orchestration tools such as Kubernetes, Apache Airflow, and Apache Spark and how to apply them.
  • Working knowledge of agile development, including DevOps concepts (IAC, CI/CD, etc.)
  • Excellent communication skills (oral and written).


Roles & Responsibility

  • Design and implement a self-service platform to speed up development and automate the deployment and scaling of data pipelines and ML models in production.
  • Assemble large, complex data sets that meet requirements to support data science and analytics projects from external and internal sources.
  • You will be responsible for managing product milestones, deployment cycles & delivery of the overall Data Engineering roadmap
  • Responsible for architecting and managing the infrastructure to support all stages of the machine learning model lifecycle, including feature engineering, feature store, model training, testing, monitoring, and deployment in a production environment.
  • Build and maintain A/B testing infrastructure for analytics and data science experimentation.
  • Proactively identify, design, and implement internal process improvements including automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
  • Build applications using open-source frameworks for big data processing and data orchestration (e.g., Apache Airflow, Kubernetes, Spark).
  • Support daily operations and production of our data platform, including monitoring, quality, and governance utilities, CI/CD pipelines, and all data integration touchpoints.
  • You will be responsible for creating and managing complex data engineering pipelines at scale and manage their availability, throughputs, and security.
  • You will be responsible for managing product milestones, deployment cycles & delivery of the overall Data Engineering roadmap

Submit Your Application

You have successfully applied
  • You have errors in applying