Workshop: Building reproducible workflows for earth sciences

Scaling Machine Learning with the help of Cloud Computing

Speaker

Mr Salman Iqbal (ONS / Learnk8s)

Description

One of the most common hurdles with developing data science/machine learning models is to design end-to-end pipelines that can operate at scale and in real-time. Data scientists and engineers are often expected to learn, develop and maintain the infrastructure for their experiments. This process takes time away from focussing on training and developing the models.

What if there was a way of abstracting away the non Machine Learning related tasks while still retaining control? This talk will discuss the merits of using Kubeflow. Kubeflow is an open source Kubernetes based platform. With the help of Kubeflow, users can:

  • Develop Machine Learning models easily and make repeatable, portable deployments on a diverse infrastructure e.g. laptop to production cluster.
  • Scale infrastructure based on the demand.

This talk will also present the current use cases of Kubeflow and how teams from other industries have been utilising the cloud to scale their machine learning operations.

Primary author

Mr Salman Iqbal (ONS / Learnk8s)

Presentation materials

#repwork19