How to set up Google Colab for high-performance computing
Articles,  Blog

How to set up Google Colab for high-performance computing


[Music] Google Colab is a free Jupyter notebook environment that requires no setup and runs entirely in the cloud. Have you asked yourself “Can I do interactive big computing in the Colab?” Have you seen the Techila Distributed Computing Engine demo, where we are running Jupyter Notebook interactively in “Google scale” using Techila Distributed Computing Engine (TDCE) and 1000 Google CPU cores? You can do this in Colab too. This tutorial shows how to set up your Google Colab for the use with TDCE. In the next tutorial we will scale it up to 1000X. We assume that you have already launched TDCE in your Google account. TDCE is available in the GCP Marketplace. You can launch it with a couple of mouse clicks. Let’s connect your Colab with the TDCE system that is running in your Google account. There is a notebook on GitHub that does this automatically. Enter the address and password of your Techila server into the fields on the right. Click the “Play” button to run the cell. There will be a security warning for code from an external source. This downloads a pre-configured Techila SDK from your Techila Server into your Colab and takes care of the required configuration steps. The Techila SDK connects your Colab to the TDCE system in your own Google account. The setup is now complete. you can test the connection between your Colab and TDCE. Before running any tests, please make sure you have some Techla Workers running. These are computers in your TDCE environment that do the actual processing. More Techila Workers translates to more throughput. Go to the Techila Configuration Wizard on your Techila Server. Go to the “Adjust Computing Throughput” page. The platform of Google Colab is Linux. Check that the “Computing Environment” is set to Linux. For a small connection test run you don’t need more than one or two Techila Workers”. Click the “Start” button. The Techila Workers will start automatically in your own Google account. Now you can run a simple test code to test the connection between your Colab and TDCE. There is a little test script on the second cell of the Notebook that we downloaded from GitHub. If the output is “2, 2, 2, 2, 2” your Google Colab is successfully configured for use with your Techila Distributed Computing Engine system. Would you like to see how to go bigger? In the next video we will scale our system to 1,000 Google CPU cores. Watch the video to see how Techila Distributed Computing Engine along with the 1000 Google CPU cores allows us to run a two and a half day long Colab work interactively, in just four minutes.

Leave a Reply

Your email address will not be published. Required fields are marked *