Image Classification using TensorFlow.js

Hello Everyone !!

Thanks for continuing with this post.

In this post, I want to show you that how easy it is to use an existing Machine Learning model and using it using TensorFlow.js to make predictions on images in the browser. So, let's get started.

First, let me begin by showing you a simple demo of what we'll be building in this tutorial. Below, you can see some images. If you click on any image, you see that some text appears below that tells that what the machine learning model thinks is in the image i.e. predicted labels and their probabilities. Go ahead and try it out.

So, what's going on here. I am making use of TensorFlow.js to load a MobileNet model and providing the selected images as input to the model to get some predictions from the model. What's different in this approach as compared to others is that all the prediction using the model is taking place in your browser.

Yes, you heard that right !! It's hapenning right here on your browser, be it the phone or a laptop. No server required. Don't believe me yet ? Go ahead and open your browser's console and click on any image. Believe me now ?

As you open the console and click on an image above, you should see some output like in the image below. This is the output that I get when I select the Panda image:

Let's see how this all is done in code. The first step is to load TensorFlow.js library and the MobileNet model. We can doo this in two ways: load the library hosted using a CDN or load the library locally using a tfjs.json and model.json files. In this tutorial we'll be using the CND. Let's see this in code.

Next, we define a function to get the ID of the image that the user selects. For this, we provide all images with a unique ID. Then to get this ID on selecting an image and process it, we add a OnClick event to each image that calls a function in JavaScript.

The OnClick event calls the imageSelected(image_id) function in JavaScript and passes in the selected image's ID as an argument. The JavaScript function for doing this looks like follows:

Now that we know that which image was selected, we can use this information to get the image from the source for processing like resizing the image to 224x224 for input to MobileNet model as follows:

Since, we have done all the pre-processing required, we are now ready to make predictions on the selected image. For doing this, we just load our MobileNet model using CDN or you can save and load the model locally as a model.json file and call the model.classify(image) function. The model.classify(image) function returns two things, the className i.e. the class labels for the image and probability i.e. the probability the model thinks this image belongs to that class. This can be done in the code as follows:

So, in this tutorial, we saw that how to use TensorFlow.js for performing image classification in the browser using a machine learning model. You can follow almost similar steps to use a custom machine learning model and run in the browser.

I'll leave you with some more examples to play around with below.

Now that we know how to use pre-trained model to make prediction on images in the browser, now it's time to see how we can use the same model to make predictions on live video using our webcam in the browser.

Great work on completing this tutorial. Next, we'll see how to use a pre-trained model using TensorFlow.js in the browser and make predictions on live stream video. Image Classification on Live Video using TensorFlow.js

For more projects and code, follow me on Github

Please feel free to leave any comments, suggestions, corrections if any, below.

comments powered by Disqus