Intel OpenVino Boost Semantic Segmentation prediction

A booster to predict ML model on production line.

I’m here to share my experience in Deep Learning Computer Vision model deployment. As we all know that deep learning models are bulky and they take lazy time to load the model and infer incoming data.

I will be just walking through the steps I followed.

1. Train Tensorflow Keras model, predict h5 file and note the time taken.
2. Create conda enviroment
3. Install openvino “pip install openvino”
4. Convert h5 files to openvino format
5. Predict openvino model and note time.

I trained UNet(with different backbone)and DeepLabV3+ segmentation models on Open Images Dataset v6 + Extension (I will explain thoroughly about downloading dataset and converting masks to trainable masks in other blog). DeepLabV3+ outperformed in comparison with UNet with different variants.

When it came to model deployment on CPU machine, h5 model took 1.9 to 2.3 seconds to load model with prediction rate of 3 FPS, openvino took hardly 0.6 seconds to load model with prediction rate of 11 FPS … sonic boom….!!!

Comparison Between openvino and h5 file prediction

Rather than pasting the code I will add the reference link which helped me to code this. Please comment for any suggestion.

Install Intel® Distribution of OpenVINO™ toolkit from Anaconda Cloud - OpenVINO™ documentation

From Training to Deployment with TensorFlow and OpenVINO - OpenVINO™ documentation

Converting a TensorFlow* Model - OpenVINO™ documentation