Run CNN model in Flutter
Published in
6 min readJun 5, 2020
Run any type of Machine learning model in a flutter app.
TensorFlow Lite is a set of tools to help developers run TensorFlow models on mobile, embedded, and IoT devices. It enables on-device machine learning inference with low latency and small binary size.
PreRequisites
- > We will assume that you have your model already build
- > Yes you can use any model in this but here we are going to use a CNN
- > We will learn how to deploy your model in a flutter app, that means automatically on both IOS and Android as flutter is a cross-platform framework.
- > Okay then, there are lots of articles on how to deploy your model in a flutter app, but everywhere it lacks some of the key features when dealing with the real issues.
- > In this article we’ll see the minor details we need to have your focus onto a successful deployment
- > I’ll mention the most useful libraries imported at the very end just in case anyone has any problems. But if you have trained your model already there’s a good chance you have already imported most Libraries we need
Trained Model
- > As we mentioned earlier we will assume you have your model trained.
- > Just for those folks who are still confused, we are assuming you have your trained model with you after the execution of code which looks similar to this
Model = model.Squential(....)
.
.
trained_model = Model.fit(traningData , TrainingLabels)ORtrained_model = model.fit_generator(train_batches,epochs=10)
- > trained_model is what we need and from where we will start this article
Save the Model
- > First we will save the model, just in case we need it for any future use
model.save("augmented_model.model")Get your class indices
- > Write down or save the index of the classes
- > When the model predicts something or some image or some input it returns a vector (1D matrix) all the values will be 0 and One value will be 1 stating that the model has predicted and we need to know that what our model indices are
- > if you have made a multiclass CNN, most probably you have used a softmax function, that means your result vector will have probability values instead of 1 and 0
- > we’ll select the highest value as our result
test_batches.class_indices#output :-
{'dettol_sanatiser': 0,
'fevicol': 1,
'lifeboy_sanatiser': 2,
'scissors': 3,
'setwet_hair_Gel': 4}
Save the model in HDFS format
- > Don’t think too much on what this is, you don’t need to know what this format is for right now, but it’ll be good if you know this, might come in handy for your future.
import tensorflow as tfkeras_file = "My_saved_Model.h5"
keras.models.save_model(trained_model , keras_file)
Convert your model into a TFlite model
- > Now this is a tricky part, which you can’t get through just by knowing the code
- > By doing some research I came to know things have changed on what actually tutorials and articles have in them and in actual working, that’s fine things upgrade
- > Converting an .h5 model into the TFlite model was easy with Tensorflow 1.x. But it’s tricky with 2.x
- > you’ll find the code for converting it for sure, but some of you will have many issues.
- > That’s because the TensorFlow you have installed with pip or conda installer doesn’t contain the library of this converting code.
- > So you’ll have to get the Orginal, Full version of TF from GitHub directly which is tricky also, but if you are a genius go ahead do it, again this might come handy in future for you
- > But if that’s too much, you could use Google Colabs,
- > Google Colab is is a free cloud service hosted by Google to encourage Machine Learning and Artificial Intelligence research, where often the barrier to learning and success is the requirement of tremendous computational power.
- > Just consider it as a virtual Jupyter Notebook stored in your Gdrive.
- > Go ahead and create a notebook in google colab with python3, IT’S FREE
Upload your saved Model to Colab
- > This Step frustrated me also as the model does get upload but the actual code doesn’t recognize or detect the uploaded file
- > Do this to ensure the model is uploaded at the correct place
- Click on the files tab
- Click on the … folder
it’ll take you to the whole directory of your virtual machine
- > We need to upload our saved model in this folder “content”
- > for many of you simple uploading also might work, but if this doesn't work just follow the above procedure
- > Right Click on Content Folder and Upload your saved model
- > Upload “My_saved_Model.h5”
- > Wait for some time, it’ll take time to upload and when it’s uploaded do not close your session for the notebook, if closed your model will be removed
Finally some CODE
- > run this code on your Colab notebook
import os
import tensorflow as tf
from tensorflow import keras
print(tf.__version__)
!pwd
!ls
###OUTPUT### :-2.2.0-rc4
/content sample_data My_saved_Model.h5
tensflow version might be different but still in 2.x+
Convert the Model into TFLITE now with CODE
My_TFlite_Model = tf.keras.models.load_model('My_saved_Model.h5')converter = tf.lite.TFLiteConverter.from_keras_model(minor_model)tflite_model = converter.convert()open("My_TFlite_Model.tflite", "wb").write(tflite_model)
- > this code will save your converted model in the same directory
- > Download your Precious Tflite model from the directory
if your model is successfully converted ignore this next part and skip to the Flutter Part, but if you are still having an error try this quickly,
try to save a random model trained and saved in the colab and then reload it, if you are able to reload it in the code, use the directory of this random model. which means upload your model to the directory where this random model is saved and then try to load your model too with this directory path.
and wait for at least 5–6 mins before giving up, the colab might not be synced yet with the latest uploaded files.
create and save your random model
x = [-1, 0, 1, 2, 3, 4]
y = [-3, -1, 1, 3, 5, 7]
model = tf.keras.models.Sequential([tf.keras.layers.Dense(units=1, input_shape=[1])])
model.compile(optimizer='sgd', loss='mean_squared_error')
model.fit(x, y, epochs=1)
model.save('sample_model.h5')#load the saved model again, do this in a different cell
new_model = tf.keras.models.load_model('sample_model.h5')
new_model.summary()
Flutter Part
- > Create a assets folder in the flutter app
- > copy your downloaded TFlite model in the assets folder
- > create a text file in assets file with name label.txt
- > in this text fill out the names of your result object in the same order we got earlier from class indices but without the index number, just the correct order
dettol_sanatiser
fevicol
lifeboy_sanatiser
scissors
setwet_hair_Gelpubspec.yaml
dependencies:
flutter:
sdk: flutter
cupertino_icons: ^0.1.2
tflite: ^1.0.4
image_picker: ^0.6.2+3main.dart
import 'package:tflite/tflite.dart';
import 'package:image_picker/image_picker.dart';variables
List _outputs;
File _image;initState()
@override
void initState() {
super.initState();
loadModel().then((value) {setState((){});});
}Functions
pickImage() async {
var image = await ImagePicker.pickImage(source: ImageSource.gallery);
if (image == null) return null;
setState(() {
_image = image;
});
classifyImage(image);
}classifyImage(File image) async {
var output = await Tflite.runModelOnImage(
path: image.path,
);
print("predict = "+output.toString());
setState(() {
_outputs = output;
});
}loadModel() async {
await Tflite.loadModel(
model: "assets/My_TFlite_Model.tflite",
labels: "assets/labels.txt",
);
}@override
void dispose() {
Tflite.close();
super.dispose();
}
- > Just put a button on the UI and Call pickImage()
- > when you pick an image it’ll automatically call the classify the image and store the result in _outputs variable
- > _outputs will be a list with the most suitable probability of the prediction, the first element will have the highest probability
- > _outputs[0] will be your result
YOU’RE DONE
You can see the working of this in the app in this post
Thanks for reading
