Porting this model to serve it in Android
anandcu3 opened this issue · comments
Hi @kwotsin
This is brilliant work. Thanks for sharing this.
I want to freeze this model and serve it through an android app. Can it be done? Do you have any pointers as to where to start from..
@anandcu3 You can refer to https://github.com/kwotsin/TensorFlow-Xception/blob/master/write_pb.py
to see how you can freeze the model. The freezing converts all the weight variables into constants and all you have to focus on is the input and output nodes for the frozen graph. Then you can try to incorporate the model in the app using the tensorflow library for android, which agains only focuses on input and output, which fits the use case of the frozen graph.
@kwotsin Can you please provide more information about the graph (specially the input/output nodes) I have currently been able to freeze the model and then I'm stuck in compiling TensorFlow with Bazel. But once I'm done with this (hopefully soon) I will have to write the code in Android to use the model and will need the nodes info.
Thanks
@anandcu3 For the input and output nodes, you can try doing this:
for op in graph.get_operations:
print op.values()
This will show the list of nodes, where you can locate the input and output nodes with their names and shape information.
I was able to port it successfully to android. Thanks.
It's running at 2 secs/frame. at the moment. I am looking to optimize it . Any thoughts regarding the same, mainly in the inference phase..
Would it be possible for you to share (e.g. in a GitHub Gist) how you port over the model into Android? I am currently working on integrating such models into mobile devices and optimizing their performance, so it would be interesting to learn from you as well. What FPS do you want to get and what hardware are you using?
@kwotsin I had to compile the Tensorflow Library a couple of times because many of the operations used by E-net were not part of the default operations included for Android. To do this I referred https://medium.com/joytunes/deploying-a-tensorflow-model-to-android-69d04d1b0cba which has a very useful code snippet which helps to understand which ops have to be added to the android ops. I had to remove maxpool_argmax from the network because there is no CPU implementation of the operation yet. I used basic maxpool and created a dummy array for indices. Surprisingly, the accuracy did not take a hit.
My goal is to eventually reach 5fps on the phone.
@kwotsin I have hit another issue. When I freeze the model using Google's Freeze module the output looks random and then I tried your implementation as given here tensorflow/tensorflow#9724 . But that is also not working, the output is almost entirely zeros with some noise values in the output.. The problem,I think is due to the moving mean in batch normalization. Do you know the solution for this?
I've added more description here. tensorflow/tensorflow#12450
Oh yes this is the exact problem I faced in the past. If you used the exact implementation I wrote in the tensorflow issue, there's a problem because you should set is_training=True for ENet. I'm not exactly sure why, but turns out this is very essential for the model performance. I highly suspect it has to do with the samples available in the dataset (i.e. the mean and variance for training are quite different from the moving averages used during inference). This could be due to a batch size limitation - because for segmentation, the batch size is usually much smaller than classification or other problems due to memory constraints, and so the moving averages computed might not be representative of the population mean and variances.
@kwotsin
I made the following changes
- Removed Batch Normalization completely from the network -
- Reduced Stage_two_repeat to one because I noticed many layers in the second stage had zero/close-to-zero weights' standard deviation and mean.
- Changed weighing to ENet weighing
- reduced the classes to just 4 (I chose classes which were relevant to my application)
and I was able to get over 95% streaming accuracy . And also the freezing of the weights also is not a problem now because there is no issue of batch normalization (I used the default TensorFlow freeze tool).
Great to hear that it is working for you. The removal of batch norm is a little surprising, but I'll take note of it when experimenting in the future. I find that for ENet, several class predictions actually work quite well given say around a few hundred images, as the learning capacity is quite appropriate at that level.
