How to translate a caffe model into a .rlv model?
RPastorino opened this issue · comments
Hi, i'm trying to learn how to use PLANET in a complete way because a want to use it for my master degree's final project about verification of neural nets.
I 've translatedt he original keras models into caffe models specially because i read about the tool for the conversion in the read me file.
Can someone teach me how to translate the caffe model into .rlv file for the verification?
Thanks in advance
R. Pastorino
Dear RPastorino,
the first step is to convert the ".prototxt" (as produced by caffe) into a "JSON" format. For this step, you use the script "caffemodel2json.py" that should be in the "tool" directory if you followed the setting-up instructions from the README file. Running
/path/to/caffemodel2json.py <inputFile> --data <outputFile>
will give you a JSON output file. This can then be translated to the network specification part of an RLV file with the "json_network_to_rlv_translator.py" script from the "tools" directory.
Note that the script currently supports only few layer types. That is not always a big deal, as during the learning phase, you can make sure that you only use the layer types supported. When you translate from a Keras model to a caffe model, I don't quite know what layers are defined by the translator, and you may have to extend the translations script with support for some new layer types before it works. Non-piecewise affine layers cannot be supported as the verification engine depends on all neuron activation functions to be piecewise affine.
Hi progirep, and thank you so much for the reply.
i was stuck with the converstion cause, in accordance with the script's help lines, it needs either the .prototxt file and the .caffemodel file that in fact i have after the converstion from keras( i had to use the MMdnn tool by microsoft for this task).
Btw, trying to follow both your way and that one suggested by the script creator, i obtain the following error:
Traceback (most recent call last):
File "caffemodel2json.py", line 56, in
subprocess.check_call(['protoc', '--proto_path', os.path.dirname(local_caffe_proto), '--python_out', args.codegenDir, local_caffe_proto])
File "/Users/riccardopastorino/anaconda2/lib/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['protoc', '--proto_path', '/Users/riccardopastorino/Desktop/toolsTesi', '--python_out', '/Users/riccardopastorino/Desktop/toolsTesi', '/Users/riccardopastorino/Desktop/toolsTesi/4x125relu+softmax.prototxt']' returned non-zero exit status 1
i've used these commands:
/path/to/caffemodel2json.py <path/to/inputFile.prototxt> --data <path/where/to/save/outputFile.json>
/path/to/caffemodel2json.py <path/to/inputFile.prototxt> <path/to/inputFile.caffemodel>
if someone has any suggestion it would be really appreciated.
Thanks in advance and rgds.
Can you check whether the "protoc" tool is installed on your computer? For instance, in Ubuntu Linux, you can get it by installing the package "protobuf-compiler". I will add this information to the README file.
I'm trying to run the conversion script on a mac with High Sierra 10.13.4 .
I've installed the protobuf compiler but the problem remains the same.
About the conversion from json to .rlv, i know that also the keras framework provide the possibility to save the architecture of a net in a json file.
I taken a look at the json_network_to_rlv_translator code but i don't understand if does it able to be working just with the json transleted from caffe models?
Btw, thanks again for the indications, i'm trying to translate and verify a simple MNIST classifier with few hundred of Relu nodes. Hence the model should respect the restrictions.
So the script assumes that the compiler can be run from the terminal without providing an explicit path. Does that work on the terminal of your Mac? Otherwise, you may want to slightly alter the script so that it contains a full path to the protobuf compiler.
It is certainly correct that the compiler script is a bit difficult to understand. All of the difficult work is done in the "recurseProcessLayer" function. At the end of the script, it is called on the last layer in the network. The "recurseProcessLayer" function works recursively returns the RLV file neuron names of a layer after processing the layer. The reason for this design decision was that the JSON files with the learned network do not contain the layer input sizes needed for the translation, and performing the translation recursively was easier than iterating over the layers in a big loop. All results are cached in the dictionary "neurons".
Some layers have a relatively straight-forward definition, such as the ReLU layer -- first the function calls itself recursively on the input to the ReLU layer, then the function generates RLV file lines for the ReLU behavior and finally it returns a list with all the ReLU layer node names.
In contrast, the "Convolution" layer definition is quite...convoluted... as the definition of this operation is a bit involved. The operation works with multi-dimensional data. The definition of this layer is quite similar to the one of a pooling layer, though, so if you understand the code for one operation, the other one is not easy.
I'm pretty unsure is developing a translation script from scratch or modifying the one for caffe models that comes with planet is easier for you -- I think that depends on how your Keras JSON models look like.
Hi Professor Ehlers, thanks again for the answer.
I've installed ubuntu 16.04 in dual boot on my mac book.
After the configuration i've tried to run the caffemodel2json script but unfortunately i obtain the same error as on mac os environment.
Can i know if you or your collaborators have succesfully run and thence convert some caffe models?
furthermore, thanks for the clear explanation above.
thx again and rgds
Riccardo P.
What is the output of runing "protoc" on the terminal (without giving any file name to protoc and without providing a full path to the protoc executable)?
the output is:
Usage: protoc [OPTION] PROTO_FILES
Parse PROTO_FILES and generate output based on the options given:
-IPATH, --proto_path=PATH Specify the directory in which to search for
imports. May be specified multiple times;
directories will be searched in order. If not
given, the current working directory is used.
--version Show version info and exit. ..
..and the others help's lines.
the version of the libprotoc is the 3.5.1 and i've already upgrade it in order to try if it was matter of library's version
Hi, using another model and specifying the path to the file caffe.proto i've obtain the following outoput:
{
"layer": [
{
"phase": 0,
"input_param": {
"shape": [
{
"dim": [
1,
784
]
}
]
},
"top": [
"input_1"
],
"type": "Input",
"name": "input_1"
},
{
"blobs": [
{
"shape": {
"dim": [
125,
784
]
},
"data": [
0.07819511741399765,
0.07097464054822922,
-0.040203168988227844,
0.040613606572151184,
-0.07233423739671707,
"(97995 elements more)"
]
},
{
"shape": {
"dim": [
125
]
},
"data": [
0.0,
0.0,
0.0,
0.0,
0.0,
"(120 elements more)"
]
}
],
"bottom": [
"input_1"
],
"inner_product_param": {
"bias_term": true,
"num_output": 125
},
"top": [
"dense_1"
],
"phase": 0,
"type": "InnerProduct",
"name": "dense_1"
},
{
"phase": 0,
"top": [
"dense_1"
],
"type": "ReLU",
"name": "dense_1_activation",
"bottom": [
"dense_1"
]
},
{
"blobs": [
{
"shape": {
"dim": [
125,
125
]
},
"data": [
-0.10211414098739624,
0.12129108607769012,
-0.1027422696352005,
-0.05443313717842102,
0.06391389667987823,
"(15620 elements more)"
]
},
{
"shape": {
"dim": [
125
]
},
"data": [
0.0,
0.0,
0.0,
0.0,
0.0,
"(120 elements more)"
]
}
],
"bottom": [
"dense_1"
],
"inner_product_param": {
"bias_term": true,
"num_output": 125
},
"top": [
"dense_2"
],
"phase": 0,
"type": "InnerProduct",
"name": "dense_2"
},
{
"phase": 0,
"top": [
"dense_2"
],
"type": "ReLU",
"name": "dense_2_activation",
"bottom": [
"dense_2"
]
},
{
"blobs": [
{
"shape": {
"dim": [
125,
125
]
},
"data": [
-0.10090261697769165,
0.11097587645053864,
0.029438048601150513,
0.1526545137166977,
0.12442095577716827,
"(15620 elements more)"
]
},
{
"shape": {
"dim": [
125
]
},
"data": [
0.0,
0.0,
0.0,
0.0,
0.0,
"(120 elements more)"
]
}
],
"bottom": [
"dense_2"
],
"inner_product_param": {
"bias_term": true,
"num_output": 125
},
"top": [
"dense_3"
],
"phase": 0,
"type": "InnerProduct",
"name": "dense_3"
},
{
"phase": 0,
"top": [
"dense_3"
],
"type": "ReLU",
"name": "dense_3_activation",
"bottom": [
"dense_3"
]
},
{
"blobs": [
{
"shape": {
"dim": [
125,
125
]
},
"data": [
0.12811337411403656,
-0.09762409329414368,
0.12216831743717194,
-0.02165699005126953,
0.061633601784706116,
"(15620 elements more)"
]
},
{
"shape": {
"dim": [
125
]
},
"data": [
0.0,
0.0,
0.0,
0.0,
0.0,
"(120 elements more)"
]
}
],
"bottom": [
"dense_3"
],
"inner_product_param": {
"bias_term": true,
"num_output": 125
},
"top": [
"dense_4"
],
"phase": 0,
"type": "InnerProduct",
"name": "dense_4"
},
{
"phase": 0,
"top": [
"dense_4"
],
"type": "ReLU",
"name": "dense_4_activation",
"bottom": [
"dense_4"
]
},
{
"blobs": [
{
"shape": {
"dim": [
10,
125
]
},
"data": [
0.16188667714595795,
-0.1518039107322693,
0.08921857178211212,
0.15013252198696136,
0.13840819895267487,
"(1245 elements more)"
]
},
{
"shape": {
"dim": [
10
]
},
"data": [
0.0,
0.0,
0.0,
0.0,
0.0,
"(5 elements more)"
]
}
],
"bottom": [
"dense_4"
],
"inner_product_param": {
"bias_term": true,
"num_output": 10
},
"top": [
"dense_5"
],
"phase": 0,
"type": "InnerProduct",
"name": "dense_5"
},
{
"phase": 0,
"top": [
"dense_5_activation"
],
"type": "Softmax",
"name": "dense_5_activation",
"bottom": [
"dense_5"
]
}
],
"name": ""
it seems to be the model in protobuf format, what could it mean?
thx in advance and rgds,
Riccardo
Hi RPastino,
I tried to dig into this and found out that the problem lies in using a version of "protoc" that is too new. In particular, version 3.5* appears to be problematic if other software is installed that depends on an older version (see., e.g., facebookarchive/caffe2#2441). Since the latest version that ships with Ubuntu is version 3.0.something, I believe that you installed some newer version by hand (e.g., using "make install"), which is causing the problem.
As far as your latest message is concerned: this appears to be exactly the JSON file that can then be translated to RLV. What concerns me a bit are the messages of the shape "(n elements more)", which I do not get when using "protoc" version 3.0 with a model generated by caffe.
It may be the case that the protobuf file that you generate uses a "FD.LABEL_REPEATED" output label. If this is true, try changing line 39 or the translator script to "js_value = ftype(value)" -- perhaps that solves this issue.
ok, thx. i've uninstall protoc and reinstall the 3.0.0 version.
about the messages of the form "( n elements more )" i've forgotten to add the command --data after the name of the script, with --data i see the entire sets of values.
Now i'm trying to convert the json in .rlv, running the script i get this error:
File "json_network_to_rlv_translator.py", line 12, in
data = data["layers"]
KeyError: 'layers'
I've removed the final s in "layers", and the output has become:
Warning: No 'Accuracy' layer found, hence nothing was translated.
thus i guess it is just a matter of "correct topology" of the net that the script can parses and translates, is not it?
rgds,
Riccardo
Hi Riccardo,
yes, you are correct -- the protobuf file that you got does not have exactly the same format as the ones that caffe produces for networks similar to the ones in the example. A layer of the type "Accuracy" is currently used by the script as starting point for the conversion. See http://caffe.berkeleyvision.org/tutorial/layers/accuracy.html for a description of the layer type. You could add a layer of that type to the network to solve this.
I do not know why the "layers" map is called "layer" for you -- perhaps your translation program targets a different version of caffe?