Is your model CNN or BCNN?
ShellingFord221 opened this issue · comments
Hi, I have read your paper and code. In file Dropout_Bald_Q10_N1000_Paper.py, it seems that your model is a CNN in Keras, but in your paper, the model is a BCNN with prior probability distributions placed over its weights. So how do you implement BCNN by Keras? Especially the implementation of training method (Bayes by Backprop, for example) .
Besides, could you please tell me which code file is corresponding to section 5.2. Importance of model uncertainty in your paper (i.e. the code that gets results in Figure 2)?
Thanks!
I think it s a BCNN because of the dropout layers
But if so, then all the neural networks with just dropout layers can be called Bayesian neural networks? I don't think so!
As @damienlancry say, the experiment in section 5.2 is a comparison of CNN with dropout layers and CNN without dropout layers. But dropout layer is very common in recent models, who would use a CNN without dropout layers? It doesn't make sense!
Dropout is commonly used during training as a regularization technique to avoid overfitting since 2012. however it is not common to use dropout at test time. dropout layers are usually deactivated at test time thanks to model.eval() in pytorch for example. In a bayesian setting, dropout is never deactivated, we run several passes through the stochastic network to obtain a distribution of outputs and compute uncertainties.
Thanks! So in a word, a BCNN is a CNN using dropout layers at test time, but the same as common CNN when training?
I think so yeah
I have another question... In MC_Dropout_Keras/Dropout_Bald_Q10_N1000_Paper.py for example, the Bayesian part I think is shown in line 228:
dropout_score = model.predict_stochastic(X_Pool_Dropout,batch_size=batch_size, verbose=1)
, which measures the uncertainties. But I didn't find the implementation of the function predict_stochastic...
It is a method they implemented themselves, it is not implemented in the common keras available with pip, that is why they included their own version of keras in this repository
To use it, you have to add this keras to your python path. Personnally I simply added that to the beginning of every script:
import sys
sys,path.append("/home/damien/Documents/Deep-Bayesian-Active-Learning/keras")
@damienlancry Thank you so much for your kindly help! I still wonder that if there is only 1 new function, is it necessary to rewrite the whole keras? I don't know much about keras, sorry :(
Besides, is there any Pytorch version of BCNN? I think it will be a little bit easy in Pytorch to implement MC dropout when testing.