Giters

nvcuong / variational-continual-learning

Implementation of the variational continual learning method

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

get headnet params

qichenglao opened this issue · comments

commented

Hi, there is a typo here

variational-continual-learning/dgm/alg/onlinevi.py

Line 16 in 12e1883

var_list = [var for var in t_vars if 'gen_head%d' % task in var.name]

shouldn't it be if 'gen_%d_head' % task in var.name?

commented

@YingzhenLi, can you have a look?

commented

Hi qichenglao,

You are right, it should be 'gen_%d_head', however:

In practice I would suggest a Bayesian treatment for the shared network only. In other words, in our experiments, we used VCL to train the shared network, while we used MLE (thus no KL penalty) for the private head network for each task. You can also train the head network by adding this KL penalty as well (with the q_{t-1} for the private network being standard Gaussian), but I would imagine it to work worse.

commented

yeah, it eventually has the same effect as not computing the second part in KL_param function. Thanks a lot for your comments!

commented

Thanks, @YingzhenLi and @qichenglao. Since the bug has been fixed, I will close the issue.