Giters

jaanli / variational-autoencoder

Variational autoencoder implemented in tensorflow and pytorch (including inverse autoregressive flow)

Home Page:https://jaan.io/what-is-variational-autoencoder-vae-tutorial/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Reduce_sum instead of reduce_mean

Kh4L opened this issue · comments

commented

Hi @altosaar

First, thank you for this small tutorial on using tf.distributions with a VAE!

elbo = tf.reduce_sum(expected_log_likelihood - kl, 0)

I wanted to know why you chose to use reduce_sum instead of reduce_mean when computing the ELBO?

Thanks!

commented

reduce_mean also works - there might be an advantage empirically if the gradients are very large. I chose the sum to match the math which hopefully makes the code easier to understand. The ELBO for the dataset is the sum of the per-datapoint ELBOs.