NVP_7: NVP_5 w/ increased decoder complexity
Micky774 opened this issue · comments
Meekail Zain commented
Add resnet blocks to NVP_5's decoder until it reaches ~5.4mil parameters (same as NVP_4). See #39
Meekail Zain commented
While designing a prototype NVP_7, I observed that our memory footprint goes from ~350MB to ~950MB. This translates into having to divide our operable batch-size by almost 3 and thus nearly tripling training costs in terms of gpu-hours. Hence, rather than comparable parameter count, we may want to aim for a comparable memory footprint when enhancing the decoder.