speculator with adaptive mcmc

Question

speculator with adaptive mcmc

changhoonhahn opened this issue 4 years ago · comments

@kgb0255 implemented adaptive MCMC. We need to re-run LGal SED fits with speculator with this new feature.

ChangHoon Hahn · Answer 1 · Thu Apr 30 2020 05:09:47 GMT+0800 (China Standard Time)

In commit 5703716 , I've updated gqp_mc.fitters.py so that ARM is now optional. It only runs when you set kwarg niter='adaptive' and runs normal mcmc when you specify an integer.

@kgb0255 I've tested the updates locally but you should test that they all work on NERSC.

James Kwon · Answer 2 · Thu Apr 30 2020 17:57:17 GMT+0800 (China Standard Time)

@changhoonhahn Thank you for the update. I'll go ahead and test run some chains on NERSC.

James Kwon · Answer 3 · Sun May 03 2020 16:37:14 GMT+0800 (China Standard Time)

I received following error when I ran the chain on the interactive mode.

Traceback (most recent call last):
File "/global/homes/k/kgb0255/packages/gqp_mc/run/mini_mocha.py", line 743, in
justplot=justplot)
File "/global/homes/k/kgb0255/packages/gqp_mc/run/mini_mocha.py", line 509, in MP_sed_fit
for igal in args: fit_func(igal, **kwargs)
File "/global/homes/k/kgb0255/packages/gqp_mc/run/mini_mocha.py", line 312, in fit_spectrophotometry
specs, meta = Data.Spectra(sim=sim, noise=noise_spec, lib='bc03', sample='mini_mocha')
File "/global/homes/k/kgb0255/.local/lib/python3.7/site-packages/gqp_mc-0.1-py3.7.egg/gqp_mc/data.py", line 54, in Spectra
meta, mock = read_data(sim=sim, noise=noise, lib=lib, sample=sample)
File "/global/homes/k/kgb0255/.local/lib/python3.7/site-packages/gqp_mc-0.1-py3.7.egg/gqp_mc/data.py", line 153, in read_data
'lgal.%s.hdf5' % dat_descrip), 'r')
File "/global/homes/k/kgb0255/.conda/envs/gqp/lib/python3.7/site-packages/h5py/_hl/files.py", line 408, in init
swmr=swmr)
File "/global/homes/k/kgb0255/.conda/envs/gqp/lib/python3.7/site-packages/h5py/_hl/files.py", line 173, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (unable to lock file, errno = 524, error message = 'Unknown error 524')

This error was resolved when I added this in ~/.bashrc.ext

export HDF5_USE_FILE_LOCKING='FALSE'

ChangHoon Hahn · Answer 4 · Mon May 04 2020 00:39:23 GMT+0800 (China Standard Time)

@msiudek The above solution by @kgb0255 is also what I did to solve the issue.

James Kwon · Answer 5 · Wed May 06 2020 15:28:26 GMT+0800 (China Standard Time)

Chain was ran for igal = 2. Some walkers did not move at all. I sent the report to @changhoonhahn email address and it can also be found here. Plots are included in the report.