stan-dev / loo

loo R package for approximate leave-one-out cross-validation (LOO-CV) and Pareto smoothed importance sampling (PSIS)

Home Page:https://mc-stan.org/loo

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Calibration/Refinement Decomposition of LOO Scores

ParadaCarleton opened this issue · comments

I think a breakdown of LOO scores into calibration/refinement would be interesting and useful to users. Mentioning @avehtari because Jonah mentioned you'd proposed adding this decomposition and said you might have some related work.

@ParadaCarleton can you add here a reference to "calibration/refinement"

It has now been 2 years and I have no idea what I was talking about then 😄

But I think this was the relevant paper that inspired this suggestion, though I'm not 100% sure.