Giters

dfm / corner.py

Make some beautiful corner plots

Home Page:http://corner.readthedocs.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

A confusing note about sigmas

janewman-pitt-edu opened this issue · comments

commented

Hi --

I appreciate you prominently linking the 'note about sigmas' in the documentation, but I think there are a couple of issues with that page and related documentation that are likely to confuse people.

  1. It looks like the levels keyword is actually specifying the levels to plot as quantiles in the lnprob ? The documentation doesn't specify that anywhere. (It just says 'levels (array_like) – The contour levels to draw.').

  2. The page describes the default method of drawing contours as the 'correct' one, but it is NOT correct if you are trying to plot credible regions / confidence regions, so I find that labelling rather confusing -- I think it either needs to be changed or needs more caveating.

A 68% credible region (in N dimensions) should contain 68% of the probability over that N-dimensional space. It should not line up with the marginalized 68% regions in single dimensions; as the two are aiming at different things.

I tend to think that what people want from corner is credible regions, so making the default otherwise is rather non-intuitive...

commented

I'd be happy to merge a pull request that adds more detail to the levels description (I agree that it's a bit lacking currently) and perhaps a softening of the language in the "sigmas" tutorial, but I'm not going to change the default behavior because it currently does what I want (perhaps you and some other people want something different, and that's fine!).

commented

I'll try to remember this in a couple weeks when I have time.

I'm pretty surprised it's what you want, to be honest, since again, the default contours do not actually correspond to credible regions. I totally understand making sure the defaults are convenient for yourself, though :)

commented

hi guys, i agree with @janewman-pitt-edu : i have seen many papers where they say they are plotting 1, 2 sigma contours (meaning 68% and 95% prob) but they are erronously plotting the 39% and 86%. I understand that for @dfm 1, 2 sigmas are linear distances but the vast majority of researchers adopt the convention that 1, 2 sigma means 68% and 95% of probability. Many researchers (especially students) get confused and end up displaying wrong plots.

commented

I doubt that you have data backing up your "vast majority" there, but either way (as above) you're welcome to submit a pull request that adds wording that would satisfy you to the documentation. If your students are getting confused then you should encourage them to read the documentation of the code that they're using!

commented

the thing is that in the documentation the words "correct" and "incorrect" are used. I think this may confuse people.

commented

I'm afraid this moved off my memory queue long ago :) Sorry I haven't had a chance to make a change... if you ping me around late september (after the annual reviews deadline) I may be able to think about this again.

commented

@valerio-marra: that's not what you were concerned about in your previous comment and I disagree with you about the use of "correct" being confusing. It might annoy you (sorry!) but there's no ambiguity in that documentation about the amount of posterior density within the contour. If you feel that there is something missing from the documentation, then it would be great to see a pull request suggesting changes.

@janewman-pitt-edu: this was not meant as a ping (and I'm not going to ping you to ask you to do anything) because I only want people to make contributions that they're excited about!