Inaccurate probabilityAlive calculations for some couples

Question

Inaccurate probabilityAlive calculations for some couples

brian-courts opened this issue 5 years ago · comments

When spouses are not the same age, the current probabilityAlive calculation can produce inaccurate results. If both spouses are under 62 and not disabled when beginning the calculations, each spouse's denominatorAge is taken as 62, and the base counts for mortality calculations are taken at that age. If they are born in 1958 and 1963, their base counts come from 2020 and 2025.

I do not believe an actuary would calculate joint probabilities for a couple taking data from different starting points. It can also create spurious results, such as a person's probabilityAlive calculated as greater than one and their probabilityDeceased less than zero, as shown in the attached spreadsheet. I have submitted a pullRequest with revisions to address this issue.

ProbabilityAliveCouple.pdf

MikePiper · Answer 1 · Fri Aug 02 2019 20:39:48 GMT+0800 (China Standard Time)

Thanks, Brian!

My busy period with my "normal" work is winding down, so I should have more time to look at this project again going forward. (Though this very next week is a week of travel.)

Also, I'm now "watching" this repository. I had assumed that because it was a repository I started, I would automatically be emailed when somebody created an "issue." Apparently not. Lesson learned.

MikePiper · Answer 2 · Tue Aug 13 2019 03:01:36 GMT+0800 (China Standard Time)

Hi Brian.

I've been looking this over today.

Great idea with the baseFactor concept to reduce the number of operations necessary.

You're absolutely right that the existing logic results in problems when the couple differs significantly in age.

After looking through it all, my primary thought is: maybe when implementing your suggestion we should also completely drop the assumption that a person lives to 62, rather than just dropping that assumption in some cases. It seems that would simplify the logic -- as well as make it more internally consistent.

It feels like I made a mistake in including that assumption in the first place. (I've obviously been away from deep work on this project for a while and am just ramping back up. So it's possible that that assumption was necessary for some reason that I cannot remember offhand. I'll poke around for a bit before actually making any changes.)

Thanks again.

Brian Courts · Answer 3 · Tue Aug 13 2019 23:42:34 GMT+0800 (China Standard Time)

I'm glad that you are still open to changes in this massive project, which could benefit lots of people. I look forward to learning how you proceed. On Monday, August 12, 2019, 02:01:38 PM CDT, MikePiper <notifications@github.com> wrote: Hi Brian. I've been looking this over today. Great idea with the baseFactor concept to reduce the number of operations necessary. You're absolutely right that the existing logic results in problems when the couple differs significantly in age. After looking through it all, my primary thought is: maybe when implementing your suggestion we should also completely drop the assumption that a person lives to 62, rather than just dropping that assumption in some cases. It seems that would simplify the logic -- as well as make it more internally consistent. It feels like I made a mistake in including that assumption in the first place. (I've obviously been away from deep work on this project for a while and am just ramping back up. So it's possible that that assumption was necessary for some reason that I cannot remember offhand. I'll poke around for a bit before actually making any changes.) Thanks again. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

MikePiper · Answer 4 · Thu Aug 15 2019 09:19:00 GMT+0800 (China Standard Time)

Hi Brian.

I just pushed out an update that addresses the issue you raised here (and which incorporates your idea of a base factor, to eliminate some unnecessary repeat calculations).

I also did just completely eliminate the assumption that anybody automatically lives to 62.

After going over some old notes, I realized that I had two reasons for originally including that assumption, and neither applies anymore. (The larger reason was that I was thinking that tests would break as "today" changes each month. I hadn't realized yet that I can hardcode a fake "today" for the sake of tests.)

Thanks again for pointing out the issue and for contributing the solution!

(I'll leave the issue open for now so that discussion can continue, in case you have any questions/concerns/suggestions.)

Brian Courts · Answer 5 · Sun Aug 18 2019 22:26:54 GMT+0800 (China Standard Time)

It's great that you were able to make those changes. I haven't had time to look at the update, but I expect that it now works as we thought it should. On Wednesday, August 14, 2019, 08:19:03 PM CDT, MikePiper <notifications@github.com> wrote: Hi Brian. I just pushed out an update that addresses the issue you raised here (and which incorporates your idea of a base factor, to eliminate some unnecessary repeat calculations). I also did just completely eliminate the assumption that anybody automatically lives to 62. After going over some old notes, I realized that I had two reasons for originally including that assumption, and neither applies anymore. (The larger reason was that I was thinking that tests would break as "today" changes each month. I hadn't realized yet that I can hardcode a fake "today" for the sake of tests.) Thanks again for pointing out the issue and for contributing the solution! (I'll leave the issue open for now so that discussion can continue, in case you have any questions/concerns/suggestions.) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

Brian Courts · Answer 6 · Sat Jan 18 2020 01:59:38 GMT+0800 (China Standard Time)

Thanks for making those adjustments. I saw in the Bogleheads forum that you corrected a bug regarding married couple calculations. I hope my proposed changes did not cause that bug. I read some of the recent items in that forum and agree with the consensus that your program is a great tool. I am impressed that you continue to update it as SSA changes their calculation parameters. Thanks again, and keep up the good work. On Thursday, January 16, 2020, 06:43:45 AM CST, MikePiper <notifications@github.com> wrote: Closed #7. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

MikePiper · Answer 7 · Sun Jan 19 2020 05:55:28 GMT+0800 (China Standard Time)

Hi Brian.

Your contribution did not cause any problems. Thank you again for taking the time.