vivekjoshy / openskill.py

Multiplayer Rating System. No Friction.

Home Page:https://openskill.me

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

predict_win and predict_rank do not work on 2x2 and more games

JustRoxy opened this issue · comments

Describe the bug
predict_win and predict_rank do not work properly on 3x3x3 games

To Reproduce
Step 1:

from openskill.models import PlackettLuce

model = PlackettLuce()

p1 = model.rating(mu=34, sigma=0.25)
p2 = model.rating(mu=34, sigma=0.25)
p3 = model.rating(mu=34, sigma=0.25)

p4 = model.rating(mu=32, sigma=0.5)
p5 = model.rating(mu=32, sigma=0.5)
p6 = model.rating(mu=32, sigma=0.5)

p7 = model.rating(mu=30, sigma=1)
p8 = model.rating(mu=30, sigma=1)
p9 = model.rating(mu=30, sigma=1)

team1, team2, team3 = [p1, p2, p3], [p4, p5, p6], [p7, p8, p9]

r = model.predict_win([team1, team2, team3])
print(r)

Results in:
[0.439077174955099, 0.3330210112526078, 0.2279018137922932]

Step 2, change p9 mu to 40:

from openskill.models import PlackettLuce

model = PlackettLuce()

p1 = model.rating(mu=34, sigma=0.25)
p2 = model.rating(mu=34, sigma=0.25)
p3 = model.rating(mu=34, sigma=0.25)

p4 = model.rating(mu=32, sigma=0.5)
p5 = model.rating(mu=32, sigma=0.5)
p6 = model.rating(mu=32, sigma=0.5)

p7 = model.rating(mu=30, sigma=1)
p8 = model.rating(mu=30, sigma=1)
p9 = model.rating(mu=40, sigma=1)

team1, team2, team3 = [p1, p2, p3], [p4, p5, p6], [p7, p8, p9]

print([team1, team2, team3])
r = model.predict_win([team1, team2, team3])
print(r)

Results are the same:
[0.439077174955099, 0.3330210112526078, 0.2279018137922932]

Expected behavior
After p9 mu increase team3 are expected to have a bigger chance of victory

Platform Information

  • openskill.py Version: 5.1.0

Additional context
https://github.com/OpenDebates/openskill.py/blob/f76df19c3e388f31050c988a0059367bd1dadc76/openskill/models/weng_lin/bradley_terry_full.py#L765

I have no idea what is going on here, and why it selects rating only of the first player, but it just does not work as intended

It seems I forgot to copy over the original code before the API overhaul. Amazing how it went unnoticed for so long.