Skip to content

Conversation

@xefoci7612
Copy link

Currently we handle the UCI_Elo with a double randomization. This
seems not necessary and a bit involuted.

This patch removes the first randomization and unifies the 2 cases.

No functional change.

@snicolet
Copy link
Member

Any feedback on this PR?

@vondele
Copy link
Member

vondele commented Oct 18, 2021

this looks reasonable, but would be good to see if things are actually equivalent, it is, for a given UCI_Elo a functional change. Probably a match could be done with master and patch for a given UCI_Elo, to check this doesn't introduce a large change.

Note that it also overlaps with what is discussed in #3635

@xefoci7612
Copy link
Author

Current implementation maps ELO values into a (float) level

double floatLevel = Options["UCI_LimitStrength"] ?
                      std::clamp(std::pow((Options["UCI_Elo"] - 1346.6) / 143.4, 1 / 0.806), 0.0, 20.0) :
                        double(Options["Skill Level"]);

And then converts the level into the upper or lower integer according to a statistical formula that takes into account how much the float differs from the lower bounding integer.

int intLevel = int(floatLevel) +
                 ((floatLevel - int(floatLevel)) * 1024 > rng.rand<unsigned>() % 1024  ? 1 : 0);

If you agree, I'd propose the following.

  1. Verify smoothness: run tests among new patched engines (not vs master) with different UCI_Elo settings and verify that new formula is consistent, i.e. winning ratio increases smoothly with UCI_Elo

  2. Verify mapping: run test of new patch against master but without enabling the float to int statistical tweak, because it would just add noise for the target of this test, and verify that the two engines have comparable strenght. To do this we could test patch vs master setting an UCI_Elo for which floatLevel - int(floatLevel) == 0, so that second formula becomes moot.

@vondele
Copy link
Member

vondele commented Oct 23, 2021

I would suggest to run just 3 matches patch against master, both engines with the same UCI_Elo at fixed values (e.g. 1800, 2000, 2200), each for e.g. 20k games, and see that the Elo difference is 'small'. This can be done on fishtest, passing additional options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants