Make net nn-190f102a22c3.nnue the default net. #3576
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bench: 5113198
Member
|
Nice. Can I suggest you to make a PR with the exact change you made to the trainer to the nnue-trainer repository as well? |
MichaelB7
added a commit
to MichaelB7/nnue-pytorch
that referenced
this pull request
Jun 21, 2021
LTC https://tests.stockfishchess.org/tests/view/60d09f52b4c17000d679517f LLR: 2.93 (-2.94,2.94) <0.50,3.50> Total: 32184 W: 1100 L: 970 D: 30114 Ptnml(0-2): 10, 878, 14193, 994, 17 STC https://tests.stockfishchess.org/tests/view/60d086c02114332881e7368e LLR: 2.93 (-2.94,2.94) <-0.50,2.50> Total: 11360 W: 1056 L: 906 D: 9398 Ptnml(0-2): 25, 735, 4026, 853, 41 official-stockfish/Stockfish#3576 official-stockfish#130
MichaelB7
added a commit
to MichaelB7/Stockfish
that referenced
this pull request
Jul 6, 2021
Trained with the pytorch trainer: https://github.com/glinscott/nnue-pytorch python train.py i:/bin/all.binpack i:/bin/all.binpack --gpus 1 --threads 4 --num-workers 30 --batch-size 16384 --progress_bar_refresh_rate 300 --smart-fen-skipping --random-fen-skipping 3 --features=HalfKAv2^ --lambda=1.0 --max_epochs=440 --seed %random%%random% --default_root_dir exp/run_17 --resume-from-model ./pt/nn-75980ca503c6.pt This run is thus started from the previous master net. all.binpack equaled 4 parts Wrong_NNUE_2.binpack https://drive.google.com/file/d/1seGNOqcVdvK_vPNq98j-zV3XPE5zWAeq/view?usp=sharing plus two parts of Training_Data.binpack https://drive.google.com/file/d/1RFkQES3DpsiJqsOtUshENtzPfFgUmEff/view?usp=sharing Each set was concatenated together - making one large Wrong_NNUE 2 binpack and one large Training so the were approximately equal in size. They were then interleaved together. The idea was to give Wrong_NNUE.binpack closer to equal weighting with the Training_Data binpack passed LTC https://tests.stockfishchess.org/tests/view/60d09f52b4c17000d679517f LLR: 2.93 (-2.94,2.94) <0.50,3.50> Total: 32184 W: 1100 L: 970 D: 30114 Ptnml(0-2): 10, 878, 14193, 994, 17 passed STC https://tests.stockfishchess.org/tests/view/60d086c02114332881e7368e LLR: 2.93 (-2.94,2.94) <-0.50,2.50> Total: 11360 W: 1056 L: 906 D: 9398 Ptnml(0-2): 25, 735, 4026, 853, 41 closes official-stockfish#3576 Bench: 4631244
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
LTC
https://tests.stockfishchess.org/tests/view/60d09f52b4c17000d679517f
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 32184 W: 1100 L: 970 D: 30114
Ptnml(0-2): 10, 878, 14193, 994, 17
STC
https://tests.stockfishchess.org/tests/view/60d086c02114332881e7368e
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 11360 W: 1056 L: 906 D: 9398
Ptnml(0-2): 25, 735, 4026, 853, 41
Trained with pytorch. https://github.com/glinscott/nnue-pytorch
python train.py i:/bin/all.binpack i:/bin/all.binpack --gpus 1 --threads 4 --num-workers 30 --batch-size 16384 --progress_bar_refresh_rate 300 --smart-fen-skipping --random-fen-skipping 3 --features=HalfKAv2^ --lambda=1.0 --max_epochs=440 --seed %random%%random% --default_root_dir exp/run_17 --resume-from-model ./pt/nn-75980ca503c6.pt (previous master)
all.binpack equaled 4 parts Wrong_NNUE_2.binpack https://drive.google.com/file/d/1seGNOqcVdvK_vPNq98j-zV3XPE5zWAeq/view?usp=sharing plus two parts of Training_Data.binpack https://drive.google.com/file/d/1RFkQES3DpsiJqsOtUshENtzPfFgUmEff/view?usp=sharing
Each set was concatenated together - making one large Wrong_NNUE 2 binpack and one large Training so the were approximately equal in size. They were then interleaved together. The idea was to give Wrong_NNUE.binpack closer to equal weighting with the Training_Data binpack
model.py changes:
t = outcome
p = (score / in_scaling).sigmoid()
loss = torch.pow(torch.abs(p - q), 2.6).mean()
def configure_optimizers(self):
# Train with a lower LR on the output layer
LR = 8.75e-4
gamma=0.992
source of suggestion: official-stockfish/nnue-pytorch#130 .
Pull request: official-stockfish/nnue-pytorch#132 .
Bench: 5113198