Commit e1044c1
Dual NNUE with L1-128 smallnet
Credit goes to @mstembera for:
- writing the code enabling dual NNUE: #4898
- the idea of trying L1-128 trained exclusively on high simple eval positions
The L1-128 smallnet is:
- epoch 399 of a single-stage training from scratch
- trained only on positions from filtered data with high material difference
- defined by abs(simple_eval) > 1000
```yaml
experiment-name: 128--S1-only-hse-v2
training-dataset:
- /data/hse/S3/dfrc99-16tb7p-eval-filt-v2.min.high-simple-eval-1k.binpack
- /data/hse/S3/leela96-filt-v2.min.high-simple-eval-1k.binpack
- /data/hse/S3/test80-apr2022-16tb7p.min.high-simple-eval-1k.binpack
- /data/hse/S7/test60-2020-2tb7p.v6-3072.high-simple-eval-1k.binpack
- /data/hse/S7/test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack
- /data/hse/S7/test77-nov2021-2tb7p.v6-3072.min.high-simple-eval-1k.binpack
- /data/hse/S7/test77-dec2021-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack
- /data/hse/S7/test77-jan2022-2tb7p.high-simple-eval-1k.binpack
- /data/hse/S7/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack
- /data/hse/S7/test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack
- /data/hse/S7/test79-apr2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack
- /data/hse/S7/test79-may2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack
# T80 2022
- /data/hse/S7/test80-may2022-16tb7p.high-simple-eval-1k.binpack
- /data/hse/S7/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack
- /data/hse/S7/test80-jul2022-16tb7p.v6-dd.min.high-simple-eval-1k.binpack
- /data/hse/S7/test80-aug2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack
- /data/hse/S7/test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack
- /data/hse/S7/test80-oct2022-16tb7p.v6-dd.high-simple-eval-1k.binpack
- /data/hse/S7/test80-nov2022-16tb7p-v6-dd.min.high-simple-eval-1k.binpack
# T80 2023
- /data/hse/S7/test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack
- /data/hse/S7/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack
- /data/hse/S7/test80-mar2023-2tb7p.v6-sk16.min.high-simple-eval-1k.binpack
- /data/hse/S7/test80-apr2023-2tb7p-filter-v6-sk16.min.high-simple-eval-1k.binpack
- /data/hse/S7/test80-may2023-2tb7p.v6.min.high-simple-eval-1k.binpack
- /data/hse/S7/test80-jun2023-2tb7p.v6-3072.min.high-simple-eval-1k.binpack
- /data/hse/S7/test80-jul2023-2tb7p.v6-3072.min.high-simple-eval-1k.binpack
- /data/hse/S7/test80-aug2023-2tb7p.v6.min.high-simple-eval-1k.binpack
- /data/hse/S7/test80-sep2023-2tb7p.high-simple-eval-1k.binpack
- /data/hse/S7/test80-oct2023-2tb7p.high-simple-eval-1k.binpack
start-from-engine-test-net: False
nnue-pytorch-branch: linrock/nnue-pytorch/L1-128
engine-test-branch: linrock/Stockfish/L1-128-nolazy
engine-base-branch: linrock/Stockfish/L1-128
num-epochs: 500
lambda: 1.0
```
Training data can be found at:
https://robotmoon.com/nnue-training-data/
Local elo at 25k nodes per move of
L1-128 smallnet (nnue-only eval) vs. L1-128 trained on standard S1 data:
nn-epoch399.nnue : -318.1 +/- 2.1
Passed STC:
https://tests.stockfishchess.org/tests/view/6574cb9d95ea6ba1fcd49e3b
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 62432 W: 15875 L: 15521 D: 31036
Ptnml(0-2): 177, 7331, 15872, 7633, 203
Passed LTC:
https://tests.stockfishchess.org/tests/view/6575da2d4d789acf40aaac6e
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 64830 W: 16118 L: 15738 D: 32974
Ptnml(0-2): 43, 7129, 17697, 7497, 49
bench 1485861
Co-authored-by: mstembera <m_stembera@yahoo.com>1 parent 36db936 commit e1044c1
File tree
13 files changed
+375
-220
lines changed- src
- nnue
13 files changed
+375
-220
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
791 | 791 | | |
792 | 792 | | |
793 | 793 | | |
| 794 | + | |
794 | 795 | | |
795 | 796 | | |
796 | 797 | | |
| |||
857 | 858 | | |
858 | 859 | | |
859 | 860 | | |
860 | | - | |
| 861 | + | |
861 | 862 | | |
862 | 863 | | |
863 | | - | |
| 864 | + | |
864 | 865 | | |
865 | 866 | | |
866 | | - | |
| 867 | + | |
867 | 868 | | |
868 | 869 | | |
869 | 870 | | |
| |||
907 | 908 | | |
908 | 909 | | |
909 | 910 | | |
910 | | - | |
| 911 | + | |
911 | 912 | | |
912 | 913 | | |
913 | 914 | | |
| |||
951 | 952 | | |
952 | 953 | | |
953 | 954 | | |
| 955 | + | |
| 956 | + | |
| 957 | + | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
| 965 | + | |
| 966 | + | |
| 967 | + | |
| 968 | + | |
| 969 | + | |
| 970 | + | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
| 978 | + | |
| 979 | + | |
| 980 | + | |
| 981 | + | |
| 982 | + | |
| 983 | + | |
| 984 | + | |
| 985 | + | |
| 986 | + | |
| 987 | + | |
| 988 | + | |
| 989 | + | |
| 990 | + | |
| 991 | + | |
| 992 | + | |
| 993 | + | |
| 994 | + | |
| 995 | + | |
| 996 | + | |
| 997 | + | |
| 998 | + | |
| 999 | + | |
| 1000 | + | |
954 | 1001 | | |
955 | 1002 | | |
956 | 1003 | | |
| |||
1073 | 1120 | | |
1074 | 1121 | | |
1075 | 1122 | | |
1076 | | - | |
| 1123 | + | |
1077 | 1124 | | |
1078 | 1125 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
25 | 26 | | |
26 | 27 | | |
27 | 28 | | |
| |||
43 | 44 | | |
44 | 45 | | |
45 | 46 | | |
46 | | - | |
| 47 | + | |
| 48 | + | |
47 | 49 | | |
48 | | - | |
49 | | - | |
50 | | - | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
51 | 56 | | |
52 | 57 | | |
53 | 58 | | |
54 | 59 | | |
55 | 60 | | |
56 | 61 | | |
57 | 62 | | |
58 | | - | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
59 | 66 | | |
60 | 67 | | |
61 | 68 | | |
| |||
66 | 73 | | |
67 | 74 | | |
68 | 75 | | |
69 | | - | |
70 | | - | |
71 | | - | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
72 | 81 | | |
73 | 82 | | |
74 | | - | |
75 | | - | |
| 83 | + | |
| 84 | + | |
76 | 85 | | |
77 | | - | |
| 86 | + | |
78 | 87 | | |
79 | 88 | | |
80 | | - | |
81 | | - | |
| 89 | + | |
82 | 90 | | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
| 91 | + | |
91 | 92 | | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
109 | 122 | | |
110 | 123 | | |
| 124 | + | |
111 | 125 | | |
112 | 126 | | |
113 | 127 | | |
114 | 128 | | |
115 | 129 | | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
| 130 | + | |
121 | 131 | | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
122 | 135 | | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
141 | 156 | | |
142 | | - | |
143 | | - | |
| 157 | + | |
| 158 | + | |
144 | 159 | | |
145 | | - | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
153 | 160 | | |
154 | 161 | | |
155 | | - | |
156 | 162 | | |
157 | 163 | | |
158 | 164 | | |
| |||
162 | 168 | | |
163 | 169 | | |
164 | 170 | | |
165 | | - | |
| 171 | + | |
166 | 172 | | |
167 | | - | |
168 | | - | |
169 | | - | |
| 173 | + | |
| 174 | + | |
170 | 175 | | |
| 176 | + | |
171 | 177 | | |
172 | 178 | | |
173 | 179 | | |
174 | 180 | | |
175 | | - | |
176 | | - | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
177 | 187 | | |
178 | 188 | | |
179 | 189 | | |
| |||
216 | 226 | | |
217 | 227 | | |
218 | 228 | | |
219 | | - | |
| 229 | + | |
220 | 230 | | |
221 | 231 | | |
222 | 232 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
25 | | - | |
26 | 24 | | |
27 | 25 | | |
28 | 26 | | |
| 27 | + | |
29 | 28 | | |
30 | 29 | | |
31 | 30 | | |
32 | 31 | | |
33 | 32 | | |
34 | | - | |
35 | 33 | | |
36 | 34 | | |
37 | | - | |
| 35 | + | |
38 | 36 | | |
39 | 37 | | |
40 | 38 | | |
41 | 39 | | |
42 | | - | |
| 40 | + | |
| 41 | + | |
43 | 42 | | |
44 | 43 | | |
45 | 44 | | |
| |||
0 commit comments