Skip to content

Commit 3e0de3f

Browse files
linrockmstembera
authored and
Disservin
committed
Dual NNUE with L1-128 smallnet
Credit goes to @mstembera for: - writing the code enabling dual NNUE: official-stockfish#4898 - the idea of trying L1-128 trained exclusively on high simple eval positions The L1-128 smallnet is: - epoch 399 of a single-stage training from scratch - trained only on positions from filtered data with high material difference - defined by abs(simple_eval) > 1000 ```yaml experiment-name: 128--S1-only-hse-v2 training-dataset: - /data/hse/S3/dfrc99-16tb7p-eval-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/leela96-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/test80-apr2022-16tb7p.min.high-simple-eval-1k.binpack - /data/hse/S7/test60-2020-2tb7p.v6-3072.high-simple-eval-1k.binpack - /data/hse/S7/test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test77-nov2021-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test77-dec2021-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test77-jan2022-2tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test79-apr2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test79-may2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack # T80 2022 - /data/hse/S7/test80-may2022-16tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-jul2022-16tb7p.v6-dd.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-aug2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-oct2022-16tb7p.v6-dd.high-simple-eval-1k.binpack - /data/hse/S7/test80-nov2022-16tb7p-v6-dd.min.high-simple-eval-1k.binpack # T80 2023 - /data/hse/S7/test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-mar2023-2tb7p.v6-sk16.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-apr2023-2tb7p-filter-v6-sk16.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-may2023-2tb7p.v6.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-jun2023-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-jul2023-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-aug2023-2tb7p.v6.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-sep2023-2tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test80-oct2023-2tb7p.high-simple-eval-1k.binpack start-from-engine-test-net: False nnue-pytorch-branch: linrock/nnue-pytorch/L1-128 engine-test-branch: linrock/Stockfish/L1-128-nolazy engine-base-branch: linrock/Stockfish/L1-128 num-epochs: 500 lambda: 1.0 ``` Experiment yaml configs converted to easy_train.sh commands with: https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py Binpacks interleaved at training time with: official-stockfish/nnue-pytorch#259 Data filtered for high simple eval positions with: https://github.com/linrock/nnue-data/blob/32d6a68/filter_high_simple_eval_plain.py https://github.com/linrock/Stockfish/blob/61dbfe/src/tools/transform.cpp#L626-L655 Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move of L1-128 smallnet (nnue-only eval) vs. L1-128 trained on standard S1 data: nn-epoch399.nnue : -318.1 +/- 2.1 Passed STC: https://tests.stockfishchess.org/tests/view/6574cb9d95ea6ba1fcd49e3b LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 62432 W: 15875 L: 15521 D: 31036 Ptnml(0-2): 177, 7331, 15872, 7633, 203 Passed LTC: https://tests.stockfishchess.org/tests/view/6575da2d4d789acf40aaac6e LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 64830 W: 16118 L: 15738 D: 32974 Ptnml(0-2): 43, 7129, 17697, 7497, 49 closes https://github.com/official-stockfish/Stockfish/pulls Bench: 1330050 Co-Authored-By: mstembera <5421953+mstembera@users.noreply.github.com>
1 parent ec7409f commit 3e0de3f

File tree

12 files changed

+309
-208
lines changed

12 files changed

+309
-208
lines changed

‎src/Makefile‎

Lines changed: 20 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -806,7 +806,7 @@ help:
806806
@echo "help > Display architecture details"
807807
@echo "profile-build > standard build with profile-guided optimization"
808808
@echo "build > skip profile-guided optimization"
809-
@echo "net > Download the default nnue net"
809+
@echo "net > Download the default nnue nets"
810810
@echo "strip > Strip executable"
811811
@echo "install > Install executable"
812812
@echo "clean > Clean up"
@@ -922,16 +922,7 @@ profileclean:
922922
@rm -f stockfish.res
923923
@rm -f ./-lstdc++.res
924924

925-
# set up shell variables for the net stuff
926-
netvariables:
927-
$(eval nnuenet := $(shell grep EvalFileDefaultName evaluate.h | grep define | sed 's/.*\(nn-[a-z0-9]\{12\}.nnue\).*/\1/'))
928-
$(eval nnuedownloadurl1 := https://tests.stockfishchess.org/api/nn/$(nnuenet))
929-
$(eval nnuedownloadurl2 := https://github.com/official-stockfish/networks/raw/master/$(nnuenet))
930-
$(eval curl_or_wget := $(shell if hash curl 2>/dev/null; then echo "curl -skL"; elif hash wget 2>/dev/null; then echo "wget -qO-"; fi))
931-
$(eval shasum_command := $(shell if hash shasum 2>/dev/null; then echo "shasum -a 256 "; elif hash sha256sum 2>/dev/null; then echo "sha256sum "; fi))
932-
933-
# evaluation network (nnue)
934-
net: netvariables
925+
define fetch_network
935926
@echo "Default net: $(nnuenet)"
936927
@if [ "x$(curl_or_wget)" = "x" ]; then \
937928
echo "Neither curl nor wget is installed. Install one of these tools unless the net has been downloaded manually"; \
@@ -966,7 +957,24 @@ net: netvariables
966957
if [ "$(nnuenet)" = "nn-"`$(shasum_command) $(nnuenet) | cut -c1-12`".nnue" ]; then \
967958
echo "Network validated"; break; \
968959
fi; \
969-
fi; \
960+
fi;
961+
endef
962+
963+
# set up shell variables for the net stuff
964+
define netvariables
965+
$(eval nnuenet := $(shell grep $(1) evaluate.h | grep define | sed 's/.*\(nn-[a-z0-9]\{12\}.nnue\).*/\1/'))
966+
$(eval nnuedownloadurl1 := https://tests.stockfishchess.org/api/nn/$(nnuenet))
967+
$(eval nnuedownloadurl2 := https://github.com/official-stockfish/networks/raw/master/$(nnuenet))
968+
$(eval curl_or_wget := $(shell if hash curl 2>/dev/null; then echo "curl -skL"; elif hash wget 2>/dev/null; then echo "wget -qO-"; fi))
969+
$(eval shasum_command := $(shell if hash shasum 2>/dev/null; then echo "shasum -a 256 "; elif hash sha256sum 2>/dev/null; then echo "sha256sum "; fi))
970+
endef
971+
972+
# evaluation network (nnue)
973+
net:
974+
$(call netvariables, EvalFileDefaultNameBig)
975+
$(call fetch_network)
976+
$(call netvariables, EvalFileDefaultNameSmall)
977+
$(call fetch_network)
970978

971979
format:
972980
$(CLANG-FORMAT) -i $(SRCS) $(HEADERS) -style=file

‎src/evaluate.cpp‎

Lines changed: 98 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -23,18 +23,20 @@
2323
#include <cmath>
2424
#include <cstdlib>
2525
#include <fstream>
26+
#include <initializer_list>
2627
#include <iomanip>
2728
#include <iostream>
2829
#include <sstream>
2930
#include <vector>
3031

31-
#include "uci.h"
3232
#include "incbin/incbin.h"
3333
#include "misc.h"
3434
#include "nnue/evaluate_nnue.h"
35+
#include "nnue/nnue_architecture.h"
3536
#include "position.h"
36-
#include "types.h"
3737
#include "search.h"
38+
#include "types.h"
39+
#include "uci.h"
3840

3941
// Macro to embed the default efficiently updatable neural network (NNUE) file
4042
// data in the engine binary (using incbin.h, by Dale Weiler).
@@ -44,18 +46,26 @@
4446
// const unsigned int gEmbeddedNNUESize; // the size of the embedded file
4547
// Note that this does not work in Microsoft Visual Studio.
4648
#if !defined(_MSC_VER) && !defined(NNUE_EMBEDDING_OFF)
47-
INCBIN(EmbeddedNNUE, EvalFileDefaultName);
49+
INCBIN(EmbeddedNNUEBig, EvalFileDefaultNameBig);
50+
INCBIN(EmbeddedNNUESmall, EvalFileDefaultNameSmall);
4851
#else
49-
const unsigned char gEmbeddedNNUEData[1] = {0x0};
50-
const unsigned char* const gEmbeddedNNUEEnd = &gEmbeddedNNUEData[1];
51-
const unsigned int gEmbeddedNNUESize = 1;
52+
const unsigned char gEmbeddedNNUEBigData[1] = {0x0};
53+
const unsigned char* const gEmbeddedNNUEBigEnd = &gEmbeddedNNUEBigData[1];
54+
const unsigned int gEmbeddedNNUEBigSize = 1;
55+
const unsigned char gEmbeddedNNUESmallData[1] = {0x0};
56+
const unsigned char* const gEmbeddedNNUESmallEnd = &gEmbeddedNNUESmallData[1];
57+
const unsigned int gEmbeddedNNUESmallSize = 1;
5258
#endif
5359

5460

5561
namespace Stockfish {
5662

5763
namespace Eval {
5864

65+
std::string currentEvalFileName[2] = {"None", "None"};
66+
const std::string EvFiles[2] = {"EvalFile", "EvalFileSmall"};
67+
const std::string EvFileNames[2] = {EvalFileDefaultNameBig, EvalFileDefaultNameSmall};
68+
5969
// Tries to load a NNUE network at startup time, or when the engine
6070
// receives a UCI command "setoption name EvalFile value nn-[a-z0-9]{12}.nnue"
6171
// The name of the NNUE network is always retrieved from the EvalFile option.
@@ -64,87 +74,99 @@ namespace Eval {
6474
// in the engine directory. Distro packagers may define the DEFAULT_NNUE_DIRECTORY
6575
// variable to have the engine search in a special directory in their distro.
6676
void NNUE::init(const std::string& uciEvalFile,
67-
std::string& currentEvalFileName,
68-
const std::string& binaryDirectory) {
77+
const std::string& binaryDirectory,
78+
const OptionsMap& Options) {
6979

70-
std::string eval_file = uciEvalFile;
71-
if (eval_file.empty())
72-
eval_file = EvalFileDefaultName;
80+
for (NetSize netSize : {Big, Small})
81+
{
82+
// change after fishtest supports EvalFileSmall
83+
std::string eval_file =
84+
std::string(netSize == Small ? EvalFileDefaultNameSmall : Options[EvFiles[netSize]]);
85+
if (eval_file.empty())
86+
eval_file = EvFileNames[netSize];
7387

7488
#if defined(DEFAULT_NNUE_DIRECTORY)
75-
std::vector<std::string> dirs = {"<internal>", "", binaryDirectory,
76-
stringify(DEFAULT_NNUE_DIRECTORY)};
89+
std::vector<std::string> dirs = {"<internal>", "", binaryDirectory,
90+
stringify(DEFAULT_NNUE_DIRECTORY)};
7791
#else
78-
std::vector<std::string> dirs = {"<internal>", "", binaryDirectory};
92+
std::vector<std::string> dirs = {"<internal>", "", binaryDirectory};
7993
#endif
8094

81-
for (const std::string& directory : dirs)
82-
if (currentEvalFileName != eval_file)
95+
for (const std::string& directory : dirs)
8396
{
84-
if (directory != "<internal>")
97+
if (currentEvalFileName[netSize] != eval_file)
8598
{
86-
std::ifstream stream(directory + eval_file, std::ios::binary);
87-
if (NNUE::load_eval(eval_file, stream))
88-
currentEvalFileName = eval_file;
89-
}
90-
91-
if (directory == "<internal>" && eval_file == EvalFileDefaultName)
92-
{
93-
// C++ way to prepare a buffer for a memory stream
94-
class MemoryBuffer: public std::basic_streambuf<char> {
95-
public:
96-
MemoryBuffer(char* p, size_t n) {
97-
setg(p, p, p + n);
98-
setp(p, p + n);
99-
}
100-
};
101-
102-
MemoryBuffer buffer(
103-
const_cast<char*>(reinterpret_cast<const char*>(gEmbeddedNNUEData)),
104-
size_t(gEmbeddedNNUESize));
105-
(void) gEmbeddedNNUEEnd; // Silence warning on unused variable
106-
107-
std::istream stream(&buffer);
108-
if (NNUE::load_eval(eval_file, stream))
109-
currentEvalFileName = eval_file;
99+
if (directory != "<internal>")
100+
{
101+
std::ifstream stream(directory + eval_file, std::ios::binary);
102+
if (NNUE::load_eval(eval_file, stream, netSize))
103+
currentEvalFileName[netSize] = eval_file;
104+
}
105+
106+
if (directory == "<internal>" && eval_file == EvFileNames[netSize])
107+
{
108+
// C++ way to prepare a buffer for a memory stream
109+
class MemoryBuffer: public std::basic_streambuf<char> {
110+
public:
111+
MemoryBuffer(char* p, size_t n) {
112+
setg(p, p, p + n);
113+
setp(p, p + n);
114+
}
115+
};
116+
117+
MemoryBuffer buffer(
118+
const_cast<char*>(reinterpret_cast<const char*>(
119+
netSize == Small ? gEmbeddedNNUESmallData : gEmbeddedNNUEBigData)),
120+
size_t(netSize == Small ? gEmbeddedNNUESmallSize : gEmbeddedNNUEBigSize));
121+
(void) gEmbeddedNNUEBigEnd; // Silence warning on unused variable
122+
(void) gEmbeddedNNUESmallEnd;
123+
124+
std::istream stream(&buffer);
125+
if (NNUE::load_eval(eval_file, stream, netSize))
126+
currentEvalFileName[netSize] = eval_file;
127+
}
110128
}
111129
}
130+
}
112131
}
113132

114133
// Verifies that the last net used was loaded successfully
115-
void NNUE::verify(const std::string& uciEvalFile, const std::string& currentEvalFileName) {
134+
void NNUE::verify(const std::string& uciEvalFile, const OptionsMap& Options) {
116135

117-
std::string eval_file = uciEvalFile;
118-
if (eval_file.empty())
119-
eval_file = EvalFileDefaultName;
120-
121-
if (currentEvalFileName != eval_file)
136+
for (NetSize netSize : {Big, Small})
122137
{
138+
// change after fishtest supports EvalFileSmall
139+
std::string eval_file =
140+
std::string(netSize == Small ? EvalFileDefaultNameSmall : Options[EvFiles[netSize]]);
141+
if (eval_file.empty())
142+
eval_file = EvFileNames[netSize];
123143

124-
std::string msg1 =
125-
"Network evaluation parameters compatible with the engine must be available.";
126-
std::string msg2 = "The network file " + eval_file + " was not loaded successfully.";
127-
std::string msg3 = "The UCI option EvalFile might need to specify the full path, "
128-
"including the directory name, to the network file.";
129-
std::string msg4 = "The default net can be downloaded from: "
130-
"https://tests.stockfishchess.org/api/nn/"
131-
+ std::string(EvalFileDefaultName);
132-
std::string msg5 = "The engine will be terminated now.";
133-
134-
sync_cout << "info string ERROR: " << msg1 << sync_endl;
135-
sync_cout << "info string ERROR: " << msg2 << sync_endl;
136-
sync_cout << "info string ERROR: " << msg3 << sync_endl;
137-
sync_cout << "info string ERROR: " << msg4 << sync_endl;
138-
sync_cout << "info string ERROR: " << msg5 << sync_endl;
139-
140-
exit(EXIT_FAILURE);
141-
}
144+
if (currentEvalFileName[netSize] != eval_file)
145+
{
146+
std::string msg1 =
147+
"Network evaluation parameters compatible with the engine must be available.";
148+
std::string msg2 = "The network file " + eval_file + " was not loaded successfully.";
149+
std::string msg3 = "The UCI option EvalFile might need to specify the full path, "
150+
"including the directory name, to the network file.";
151+
std::string msg4 = "The default net can be downloaded from: "
152+
"https://tests.stockfishchess.org/api/nn/"
153+
+ std::string(EvFileNames[netSize]);
154+
std::string msg5 = "The engine will be terminated now.";
155+
156+
sync_cout << "info string ERROR: " << msg1 << sync_endl;
157+
sync_cout << "info string ERROR: " << msg2 << sync_endl;
158+
sync_cout << "info string ERROR: " << msg3 << sync_endl;
159+
sync_cout << "info string ERROR: " << msg4 << sync_endl;
160+
sync_cout << "info string ERROR: " << msg5 << sync_endl;
161+
162+
exit(EXIT_FAILURE);
163+
}
142164

143-
sync_cout << "info string NNUE evaluation using " << eval_file << sync_endl;
165+
sync_cout << "info string NNUE evaluation using " << eval_file << sync_endl;
166+
}
144167
}
145168
}
146169

147-
148170
// Returns a static, purely materialistic evaluation of the position from
149171
// the point of view of the given color. It can be divided by PawnValue to get
150172
// an approximation of the material advantage on the board in terms of pawns.
@@ -163,18 +185,19 @@ Value Eval::evaluate(const Position& pos, const Search::Worker& workerThread) {
163185
int v;
164186
Color stm = pos.side_to_move();
165187
int shuffling = pos.rule50_count();
166-
int simpleEval = simple_eval(pos, stm) + (int(pos.key() & 7) - 3);
167-
168-
bool lazy = std::abs(simpleEval) >= RookValue + KnightValue + 16 * shuffling * shuffling
169-
+ std::abs(workerThread.iterBestValue)
170-
+ std::abs(workerThread.rootSimpleEval);
188+
int simpleEval = simple_eval(pos, stm);
171189

190+
bool lazy = std::abs(simpleEval) > 2300;
172191
if (lazy)
173192
v = simpleEval;
174193
else
175194
{
176-
int nnueComplexity;
177-
Value nnue = NNUE::evaluate(pos, true, &nnueComplexity);
195+
bool smallNet = std::abs(simpleEval) > 1100;
196+
197+
int nnueComplexity;
198+
199+
Value nnue = smallNet ? NNUE::evaluate<NNUE::Small>(pos, true, &nnueComplexity)
200+
: NNUE::evaluate<NNUE::Big>(pos, true, &nnueComplexity);
178201

179202
int optimism = workerThread.optimism[stm];
180203

@@ -217,7 +240,7 @@ std::string Eval::trace(Position& pos, Search::Worker& workerThread) {
217240
ss << std::showpoint << std::showpos << std::fixed << std::setprecision(2) << std::setw(15);
218241

219242
Value v;
220-
v = NNUE::evaluate(pos, false);
243+
v = NNUE::evaluate<NNUE::Big>(pos, false);
221244
v = pos.side_to_move() == WHITE ? v : -v;
222245
ss << "NNUE evaluation " << 0.01 * UciHandler::to_cp(v) << " (white side)\n";
223246

‎src/evaluate.h‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,15 +40,17 @@ Value evaluate(const Position& pos, const Search::Worker& workerThread);
4040
// The default net name MUST follow the format nn-[SHA256 first 12 digits].nnue
4141
// for the build process (profile-build and fishtest) to work. Do not change the
4242
// name of the macro, as it is used in the Makefile.
43-
#define EvalFileDefaultName "nn-b1e55edbea57.nnue"
43+
#define EvalFileDefaultNameBig "nn-b1e55edbea57.nnue"
44+
#define EvalFileDefaultNameSmall "nn-c01dc0ffeede.nnue"
4445

4546
namespace NNUE {
4647

47-
void init(const std::string&, std::string& currentEvalFileName, const std::string& binaryDirector);
48-
void verify(const std::string&, const std::string& currentEvalFileName);
48+
void init(const std::string&, const std::string& binaryDirector, const OptionsMap& Options);
49+
void verify(const std::string&, const OptionsMap& Options);
4950

5051
} // namespace NNUE
5152

53+
extern std::string currentEvalFileName[2];
5254
} // namespace Eval
5355

5456
} // namespace Stockfish

‎src/main.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ int main(int argc, char* argv[]) {
3838
Tune::init(uci.options);
3939
Bitboards::init();
4040
Position::init();
41-
Eval::NNUE::init(uci.options["EvalFile"], uci.currentEvalFileName, uci.workingDirectory());
41+
Eval::NNUE::init(uci.options["EvalFile"], uci.workingDirectory(), uci.options);
4242

4343
uci.loop();
4444

0 commit comments

Comments
 (0)