Skip to content

Commit 7412bd1

Browse files
authored
Release Igel 3.5.0 (#279)
Igel 3.5.0 ------------- 22-Jun-2023 - new network architecture - new network c049c117 http://chess.grantnet.us/test/32491/ ELO | 43.10 +- 3.70 (95%) CONF | 60.0+0.60s Threads=1 Hash=64MB GAMES | N: 17568 W: 5631 L: 3463 D: 8474 http://chess.grantnet.us/test/32490/ ELO | 38.72 +- 3.35 (95%) CONF | 10.0+0.10s Threads=1 Hash=8MB GAMES | N: 22976 W: 7639 L: 5089 D: 10248 bench: 17628169
1 parent 219b659 commit 7412bd1

File tree

10 files changed

+171
-282
lines changed

10 files changed

+171
-282
lines changed

README.md

Lines changed: 6 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
[![Build Status](https://ci.appveyor.com/api/projects/status/github/vshcherbyna/igel?svg=true)](https://ci.appveyor.com/project/vshcherbyna/igel)
66

7-
Igel is a free UCI chess engine forked from GreKo 2018.01. It is not a complete chess program and requires some UCI compatible GUI software in order to be used.
7+
Igel is a free UCI chess engine from Ukraine. It is not a complete chess program and requires some UCI compatible GUI software in order to be used.
88

99
### History
1010

@@ -18,38 +18,13 @@ In March 2019 Igel was invited to a prestigious chess tournament for top chess e
1818

1919
By mid 2020 Igel 2.5.0 64-bit 4CPU reached 3245 elo in CCRL Blitz on 4CPU and entered the top 30 engines of the list.
2020

21-
In June 2020 Igel was invited by Andrew Grant to participate in OpenBench testing framework and this has further accelerated the strength improvement of the engine. In August 2020 Igel switched to NNUE as a main evaluation function using Dietrich Kappe's NiNu network file and is currently approaching the top ten strongest chess engines on CCRL list.
21+
In June 2020 Igel was invited by Andrew Grant to participate in OpenBench testing framework and this has further accelerated the strength improvement of the engine.
2222

23-
In October 2020 first Igel Generation Networks were introduced seeing participation of Igel in TCEC Cup7 and TCEC Season 20 and released later with Igel 2.9.0.
23+
In August 2020 Igel experimened with NNUE using Dietrich Kappe's NiNu network file in it's initial releases.
2424

25-
In January 2023 last bits of HCE code were removed from Igel and the evaluation is fully based on NNUE as of Igel 3.4.0.
26-
27-
### Igel Generation Networks (IGN)
28-
29-
In late 2020 there have been a lot of discussions in the chess community about NNUE techology in general and NNUE networks in particular raising important questions about authenticity of networks.
30-
31-
I have decided to dedicate time and resources to create my own class of networks that will be used for future Igel releases: Igel Generation Network (IGN).
32-
33-
The IGN networks comply with the following mandatory requirements:
34-
35-
1. Source of network data: evaluation, search, pv line is generated solely using Igel chess engine. As a starting point Igel 2.6.0 is choosen as this was the last version of Igel featuring HCE (Hand Crafted Evaluation)
25+
As of late 2020 Igel adoped NNUE and uses own NNUE implementation and own network file trained on Igel data.
3626

37-
2. Both data and validation data generation must be following the rule #1
38-
39-
3. Two versions of Igel: 2.7.0 and 2.8.0 are excluded from network training/data generation process (both for data and for validation data) because they use Dietrich Kappe's Night Nurse network as thus violate the rule #1
40-
41-
4. Data generation and network training must be done by myself on my own (or rented by me) hardware in order to make sure the rule #1 is not violated
42-
43-
5. Each generation of network must contain information (on github page) on training parameters for clarity of the source of data
44-
45-
**Technical details**
46-
47-
| Generation | Architecture | Source of data | Quantity | Data Type | Best network |
48-
| ------------- | ------------------------| ---------------------------------- | ----------------- | -------------- | -------------- |
49-
| ign-0 | halfkp_256x2-32-32 | Igel 2.6.0 | 2.3b d8, 500m d12 | HCE | ign-0-9b1937cc |
50-
| ign-1 | halfkp_256x2-32-32 | Igel 2.6.0, Igel 2.9.0, Igel 3.0.0 | 12b d8, 1b d12 | HCE, NNUE | ign-1-9a48854b |
51-
| ign-2 | halfkav2_512x2-16-32 | Igels as of 2.9.0 | d5+d8+d16 >30b | Mostly NNUE | ign-2-51ba2968 |
52-
| ign-3 | halfkav2_hm_1024x2-8-32 | Igels as of 2.9.0 | d5+d8+d16 >30b | Fully NNUE | ign-3-debc71a4 |
27+
In January 2023 last bits of HCE code were removed from Igel and the evaluation is fully based on NNUE as of Igel 3.4.0.
5328

5429
### Acknowledgements
5530

@@ -87,7 +62,7 @@ Using cmake/gcc:
8762
git clone https://github.com/vshcherbyna/igel.git ./igel
8863
cd igel
8964
git submodule update --init --recursive
90-
wget https://github.com/vshcherbyna/igel/releases/download/3.4.0/ign-3-debc71a4 -O ./network_file
65+
wget https://github.com/vshcherbyna/igel/releases/download/3.5.0/c049c117 -O ./network_file
9166
cmake -DEVALFILE=network_file -DUSE_PEXT=1 -DUSE_AVX2=1 -D_BTYPE=1 -DSYZYGY_SUPPORT=TRUE .
9267
make -j
9368
```

history.txt

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,13 @@ Igel - open source chess engine forked from GreKo 2018.01.
33
(c) 2018-2023 Volodymyr Shcherbyna <volodymyr@shcherbyna.com> (Igel author)
44
(c) 2002-2018 Vladimir Medvedev <vrm@bk.ru> (GreKo author)
55

6+
Igel 3.5.0
7+
-------------
8+
22-Jun-2023
9+
10+
- new network architecture
11+
- new network c049c117
12+
613
Igel 3.4.0
714
-------------
815
30-Jan-2023

src/gen.cpp

Lines changed: 5 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -25,19 +25,13 @@
2525
using namespace std::chrono_literals;
2626
std::vector<std::list<Move>> g_movesBook;
2727

28-
Generator::Generator(int depth, int threads, int fFash):
28+
Generator::Generator(int depth, int threads):
2929
m_maxDepth(depth),
30-
m_maxThreads(threads),
31-
m_maxfHash(fFash)
30+
m_maxThreads(threads)
3231
{
3332
std::cout << "Igel training data generator" << std::endl;
3433
std::cout << "Training depth:" << m_maxDepth << std::endl;
3534
std::cout << "Training threads:" << m_maxThreads << std::endl;
36-
std::cout << "Training fens hash:" << m_maxfHash << std::endl;
37-
38-
std::cout << "Allocating fen hash table (" << fFash << "GB) ..";
39-
FenHashTT::instance(fFash);
40-
std::cout << ". Done" << std::endl;
4135
}
4236

4337
void Generator::onGenerate()
@@ -136,25 +130,18 @@ void Generator::onGenerate()
136130
while (true) {
137131

138132
uint64_t before_processed = 0;
139-
uint64_t before_skipped = 0;
140133

141-
for (auto i = 0; i < m_maxThreads; ++i) {
134+
for (auto i = 0; i < m_maxThreads; ++i)
142135
before_processed += m_workers[i].m_counter;
143-
before_skipped += m_workers[i].m_skipped;
144-
}
145136

146137
std::this_thread::sleep_for(15s);
147138

148139
uint64_t after_processed = 0;
149-
uint64_t after_skipped = 0;
150140

151-
for (auto i = 0; i < m_maxThreads; ++i) {
141+
for (auto i = 0; i < m_maxThreads; ++i)
152142
after_processed += m_workers[i].m_counter;
153-
after_skipped += m_workers[i].m_skipped;
154-
}
155143

156-
std::cout << "[Processed " << after_processed << " FENs, " << ((after_processed - before_processed) / 15) << " per sec] [" ;
157-
std::cout << "Skipped " << after_skipped << " FENs, " << ((after_skipped - before_skipped) / 15) << " per sec]" << std::endl;
144+
std::cout << "[Processed " << after_processed << " FENs, " << ((after_processed - before_processed) / 15) << " per sec]" << std::endl;
158145

159146
//
160147
// every epoch create a new data file

src/gen.h

Lines changed: 30 additions & 130 deletions
Original file line numberDiff line numberDiff line change
@@ -36,129 +36,35 @@
3636

3737
#include <iostream>
3838
#include <fstream>
39+
#include <chrono>
3940

4041
#if defined(__linux__) && !defined(__ANDROID__)
4142
#include <sys/mman.h>
4243
#endif
4344

4445
extern std::vector<std::list<Move>> g_movesBook;
4546

46-
class FenHashTableEntry
47-
{
48-
public:
49-
typedef union {
50-
struct {
51-
U32 move : 24, age : 8, type : 2;
52-
I32 score : 22, depth : 8;
53-
};
54-
U64 raw;
55-
}FenHashEntry;
56-
static_assert(sizeof(FenHashEntry) == 8, "FenHashEntry must be 8 bytes");
57-
58-
FenHashTableEntry()
59-
{
60-
m_data.raw = 0;
61-
m_key = 0;
62-
}
63-
64-
void store(Move mv, EVAL score, I8 depth, U8 type, U64 hash0, U8 age)
65-
{
66-
assert(type >= 0 && type <= 2);
67-
assert(age >= 0 && age <= 255);
68-
assert(depth >= -128 && depth <= 127);
69-
assert(score >= -50000 && score <= 50000);
70-
assert(mv >= 0 && mv <= 16777216);
71-
72-
m_data.age = age;
73-
m_data.type = type;
74-
m_data.move = mv;
75-
m_data.depth = depth;
76-
m_data.score = score;
77-
78-
m_key = hash0 ^ m_data.raw;
79-
80-
// when debugging a multi cpu configuration these may give you a trouble:
81-
assert(age == m_data.age);
82-
assert(type == m_data.type);
83-
assert(mv == m_data.move);
84-
assert(depth == m_data.depth);
85-
assert(score == m_data.score);
86-
}
87-
public:
88-
FenHashEntry m_data;
89-
U64 m_key;
90-
};
91-
static_assert(sizeof(FenHashTableEntry) == 16, "TEntry must be 16 bytes");
92-
93-
class FenHashTT
94-
{
95-
public:
96-
FenHashTT(int hSize) {
97-
size_t fenHash = static_cast<size_t>(hSize) * 1024;
98-
m_hashSize = static_cast<size_t>(static_cast<size_t>((1024 * 1024) * fenHash) / sizeof(FenHashTableEntry));
99-
100-
#if defined(__linux__) && !defined(__ANDROID__)
101-
// on linux systems we align on 2MB boundaries and request Huge Pages
102-
// idea comes from Sami Kiminki as used in Ethereal
103-
m_hash = reinterpret_cast<FenHashTableEntry*>(aligned_alloc(2 * MB, sizeof(FenHashTableEntry) * m_hashSize));
104-
madvise(m_hash, sizeof(FenHashTableEntry) * m_hashSize, MADV_HUGEPAGE);
105-
#else
106-
// otherwise, we simply allocate as usual and make no requests
107-
m_hash = reinterpret_cast<FenHashTableEntry*>(malloc(sizeof(FenHashTableEntry) * m_hashSize));
108-
#endif
109-
assert(m_hash);
110-
memset(reinterpret_cast<void*>(m_hash), 0, sizeof(FenHashTableEntry) * m_hashSize);
111-
}
112-
static FenHashTT & instance(int hSize) {
113-
static FenHashTT instance(hSize);
114-
return instance;
115-
}
116-
117-
public:
118-
void record(Move mv, EVAL score, I8 depth, int ply, U8 type, U64 hash0) {
119-
assert(m_hash);
120-
assert(m_hashSize);
121-
122-
size_t index = hash0 % m_hashSize;
123-
FenHashTableEntry & cluster = m_hash[index];
124-
cluster.store(mv, score, depth, type, hash0, 0);
125-
}
126-
bool retrieve(U64 hash) {
127-
assert(m_hash);
128-
assert(m_hashSize);
129-
130-
size_t index = hash % m_hashSize;
131-
auto pCluster = m_hash + index;
132-
133-
if ((pCluster->m_key ^ pCluster->m_data.raw) == hash)
134-
return true;
135-
136-
return false;
137-
}
138-
private:
139-
mutable FenHashTableEntry* m_hash;
140-
mutable size_t m_hashSize;
141-
static constexpr uint64_t MB = 1ull << 20;
142-
};
143-
14447
class GenWorker
14548
{
14649
friend class Generator;
14750

14851
public:
149-
GenWorker() : m_exit(false), m_pFile(nullptr), m_pMutex(nullptr), m_counter(0), m_skipped(0), m_search(new Search), m_finished(true), m_maxDepth(1) {}
52+
GenWorker() : m_exit(false), m_pFile(nullptr), m_pMutex(nullptr), m_counter(0), m_search(new Search), m_finished(true), m_maxDepth(1) {}
15053
bool isTaskReady() { return !m_tasks.empty(); }
15154
void workerRoutine()
15255
{
15356
std::vector<std::string> p = { "go", "depth", std::to_string(m_maxDepth) };
15457
Time time;
15558
time.parseTime(p, true);
59+
RandSeed(std::chrono::high_resolution_clock::now().time_since_epoch().count());
15660

15761
while (!m_exit) {
15862

15963
new_game:
16064
m_search->m_position.SetInitial();
16165
m_search->clearStacks();
66+
m_search->clearHistory();
67+
m_search->clearKillers();
16268

16369
//
16470
// pick up a book move
@@ -186,7 +92,7 @@ class GenWorker
18692
std::string move;
18793
EVAL score;
18894
int ply;
189-
bool quiet;
95+
COLOR side;
19096
};
19197

19298
std::list<genEntry> entries;
@@ -203,14 +109,22 @@ class GenWorker
203109
m_pMutex->lock();
204110

205111
for (auto const & i : entries) {
206-
if (i.ply > 6 && i.quiet && std::abs(i.score) <= (CHECKMATE_SCORE + MAX_PLY)) {
112+
113+
if (i.ply < 400) {
207114
int res = 0;
208-
if (result == "1-0")
209-
res = 1;
210-
else if (result == "0-1")
211-
res = -1;
212-
else
213-
res = 0; // result is a draw
115+
116+
if (result == "1-0") {
117+
if (i.side == WHITE)
118+
res = 1;
119+
else
120+
res = -1;
121+
}
122+
else if (result == "0-1") {
123+
if (i.side == BLACK)
124+
res = 1;
125+
else
126+
res = -1;
127+
}
214128

215129
*m_pFile << "fen " << i.fen << std::endl;
216130
*m_pFile << "move " << i.move << std::endl;
@@ -247,7 +161,7 @@ class GenWorker
247161
auto ply = m_search->m_position.Ply();
248162
auto fen = m_search->m_position.FEN();
249163
auto move = MoveToStrLong(m_search->m_best);
250-
auto check = m_search->m_position.InCheck();
164+
auto side = m_search->m_position.Side();
251165

252166
//
253167
// make a move, it must be legal
@@ -258,30 +172,17 @@ class GenWorker
258172
abort();
259173
}
260174

261-
auto hash = m_search->m_position.Hash();
262-
263175
//
264-
// avoid duplicated fen entries
176+
// store the position
265177
//
266178

267-
if (FenHashTT::instance(0).retrieve(hash) == false) {
179+
auto & entry = entries.emplace_back();
268180

269-
//
270-
// store the position
271-
//
272-
273-
auto & entry = entries.emplace_back();
274-
275-
entry.fen = fen;
276-
entry.move = move;
277-
entry.score = m_search->m_score;
278-
entry.ply = ply;
279-
entry.quiet = !check && !m_search->m_position.InCheck() && !MoveEval::isTacticalMove(m_search->m_best);
280-
281-
FenHashTT::instance(0).record(m_search->m_best, m_search->m_score, m_search->m_depth, ply, 0, hash);
282-
}
283-
else
284-
m_skipped++;
181+
entry.fen = fen;
182+
entry.move = move;
183+
entry.score = m_search->m_score;
184+
entry.ply = ply;
185+
entry.side = side;
285186

286187
if (m_exit)
287188
return;
@@ -293,7 +194,6 @@ class GenWorker
293194
std::ofstream * m_pFile;
294195
std::mutex * m_pMutex;
295196
uint64_t m_counter;
296-
uint64_t m_skipped;
297197
std::unique_ptr<Search> m_search;
298198
bool m_finished;
299199
int m_maxDepth;
@@ -303,7 +203,7 @@ class GenWorker
303203
class Generator
304204
{
305205
public:
306-
Generator(int depth, int threads, int fFash);
206+
Generator(int depth, int threads);
307207

308208
public:
309209
void onGenerate();

src/makefile

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ SRC = *.cpp fathom/tbprobe.cpp
99

1010
GCCDEFINES = $(shell echo | $(CC) -m64 -march=native -E -dM -)
1111

12-
EVALFILE = weights/ign-3-debc71a4
12+
EVALFILE = weights/c049c117
1313
NNFLAGS = -DEVALFILE=\"$(EVALFILE)\"
1414

1515
LIBS = -std=c++17 -mpopcnt -pthread
@@ -22,10 +22,12 @@ ifneq ($(findstring __AVX2__, $(GCCDEFINES)),)
2222
DEFS += -DUSE_AVX2=1
2323
endif
2424

25-
ifneq ($(findstring __AVX512VNNI__, $(GCCDEFINES)),)
26-
LIBS += -mavx512vnni
27-
DEFS += -DUSE_AVX512=1 -DUSE_VNNI=1
28-
endif
25+
#
26+
#ifneq ($(findstring __AVX512VNNI__, $(GCCDEFINES)),)
27+
# LIBS += -mavx512vnni
28+
# DEFS += -DUSE_AVX512=1 -DUSE_VNNI=1
29+
#endif
30+
#
2931

3032
CFLAGS = $(WARN) $(LIBS) $(OPTIM) $(NNFLAGS)
3133

0 commit comments

Comments
 (0)