-
Notifications
You must be signed in to change notification settings - Fork 2.7k
Simplify Pawn Scoring #2258
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplify Pawn Scoring #2258
Conversation
|
This patch probably doesn't lose any significant strength, but it is a good example of why I take issue with [-3;1] as universal simplification bounds no matter if the simplification is massive or minimal. Here * (r-2)/4 becomes -15 ; the gain in simplicity and maintainability is minimal. Losing 0.5 elo would already be way more than this simplicity is worth, but a -0.5 elo patch has over 25% odds of passing [-3;1]. Analysis done by taking together a bunch of simplification and testing their overall effect usually show that they are neutral and SF doesn't lose elo from them (while gaining in maintainability) ; but this doesn't mean each individual simplification is neutral ; rather the slightly elo-gaining one are usually compensating the losses from the slightly elo-losing ones. |
|
Without the nested ternary operators, this scope's complexity is revealed: |
|
@Alayan-stk-2 Would you feel better if -3,1 simplifications were tested twice? Presumably, this would reduce the chance of passing to 6% instead of 25%.
Also, a min # of games could also reduce the chance of introducing any regression.
|
|
Also, I place FAR more stock in the LTC test. If the LTC does not pass rather quickly (< ~70k games), I don't usually submit simplifications. |
25% is considering two passes already (the 50% odds point is at -0.56 to be exact). So a third [-3, 1] pass would mean 12% still. Having direct tests at [-2, 2] or whatever bounds best make sense for minor simplifications would make more sense imho, though it leaves the issue of appreciating what falls into "minor" simplifications.
I don't have a strong opinion on this.
Yes, I agree, the LTC result is what matters the most. From the STC & LTC, this patch looks ok. But with the error bars, there is uncertainty and after seeing the actual code change I seized up the opportunity to bring up my concerns which are not so much about this single patch than about the process. |
|
Another issue is proposing several simplifications with overlapping areas against the same master: which one are we suppose to prefer? |
|
Here's an idea: add simplifications to a separate branch like trivial fixes. Then merge when 4-5 pass stc/ltc in combination? |
|
Good idea! Another possibility is that the simplificators may want not to rush and wait a little bit before proposing conflicting simplifications. This would put less pressure on the maintainers :-) Here I merged the PR "simplify weak levers", which was a clear, non problematic simplification targeting the pawn evaluation code, so the two other pawn non-obvious "simplifications" (Remove connected array #2259, Simplify Pawn Scoring #2258) should be retested against it, imho. |
|
I will test all of the current ones in combination and report. |
|
Combo Test passed: |
|
If we make Score 64-bit and Values 32-bit, all of the type casting goes away. Also, we probably don't need to worry about overflow with * operator. Is this better? |
|
Making scores 64-bit would lose cache and hash efficiency, wouldn't it ? I think that's the whole point of the current way tapered eval is implemented with an int32 containing both mg and eg values. If going 64-bit, it could be an interesting experiment to try out 3-parts tapered eval with it... The linear scaling between mg and eg values is obviously a practical limitation in Stockfish's eval, there might be some hidden elo potential with a middle point. |
|
Yes. I've wondered about changing values and scores before, but a key
restriction is we store 16 bit values in the TT (hash). I think values
probably need to stay as 16 bit.
I wondered if we could shift values up 1 or 2 bits and still keep the 16
bit size. The range between 8k and 32k (roughly 40 to 160 pawns) doesn't
seem needed to me. Maybe even ~20 pawns is high enough for mate values?
That suggests we could get an extra 2 or 3 bits for more accurate evals,
more randomization, search info, or anything else we can think of in the
future.
|
|
Let's move this discussion to the forum. I will post a new thread. |
|
I've tried a few tests trying to force Values to 16-bit across the board.
I get the same bench, but some machines on the framework get different
benches. Any ideas on how to figure that out?
http://tests.stockfishchess.org/tests/view/5d3f21e90ebc5925cf0f788d
http://tests.stockfishchess.org/tests/view/5d3f4a970ebc5925cf0f7a8c
…On Mon, Jul 29, 2019 at 7:37 AM xoto10 ***@***.***> wrote:
Yes. I've wondered about changing values and scores before, but a key
restriction is we store 16 bit values in the TT (hash). I think values
probably need to stay as 16 bit.
I wondered if we could shift values up 1 or 2 bits and still keep the 16
bit size. The range between 8k and 32k (roughly 40 to 160 pawns) doesn't
seem needed to me. Maybe even ~20 pawns is high enough for mate values?
That suggests we could get an extra 2 or 3 bits for more accurate evals,
more randomization, search info, or anything else we can think of in the
future.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#2258?email_source=notifications&email_token=AHCWOSGOUUYXZ5CQBF6QYCLQB3XC5A5CNFSM4IHK57DKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3AXIEY#issuecomment-515994643>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AHCWOSEQ3PHHEG63A7FEW6LQB3XC5ANCNFSM4IHK57DA>
.
--
Michael T. Whiteley
mobile: 801.707.6886
|
|
Just curious if there was any resistance to this one? It's a pretty clean simplification with good performance. |
|
I would guess Stephane, as far as possible, is not merging patches while
TCEC is running. No big deal.
|
|
Related code has recently changed, so I'm closing this one. |
This is a functional simplification.
STC
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 113400 W: 25245 L: 25306 D: 62849
http://tests.stockfishchess.org/tests/view/5d3b866a0ebc5925cf0f3339
LTC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 33159 W: 5683 L: 5582 D: 21894
http://tests.stockfishchess.org/tests/view/5d3c727d0ebc5925cf0f4810
bench 3951957