Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🔍 Add Reverse Futility Pruning (RFP) #382

Merged
merged 44 commits into from
Sep 12, 2023
Merged

🔍 Add Reverse Futility Pruning (RFP) #382

merged 44 commits into from
Sep 12, 2023

Conversation

eduherminio
Copy link
Member

@eduherminio eduherminio commented Sep 6, 2023

Summary:

0bf9142: MaxDepth 6, Scaling factor 75

8+0.08 👍🏽 56.1 +/- 18.8 vs main (1414)

Score of Lynx 1431 - rfp 6 75 vs Lynx 1414 - main: 430 - 275 - 264  [0.580] 969
...      Lynx 1431 - rfp 6 75 playing White: 251 - 105 - 128  [0.651] 484
...      Lynx 1431 - rfp 6 75 playing Black: 179 - 170 - 136  [0.509] 485
...      White vs Black: 421 - 284 - 264  [0.571] 969
Elo difference: 56.1 +/- 18.8, LOS: 100.0 %, DrawRatio: 27.2 %
SPRT: llr 2.96 (100.5%), lbound -2.94, ubound 2.94 - H1 was accepted

Overview (8+0.08, 8moves_v3):

Rank Name                          Elo     +/-   Games    Wins  Losses   Draws   Points   Score    Draw   White   Black
   0 Lynx 1421 rfp fixed 75 8        4      12    2398     940     913     545   1212.5   50.6%   22.7%   52.5%   48.6%
   1 Lynx 1422 rfp fixed 100 8      10      12    2400     979     911     510   1234.0   51.4%   21.3%   54.4%   48.4%
   2 Lynx 1417 rfp fixed 100 6       6      12    2400     969     926     505   1221.5   50.9%   21.0%   50.6%   51.2%
   3 Lynx 1419 rfp fixed 100 4      10      12    2400     979     911     510   1234.0   51.4%   21.3%   52.7%   50.2%
   4 Lynx 1416 rfp fixed 150 4      19      12    2400     987     858     555   1264.5   52.7%   23.1%   53.5%   51.8%
   5 Lynx 1418 rfp fixed 200 4      -3      12    2402     935     956     511   1190.5   49.6%   21.3%   51.3%   47.8%
   6 MinimalChess 0.5.0             77      10    3602    1862    1073     667   2195.5   61.0%   18.5%   63.2%   58.7%
   7 Cadabra 2.0.0                  30      10    3599    1591    1282     726   1954.0   54.3%   20.2%   56.3%   52.3%
   8 Lynx 1414 - main              -51      10    3599    1008    1534    1057   1536.5   42.7%   29.4%   43.8%   41.6%
   9 BBC 1.1                       -87      10    3600    1014    1900     686   1357.0   37.7%   19.1%   38.1%   37.3%

dc77eca: MaxDepth 4, Scaling factor 150

8+0.08 👍🏽 41.0 +/- 15.9 vs main (1414)

Score of Lynx 1416 rfp fixed 150 4 vs Lynx 1414 - main: 575 - 416 - 362  [0.559] 1353
...      Lynx 1416 rfp fixed 150 4 playing White: 356 - 150 - 170  [0.652] 676
...      Lynx 1416 rfp fixed 150 4 playing Black: 219 - 266 - 192  [0.465] 677
...      White vs Black: 622 - 369 - 362  [0.593] 1353
Elo difference: 41.0 +/- 15.9, LOS: 100.0 %, DrawRatio: 26.8 %
SPRT: llr 2.95 (100.2%), lbound -2.94, ubound 2.94 - H1 was accepted

37a6f10: MaxDepth 8, Scaling factor 100
8+0.08 👍🏽 41.5 +/- 16.0 vs main (1414)

Score of Lynx 1438 - rfp 8 100 vs Lynx 1414 - main: 583 - 422 - 348  [0.559] 1353
...      Lynx 1438 - rfp 8 100 playing White: 321 - 182 - 174  [0.603] 677
...      Lynx 1438 - rfp 8 100 playing Black: 262 - 240 - 174  [0.516] 676
...      White vs Black: 561 - 444 - 348  [0.543] 1353
Elo difference: 41.5 +/- 16.0, LOS: 100.0 %, DrawRatio: 25.7 %
SPRT: llr 2.95 (100.1%), lbound -2.94, ubound 2.94 - H1 was accepted

6a0c6ec: MaxDepth 6, Scaling factor 100

8+0.08 👍🏽 46.1 +/- 16.9 vs main (1414)

Score of Lynx 1417 rfp fixed 100 6 vs Lynx 1414 - main: 489 - 336 - 335  [0.566] 1160
...      Lynx 1417 rfp fixed 100 6 playing White: 289 - 123 - 168  [0.643] 580
...      Lynx 1417 rfp fixed 100 6 playing Black: 200 - 213 - 167  [0.489] 580
...      White vs Black: 502 - 323 - 335  [0.577] 1160
Elo difference: 46.1 +/- 16.9, LOS: 100.0 %, DrawRatio: 28.9 %
SPRT: llr 2.95 (100.2%), lbound -2.94, ubound 2.94 - H1 was accepted

0bf9142: MaxDepth 6, Scaling factor 75

8+0.08 👍🏽 56.1 +/- 18.8 vs main (1414)

Score of Lynx 1431 - rfp 6 75 vs Lynx 1414 - main: 430 - 275 - 264  [0.580] 969
...      Lynx 1431 - rfp 6 75 playing White: 251 - 105 - 128  [0.651] 484
...      Lynx 1431 - rfp 6 75 playing Black: 179 - 170 - 136  [0.509] 485
...      White vs Black: 421 - 284 - 264  [0.571] 969
Elo difference: 56.1 +/- 18.8, LOS: 100.0 %, DrawRatio: 27.2 %
SPRT: llr 2.96 (100.5%), lbound -2.94, ubound 2.94 - H1 was accepted

Taking 4 150 as base (1427):

f1b5504: MaxDepth 2, Scaling factor 150
8+0.08 👎🏽vs 4 150 (1427)

Score of Lynx 1429 - rfp 2 150 vs Lynx 1427 - rfp base 4 150: 1235 - 1360 - 934  [0.482] 3529
...      Lynx 1429 - rfp 2 150 playing White: 755 - 531 - 478  [0.563] 1764
...      Lynx 1429 - rfp 2 150 playing Black: 480 - 829 - 456  [0.401] 1765
...      White vs Black: 1584 - 1011 - 934  [0.581] 3529
Elo difference: -12.3 +/- 9.8, LOS: 0.7 %, DrawRatio: 26.5 %
SPRT: llr -2.94 (-100.0%), lbound -2.94, ubound 2.94 - H0 was accepted

bc78727: MaxDepth 6, Scaling factor 100
8+0.08 👍🏽 12.0 +/- 7.9 vs 4 150 (1427)

Score of Lynx 1430 - rfp base 6 100 vs Lynx 1427 - rfp base 4 150: 2087 - 1898 - 1481  [0.517] 5466
...      Lynx 1430 - rfp base 6 100 playing White: 1277 - 731 - 725  [0.600] 2733
...      Lynx 1430 - rfp base 6 100 playing Black: 810 - 1167 - 756  [0.435] 2733
...      White vs Black: 2444 - 1541 - 1481  [0.583] 5466
Elo difference: 12.0 +/- 7.9, LOS: 99.9 %, DrawRatio: 27.1 %
SPRT: llr 2.96 (100.4%), lbound -2.94, ubound 2.94 - H1 was accepted

6ce48d9: MaxDepth 8, Scaling factor 150
8+0.08 vs 4 150 (1427)

Score of Lynx 1428 - rfp 8 150 vs Lynx 1427 - rfp base 4 150: 3037 - 3033 - 2340  [0.500] 8410
...      Lynx 1428 - rfp 8 150 playing White: 1883 - 1164 - 1158  [0.585] 4205
...      Lynx 1428 - rfp 8 150 playing Black: 1154 - 1869 - 1182  [0.415] 4205
...      White vs Black: 3752 - 2318 - 2340  [0.585] 8410
Elo difference: 0.2 +/- 6.3, LOS: 52.0 %, DrawRatio: 27.8 %
SPRT: llr -1.13 (-38.3%), lbound -2.94, ubound 2.94

Taking 6 100 as base (1430):

0bf9142: MaxDepth 6, Scaling factor 75
8+0.08 👍🏽 5.8 +/- 4.6 vs 6 100 (1430)

Score of Lynx 1431 - rfp 6 75 vs Lynx 1430 - rfp base 6 100: 5788 - 5527 - 4275  [0.508] 15590
...      Lynx 1431 - rfp 6 75 playing White: 3601 - 2130 - 2064  [0.594] 7795
...      Lynx 1431 - rfp 6 75 playing Black: 2187 - 3397 - 2211  [0.422] 7795
...      White vs Black: 6998 - 4317 - 4275  [0.586] 15590
Elo difference: 5.8 +/- 4.6, LOS: 99.3 %, DrawRatio: 27.4 %
SPRT: llr 2.95 (100.2%), lbound -2.94, ubound 2.94 - H1 was accepted

03dc994: MaxDepth 8, Scaling factor 100
8+0.08 3.8 +/- 4.4 (abandoned to test vs 6 75), looking like potential small gain vs 6 100 (1430)

Score of Lynx 1432 - rfp 8 100 vs Lynx 1430 - rfp base 6 100: 6567 - 6371 - 4822  [0.506] 17760
...      Lynx 1432 - rfp 8 100 playing White: 4053 - 2395 - 2432  [0.593] 8880
...      Lynx 1432 - rfp 8 100 playing Black: 2514 - 3976 - 2390  [0.418] 8880
...      White vs Black: 8029 - 4909 - 4822  [0.588] 17760
Elo difference: 3.8 +/- 4.4, LOS: 95.8 %, DrawRatio: 27.2 %
SPRT: llr 1.35 (45.8%), lbound -2.94, ubound 2.94

Taking 6 75 as base (1431/1440):

0c7bf16: MaxDepth 8, Scaling factor 75
8+0.08 👎🏽 -5.5 +/- 7.2, looking like potential small gain vs 6 100 (1431)

Score of Lynx 1439 - rfp 8 75 vs Lynx 1431 - rfp base 6 75: 2288 - 2390 - 1771  [0.492] 6449
...      Lynx 1439 - rfp 8 75 playing White: 1380 - 945 - 899  [0.567] 3224
...      Lynx 1439 - rfp 8 75 playing Black: 908 - 1445 - 872  [0.417] 3225
...      White vs Black: 2825 - 1853 - 1771  [0.575] 6449
Elo difference: -5.5 +/- 7.2, LOS: 6.8 %, DrawRatio: 27.5 %
SPRT: llr -2.94 (-100.0%), lbound -2.94, ubound 2.94 - H0 was accepted

6ea5bb9: MaxDepth 8, Scaling factor 100
8+0.08 👎🏽 -1.8 +/- 5.3 vs 6 75 (1431)

Score of Lynx 1438 - rfp 8 100 vs Lynx 1431 - rfp 6 75: 4391 - 4453 - 3328  [0.497] 12172
...      Lynx 1438 - rfp 8 100 playing White: 2712 - 1710 - 1665  [0.582] 6087
...      Lynx 1438 - rfp 8 100 playing Black: 1679 - 2743 - 1663  [0.413] 6085
...      White vs Black: 5455 - 3389 - 3328  [0.585] 12172
Elo difference: -1.8 +/- 5.3, LOS: 25.5 %, DrawRatio: 27.3 %
SPRT: llr -2.96 (-100.6%), lbound -2.94, ubound 2.94 - H0 was accepted

436e350: RFP before NMP (1442):
8+0.08 👎🏽 -4.8 +/- 6.9 vs 6 75 (1440)

Score of Lynx 1442 - rfp before nmp vs Lynx 1440 - rfp base 6 75: 2460 - 2557 - 1970  [0.493] 6987
...      Lynx 1442 - rfp before nmp playing White: 1516 - 961 - 1017  [0.579] 3494
...      Lynx 1442 - rfp before nmp playing Black: 944 - 1596 - 953  [0.407] 3493
...      White vs Black: 3112 - 1905 - 1970  [0.586] 6987
Elo difference: -4.8 +/- 6.9, LOS: 8.5 %, DrawRatio: 28.2 %
SPRT: llr -2.95 (-100.3%), lbound -2.94, ubound 2.94 - H0 was accepted

1616e1c: Avoid RFP re-evaluation in case of NMP evaluating back

8+0.08 👎🏽 vs 6 75 (1440)

Score of Lynx 1443 - rfp before nmp vs Lynx 1440 - rfp base 6 75: 3239 - 3324 - 2440  [0.495] 9003
...      Lynx 1443 - rfp before nmp playing White: 2047 - 1254 - 1201  [0.588] 4502
...      Lynx 1443 - rfp before nmp playing Black: 1192 - 2070 - 1239  [0.402] 4501
...      White vs Black: 4117 - 2446 - 2440  [0.593] 9003
Elo difference: -3.3 +/- 6.1, LOS: 14.7 %, DrawRatio: 27.1 %
SPRT: llr -2.96 (-100.4%), lbound -2.94, ubound 2.94 - H0 was accepted

609dce9: beta condition
8+0.08 👎🏽

Score of Lynx 1441 - rfp beta vs Lynx 1440 - rfp base 6 75: 2455 - 2556 - 1838  [0.493] 6849
...      Lynx 1441 - rfp beta playing White: 1537 - 954 - 934  [0.585] 3425
...      Lynx 1441 - rfp beta playing Black: 918 - 1602 - 904  [0.400] 3424
...      White vs Black: 3139 - 1872 - 1838  [0.592] 6849
Elo difference: -5.1 +/- 7.0, LOS: 7.7 %, DrawRatio: 26.8 %
SPRT: llr -2.96 (-100.4%), lbound -2.94, ubound 2.94 - H0 was accepted
Wrong impl, inside of move loop pre-psqt:

011dd9c: MaxDepth 4, Scaling factor 150

8+0.08 👎🏽 vs v0.16.0 (no psqt)

Score of Lynx 1378 - rfp vs Lynx 0.16.0: 4484 - 4545 - 3318  [0.498] 12347
...      Lynx 1378 - rfp playing White: 2725 - 1727 - 1722  [0.581] 6174
...      Lynx 1378 - rfp playing Black: 1759 - 2818 - 1596  [0.414] 6173
...      White vs Black: 5543 - 3486 - 3318  [0.583] 12347
Elo difference: -1.7 +/- 5.2, LOS: 26.0 %, DrawRatio: 26.9 %
SPRT: llr -2.95 (-100.2%), lbound -2.94, ubound 2.94 - H0 was accepted

post-psqt:

0a832b8: MaxDepth 2, Scaling factor 300

8+0.08 👎🏽 vs main (1399)

Score of Lynx 1402 rfp 300 2 vs Lynx 1399 - main: 3052 - 3139 - 2377  [0.495] 8568
...      Lynx 1402 rfp 300 2 playing White: 1894 - 1226 - 1164  [0.578] 4284
...      Lynx 1402 rfp 300 2 playing Black: 1158 - 1913 - 1213  [0.412] 4284
...      White vs Black: 3807 - 2384 - 2377  [0.583] 8568
Elo difference: -3.5 +/- 6.2, LOS: 13.4 %, DrawRatio: 27.7 %
SPRT: llr -2.96 (-100.6%), lbound -2.94, ubound 2.94 - H0 was accepted

b361e36: MaxDepth 4, Scaling factor 150

8+0.08 👎🏽 vs main (1399)

Score of Lynx 1403 - rfp 4 150 vs Lynx 1399 - main: 1630 - 1744 - 1309  [0.488] 4683
...      Lynx 1403 - rfp 4 150 playing White: 989 - 678 - 674  [0.566] 2341
...      Lynx 1403 - rfp 4 150 playing Black: 641 - 1066 - 635  [0.409] 2342
...      White vs Black: 2055 - 1319 - 1309  [0.579] 4683
Elo difference: -8.5 +/- 8.4, LOS: 2.5 %, DrawRatio: 28.0 %
SPRT: llr -2.95 (-100.2%), lbound -2.94, ubound 2.94 - H0 was accepted

382c41e: MaxDepth 6, Scaling factor 100

8+0.08 👎🏽 (aborted) vs main (1399)

Score of Lynx 1404 - rfp 6 100 vs Lynx 1399 - main: 2119 - 2171 - 1390  [0.495] 5680
...      Lynx 1404 - rfp 6 100 playing White: 1293 - 874 - 673  [0.574] 2840
...      Lynx 1404 - rfp 6 100 playing Black: 826 - 1297 - 717  [0.417] 2840
...      White vs Black: 2590 - 1700 - 1390  [0.578] 5680
Elo difference: -3.2 +/- 7.8, LOS: 21.4 %, DrawRatio: 24.5 %
SPRT: llr -1.77 (-60.1%), lbound -2.94, ubound 2.94

: MaxDepth 8, Scaling factor 75

8+0.08 👎🏽 vs main (1399)

Score of Lynx 1402 rfp 300 2 vs Lynx 1399 - main: 3017 - 3104 - 2358  [0.495] 8479
...      Lynx 1402 rfp 300 2 playing White: 1863 - 1176 - 1200  [0.581] 4239
...      Lynx 1402 rfp 300 2 playing Black: 1154 - 1928 - 1158  [0.409] 4240
...      White vs Black: 3791 - 2330 - 2358  [0.586] 8479
Elo difference: -3.6 +/- 6.3, LOS: 13.3 %, DrawRatio: 27.8 %
SPRT: llr -2.95 (-100.2%), lbound -2.94, ubound 2.94 - H0 was accepted

eduherminio and others added 30 commits September 6, 2023 19:36
Prefer the former, in case the condition is not true and we can avoid evaluating the second one
@eduherminio eduherminio marked this pull request as ready for review September 12, 2023 00:47
@eduherminio eduherminio merged commit a974088 into main Sep 12, 2023
@eduherminio eduherminio deleted the rfp branch September 12, 2023 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant