fix data race in roundRobinSync #3862
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #3862 +/- ##
=======================================
Coverage 59.64% 59.64%
=======================================
Files 191 191
Lines 12306 12306
=======================================
Hits 7340 7340
Misses 3968 3968
Partials 998 998 |
prestonvanloon
approved these changes
Oct 28, 2019
Closed
prestonvanloon
pushed a commit
that referenced
this pull request
Nov 1, 2019
* rm'ed in protobuf * build proto * build proto * build proto * fix core package * Gazelle * Fixed all the tests * Fixed static test * Comment out spec test for now * One more skip * fix-roundRobinSync (#3862) * Starting but need new seed function * Revert initial sync * Updated Proposer Slashing * Fixed all tests * Lint * Update inclusion reward * Fill randao mixes with eth1 data hash * Test * Fixing test part1 * All tests passing * One last test * Updated config * Build proto * Proper skip message * Conflict and fmt * Removed crosslinks and shards. Built * Format and gazelle * Fixed all the block package tests * Fixed all the helper tests * All epoch package tests pass * All core package tests pass * Fixed operation tests * Started fixing rpc test * RPC tests passed! * Fixed all init sync tests * All tests pass * Fixed blockchain tests * Lint * Lint * Preston's feedback * Starting * Remove container * Fixed block spec tests * All passing except for block_processing test * Failing block processing test * Starting * Add AggregateAndProof * All mainnet test passes
prestonvanloon
pushed a commit
that referenced
this pull request
Nov 2, 2019
* rm'ed in protobuf * build proto * build proto * build proto * fix core package * Gazelle * Fixed all the tests * Fixed static test * Comment out spec test for now * One more skip * fix-roundRobinSync (#3862) * Starting but need new seed function * Revert initial sync * Updated Proposer Slashing * Fixed all tests * Lint * Update inclusion reward * Fill randao mixes with eth1 data hash * Test * Fixing test part1 * All tests passing * One last test * Updated config * Build proto * Proper skip message * Conflict and fmt * Removed crosslinks and shards. Built * Format and gazelle * Fixed all the block package tests * Fixed all the helper tests * All epoch package tests pass * All core package tests pass * Fixed operation tests * Started fixing rpc test * RPC tests passed! * Fixed all init sync tests * All tests pass * Fixed blockchain tests * Lint * Lint * Preston's feedback * Starting * Remove container * Fixed block spec tests * All passing except for block_processing test * Failing block processing test * Starting * Add AggregateAndProof * All mainnet test passes * Unskip block util tests
prestonvanloon
pushed a commit
that referenced
this pull request
Nov 11, 2019
* Update seed domains (#3872) * Remove Transfers (#3870) * Remove active index roots and compact committee roots (#3869) * Update inclusion reward (#3886) * Alter proposer selection logic (#3884) * Fix early committee bias (#3888) * Remove shards and committees (#3896) * Epoch spec tests v0.9 (#3907) * Block spec test v0.9 (#3905) * rm'ed in protobuf * build proto * build proto * build proto * fix core package * Gazelle * Fixed all the tests * Fixed static test * Comment out spec test for now * One more skip * fix-roundRobinSync (#3862) * Starting but need new seed function * Revert initial sync * Updated Proposer Slashing * Fixed all tests * Lint * Update inclusion reward * Fill randao mixes with eth1 data hash * Test * Fixing test part1 * All tests passing * One last test * Updated config * Build proto * Proper skip message * Conflict and fmt * Removed crosslinks and shards. Built * Format and gazelle * Fixed all the block package tests * Fixed all the helper tests * All epoch package tests pass * All core package tests pass * Fixed operation tests * Started fixing rpc test * RPC tests passed! * Fixed all init sync tests * All tests pass * Fixed blockchain tests * Lint * Lint * Preston's feedback * Starting * Remove container * Fixed block spec tests * All passing except for block_processing test * Failing block processing test * Starting * Add AggregateAndProof * All mainnet test passes * Update deposit contract (#3906) * Proto spec tests v0.9 (#3908) * Starting * Add AggregateAndProof * Unskip block util tests (#3910) * rm'ed in protobuf * build proto * build proto * build proto * fix core package * Gazelle * Fixed all the tests * Fixed static test * Comment out spec test for now * One more skip * fix-roundRobinSync (#3862) * Starting but need new seed function * Revert initial sync * Updated Proposer Slashing * Fixed all tests * Lint * Update inclusion reward * Fill randao mixes with eth1 data hash * Test * Fixing test part1 * All tests passing * One last test * Updated config * Build proto * Proper skip message * Conflict and fmt * Removed crosslinks and shards. Built * Format and gazelle * Fixed all the block package tests * Fixed all the helper tests * All epoch package tests pass * All core package tests pass * Fixed operation tests * Started fixing rpc test * RPC tests passed! * Fixed all init sync tests * All tests pass * Fixed blockchain tests * Lint * Lint * Preston's feedback * Starting * Remove container * Fixed block spec tests * All passing except for block_processing test * Failing block processing test * Starting * Add AggregateAndProof * All mainnet test passes * Unskip block util tests * Slot processing spec test V0.9 (#3912) * Starting * Add AggregateAndProof * Unskip slot processing mainnet test * Unskip minimal spec test for finalization (#3920) * Remove outdated interop tests (#3922) * Rm outdated interop tests * Rm test runner * Gazelle * Update validator to use proposer slot (#3919) * Fix committee assignment (#3931) * Replace shard with committee index (#3930) * Conflict * Clean up (#3933) * Remove shard filter in db (#3936) * Remove lightouse compatibility test (#3939) * Update Committee Cache for v0.9 (#3948) * Updated committee cache * Removed shuffled indices cache * Started testing run time * Lint * Fixed test * Safeguard against nil head state * address edge case * add test * Fixed TestRoundRobinSync by doubling the epochs * Unskip TestProtoCompatability (#3958) * Unskip TestProtoCompatability * Update WORKSPACE * Fix minimal config (#3959) * fix minimal configs * fix hardcoded value in test * Simplify verify att time (#3961) * update readme for deposit contract, regen bindings for vyper 0.1.0b12 (#3963) * update readme for deposit contract, regen bindings * medium * Check nil base state (#3964) * Copy Block When Receiving it From Sync (#3966) * copy block * clone for other service methods too * Change logging of Bitfield (#3956) * change logging of bits * preston's review * Unskip Beacon Server Test (#3962) * run test till the end * fix up proto message types * fmt * resolve broken tests * better error handling * fixing new logic to use archived proposer info * fix up logic * clip using the max effective balance * broken build fix with num arg mismatch * amend archive * archival logic changed * rename test * archive both proposer and attester seeds * page size 100 * further experiments * further experimentation, archivedProposerIndex seems wrong * test passes * rem log * fix broken test * fix test * gaz * fix imports * ethapis
rauljordan
added a commit
that referenced
this pull request
Nov 12, 2019
* Update seed domains (#3872) * Remove Transfers (#3870) * Remove active index roots and compact committee roots (#3869) * Update inclusion reward (#3886) * Alter proposer selection logic (#3884) * Fix early committee bias (#3888) * Remove shards and committees (#3896) * Epoch spec tests v0.9 (#3907) * Block spec test v0.9 (#3905) * rm'ed in protobuf * build proto * build proto * build proto * fix core package * Gazelle * Fixed all the tests * Fixed static test * Comment out spec test for now * One more skip * fix-roundRobinSync (#3862) * Starting but need new seed function * Revert initial sync * Updated Proposer Slashing * Fixed all tests * Lint * Update inclusion reward * Fill randao mixes with eth1 data hash * Test * Fixing test part1 * All tests passing * One last test * Updated config * Build proto * Proper skip message * Conflict and fmt * Removed crosslinks and shards. Built * Format and gazelle * Fixed all the block package tests * Fixed all the helper tests * All epoch package tests pass * All core package tests pass * Fixed operation tests * Started fixing rpc test * RPC tests passed! * Fixed all init sync tests * All tests pass * Fixed blockchain tests * Lint * Lint * Preston's feedback * Starting * Remove container * Fixed block spec tests * All passing except for block_processing test * Failing block processing test * Starting * Add AggregateAndProof * All mainnet test passes * Update deposit contract (#3906) * Proto spec tests v0.9 (#3908) * Starting * Add AggregateAndProof * Unskip block util tests (#3910) * rm'ed in protobuf * build proto * build proto * build proto * fix core package * Gazelle * Fixed all the tests * Fixed static test * Comment out spec test for now * One more skip * fix-roundRobinSync (#3862) * Starting but need new seed function * Revert initial sync * Updated Proposer Slashing * Fixed all tests * Lint * Update inclusion reward * Fill randao mixes with eth1 data hash * Test * Fixing test part1 * All tests passing * One last test * Updated config * Build proto * Proper skip message * Conflict and fmt * Removed crosslinks and shards. Built * Format and gazelle * Fixed all the block package tests * Fixed all the helper tests * All epoch package tests pass * All core package tests pass * Fixed operation tests * Started fixing rpc test * RPC tests passed! * Fixed all init sync tests * All tests pass * Fixed blockchain tests * Lint * Lint * Preston's feedback * Starting * Remove container * Fixed block spec tests * All passing except for block_processing test * Failing block processing test * Starting * Add AggregateAndProof * All mainnet test passes * Unskip block util tests * Slot processing spec test V0.9 (#3912) * Starting * Add AggregateAndProof * Unskip slot processing mainnet test * Unskip minimal spec test for finalization (#3920) * Remove outdated interop tests (#3922) * Rm outdated interop tests * Rm test runner * Gazelle * Update validator to use proposer slot (#3919) * Fix committee assignment (#3931) * Replace shard with committee index (#3930) * Conflict * Clean up (#3933) * Remove shard filter in db (#3936) * Remove lightouse compatibility test (#3939) * Update Committee Cache for v0.9 (#3948) * Updated committee cache * Removed shuffled indices cache * Started testing run time * Lint * Fixed test * Safeguard against nil head state * address edge case * add test * Fixed TestRoundRobinSync by doubling the epochs * Unskip TestProtoCompatability (#3958) * Unskip TestProtoCompatability * Update WORKSPACE * Fix minimal config (#3959) * fix minimal configs * fix hardcoded value in test * Simplify verify att time (#3961) * update readme for deposit contract, regen bindings for vyper 0.1.0b12 (#3963) * update readme for deposit contract, regen bindings * medium * Check nil base state (#3964) * Copy Block When Receiving it From Sync (#3966) * copy block * clone for other service methods too * Change logging of Bitfield (#3956) * change logging of bits * preston's review * Unskip Beacon Server Test (#3962) * run test till the end * fix up proto message types * fmt * resolve broken tests * better error handling * fixing new logic to use archived proposer info * fix up logic * clip using the max effective balance * broken build fix with num arg mismatch * amend archive * archival logic changed * rename test * archive both proposer and attester seeds * page size 100 * further experiments * further experimentation, archivedProposerIndex seems wrong * test passes * rem log * fix broken test * fix test * gaz * fix imports * ethapis * attester server split into subpackage * attester impl split up successfully * validator cleaned up * all packages isolated * include proposer * proper naming * test fix * proper viz * naming * resolved timeout due to config values * init use minimal * added all subfiles * subfile split and gazelle * shards * validator folder * cleanup val * shay feedback
rauljordan
added a commit
that referenced
this pull request
Nov 13, 2019
* Update seed domains (#3872) * Remove Transfers (#3870) * Remove active index roots and compact committee roots (#3869) * Update inclusion reward (#3886) * Alter proposer selection logic (#3884) * Fix early committee bias (#3888) * Remove shards and committees (#3896) * Epoch spec tests v0.9 (#3907) * Block spec test v0.9 (#3905) * rm'ed in protobuf * build proto * build proto * build proto * fix core package * Gazelle * Fixed all the tests * Fixed static test * Comment out spec test for now * One more skip * fix-roundRobinSync (#3862) * Starting but need new seed function * Revert initial sync * Updated Proposer Slashing * Fixed all tests * Lint * Update inclusion reward * Fill randao mixes with eth1 data hash * Test * Fixing test part1 * All tests passing * One last test * Updated config * Build proto * Proper skip message * Conflict and fmt * Removed crosslinks and shards. Built * Format and gazelle * Fixed all the block package tests * Fixed all the helper tests * All epoch package tests pass * All core package tests pass * Fixed operation tests * Started fixing rpc test * RPC tests passed! * Fixed all init sync tests * All tests pass * Fixed blockchain tests * Lint * Lint * Preston's feedback * Starting * Remove container * Fixed block spec tests * All passing except for block_processing test * Failing block processing test * Starting * Add AggregateAndProof * All mainnet test passes * Update deposit contract (#3906) * Proto spec tests v0.9 (#3908) * Starting * Add AggregateAndProof * Unskip block util tests (#3910) * rm'ed in protobuf * build proto * build proto * build proto * fix core package * Gazelle * Fixed all the tests * Fixed static test * Comment out spec test for now * One more skip * fix-roundRobinSync (#3862) * Starting but need new seed function * Revert initial sync * Updated Proposer Slashing * Fixed all tests * Lint * Update inclusion reward * Fill randao mixes with eth1 data hash * Test * Fixing test part1 * All tests passing * One last test * Updated config * Build proto * Proper skip message * Conflict and fmt * Removed crosslinks and shards. Built * Format and gazelle * Fixed all the block package tests * Fixed all the helper tests * All epoch package tests pass * All core package tests pass * Fixed operation tests * Started fixing rpc test * RPC tests passed! * Fixed all init sync tests * All tests pass * Fixed blockchain tests * Lint * Lint * Preston's feedback * Starting * Remove container * Fixed block spec tests * All passing except for block_processing test * Failing block processing test * Starting * Add AggregateAndProof * All mainnet test passes * Unskip block util tests * Slot processing spec test V0.9 (#3912) * Starting * Add AggregateAndProof * Unskip slot processing mainnet test * Unskip minimal spec test for finalization (#3920) * Remove outdated interop tests (#3922) * Rm outdated interop tests * Rm test runner * Gazelle * Update validator to use proposer slot (#3919) * Fix committee assignment (#3931) * Replace shard with committee index (#3930) * Conflict * Clean up (#3933) * Remove shard filter in db (#3936) * Remove lightouse compatibility test (#3939) * Update Committee Cache for v0.9 (#3948) * Updated committee cache * Removed shuffled indices cache * Started testing run time * Lint * Fixed test * Safeguard against nil head state * address edge case * add test * Fixed TestRoundRobinSync by doubling the epochs * Unskip TestProtoCompatability (#3958) * Unskip TestProtoCompatability * Update WORKSPACE * Fix minimal config (#3959) * fix minimal configs * fix hardcoded value in test * Simplify verify att time (#3961) * update readme for deposit contract, regen bindings for vyper 0.1.0b12 (#3963) * update readme for deposit contract, regen bindings * medium * Check nil base state (#3964) * Copy Block When Receiving it From Sync (#3966) * copy block * clone for other service methods too * Change logging of Bitfield (#3956) * change logging of bits * preston's review * Unskip Beacon Server Test (#3962) * run test till the end * fix up proto message types * fmt * resolve broken tests * better error handling * fixing new logic to use archived proposer info * fix up logic * clip using the max effective balance * broken build fix with num arg mismatch * amend archive * archival logic changed * rename test * archive both proposer and attester seeds * page size 100 * further experiments * further experimentation, archivedProposerIndex seems wrong * test passes * rem log * fix broken test * fix test * gaz * fix imports * ethapis * setup request/response types for the committees * list beacon committees impl * beacon committees fetch from archive * full list beacon committees implementation * list beacon committees added more useful fields * actually paginate * attester server split into subpackage * attester impl split up successfully * validator cleaned up * all packages isolated * include proposer * proper naming * test fix * proper viz * naming * resolved timeout due to config values * init use minimal * added all subfiles * subfile split and gazelle * shards * validator folder * cleanup val * shay feedback * initial pagination tests passing * paginated tests pass * fix bug regarding total count * pagination tests pass * adding final test, archive * archive test works * regen protos for archival * resolve broken test * test pass * broken archive test * rem helpers * gaz * fix kv test * useful gRPC error code standards * format * resolved bad test * test resolution * ux improvements and bug fixes * complete * comments * Update beacon-chain/archiver/service.go * Update beacon-chain/rpc/beacon/committees.go * elim bad test * preston feedback
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixed the data racing and slightly changed the initial synchronization logic
Related to #3665 (might fix it)
Precisely makes initial synchronization more reliable
Major changes:
respBlocks = append (respBlocks, resp ...)