Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blockstore: Make WriteBatch operations go through LedgerColumn #3687

Merged
merged 10 commits into from
Nov 19, 2024

Conversation

steviez
Copy link

@steviez steviez commented Nov 18, 2024

Problem

  • rocksdb keys are currently allocated as Vec's; these allocations occur very frequently
  • blockstore: Move rocksdb keys from heap to stack #3603 looks to make rocksdb key allocation happen on the stack to avoid the frequent heap allocation
  • rocksdb keys can be stack allocated if we perform that logic within LedgerColumn since each column has a fixed-and-known-at-compile-time key length
  • So, we need to shift logic that creates the rocksdb byte array keys into LedgerColumn

Summary of Changes

Shift the logic of converting keys from the Index type to bytes from WriteBatch to LedgerColumn, and add methods to LedgerColumn to add puts and deletes into a WriteBatch

This is similar to #3650, with the goal of making #3603 (which will contain the more interesting changes) a smaller diff

bw-solana
bw-solana previously approved these changes Nov 19, 2024
Copy link

@bw-solana bw-solana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Left 1 nit.

Curious if you were able to observe any speedup in profiling or anything after these changes

ledger/src/blockstore.rs Outdated Show resolved Hide resolved
ledger/src/blockstore.rs Show resolved Hide resolved
@steviez
Copy link
Author

steviez commented Nov 19, 2024

Curious if you were able to observe any speedup in profiling or anything after these changes

I wouldn't expect any speed up from this one; this PR is still mostly just shuffling things around as prep for #3603. 3603 is the one that actually swaps from heap to stack.

@steviez steviez merged commit 9d588f5 into anza-xyz:master Nov 19, 2024
41 checks passed
@steviez steviez deleted the bstore_shift_wbatch branch November 19, 2024 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants