Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add dbtext decompression benchmark #44

Merged
merged 1 commit into from
Oct 11, 2024

Conversation

NickCondron
Copy link
Contributor

  1. Is decompression throughput typically defined as bytes/s of input or output?
  2. Decompression's measured speed is fast, but the iterations are slow due to large drop. Criterion complains that it doesn't get 100 samples in 5s. I could increase the sample time or reduce the minimum samples. If there was a analogous decompress_into function the memory could be reused and this wouldn't be an issue.

@CLAassistant
Copy link

CLAassistant commented Oct 9, 2024

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

@a10y a10y left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NickCondron thanks for contributing! A few points to address, see my comments.

As I mentioned, the decompression throughput is canonically measured in bytes-out per unit time.

These are the bench results I get on my M2 Max laptop:

image

benches/compress.rs Outdated Show resolved Hide resolved
benches/compress.rs Outdated Show resolved Hide resolved
@a10y
Copy link
Contributor

a10y commented Oct 10, 2024

And here are the numbers from a 10 year old Intel laptop I have lying around, which echoes the 1-3GiB/sec that the FSST paper authors seemed to get

image

@NickCondron
Copy link
Contributor Author

I appreciate the detailed comments. I pushed your recommended changes. I never figured out how to get the only the decompression benchmarks to run, so I didn't run into that issue initially haha.

benches/compress.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@a10y a10y left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@a10y a10y enabled auto-merge (squash) October 11, 2024 21:31
@a10y a10y disabled auto-merge October 11, 2024 21:31
@a10y a10y changed the title Add dbtext decompression benchmark feat: add dbtext decompression benchmark Oct 11, 2024
@a10y a10y enabled auto-merge (squash) October 11, 2024 21:31
@a10y a10y merged commit fd27344 into spiraldb:develop Oct 11, 2024
3 checks passed
@github-actions github-actions bot mentioned this pull request Oct 7, 2024
gatesn pushed a commit that referenced this pull request Dec 30, 2024
## 🤖 New release
* `fsst-rs`: 0.4.3 -> 0.4.4 (✓ API compatible changes)

<details><summary><i><b>Changelog</b></i></summary><p>

<blockquote>

## [0.4.4](v0.4.3...v0.4.4) -
2024-12-30

### Added

- add dbtext decompression benchmark (#44)

### Other

- Decode into an uninitialized byte slice
([#63](#63))
- *(deps)* lock file maintenance (#62)
- *(deps)* lock file maintenance (#60)
- *(deps)* lock file maintenance (#59)
- *(deps)* update mozilla-actions/sccache-action action to v0.0.7 (#58)
- *(deps)* lock file maintenance (#57)
- *(deps)* lock file maintenance (#56)
- *(deps)* lock file maintenance (#55)
- *(deps)* lock file maintenance (#54)
- *(deps)* lock file maintenance (#53)
- *(deps)* lock file maintenance (#50)
- *(deps)* lock file maintenance (#49)
- *(deps)* lock file maintenance (#47)
- *(deps)* lock file maintenance (#46)
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants