Skip to content

go bench receipts parquet vs pebble#2794

Open
jewei1997 wants to merge 2 commits intoparquet-receiptdbfrom
go-bench-receipts-unoptimized
Open

go bench receipts parquet vs pebble#2794
jewei1997 wants to merge 2 commits intoparquet-receiptdbfrom
go-bench-receipts-unoptimized

Conversation

@jewei1997
Copy link
Contributor

@jewei1997 jewei1997 commented Feb 3, 2026

Summary

  • Add benchmarks comparing pebble vs parquet receipt store write performance across realistic workloads (100 blocks with 10-100 receipts per block)
  • Optimize parquet WAL by batching writes per block instead of per receipt, reducing WAL overhead by ~50%
  • Switch WAL encoding from JSON to binary for faster serialization
  • Use realistic ERC20 Transfer receipts in benchmarks (3 topics, 32-byte amount data, gas fields)

Test plan

  • Run benchmarks with go test -tags duckdb -bench='BenchmarkReceiptWriteAsync' -benchtime=3x -run='^$' ./sei-db/ledger_db/receipt
  • Verify parquet store tests pass with go test -tags duckdb ./sei-db/ledger_db/receipt

Benchmark Results

Backend 1k receipts 10k receipts
pebble-async 617k receipts/s 1,340k receipts/s
parquet-async 383k receipts/s 763k receipts/s
parquet-no-wal 706k receipts/s 1,044k receipts/s

The parquet-no-wal variant (bypasses WAL for benchmarking) shows parquet's raw write performance is competitive with pebble. WAL batching improved
parquet-async throughput by ~1.5-2x compared to per-receipt WAL writes.

@github-actions
Copy link

github-actions bot commented Feb 3, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedFeb 3, 2026, 8:58 PM

@codecov
Copy link

codecov bot commented Feb 3, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 47.08%. Comparing base (432bd51) to head (75dd167).

❗ There is a different number of reports uploaded between BASE (432bd51) and HEAD (75dd167). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (432bd51) HEAD (75dd167)
sei-tendermint 1 0
Additional details and impacted files

Impacted file tree graph

@@                  Coverage Diff                  @@
##           parquet-receiptdb    #2794      +/-   ##
=====================================================
- Coverage              56.65%   47.08%   -9.57%     
=====================================================
  Files                   2019     1951      -68     
  Lines                 165709   160030    -5679     
=====================================================
- Hits                   93880    75349   -18531     
- Misses                 63597    78152   +14555     
+ Partials                8232     6529    -1703     
Flag Coverage Δ
sei-chain 41.64% <ø> (+<0.01%) ⬆️
sei-cosmos 48.12% <ø> (-0.01%) ⬇️
sei-db 68.72% <ø> (ø)
sei-tendermint ?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 310 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant