feat: add shuffle benchmark variants with native write config support

[andygrove]

Rationale

Run more benchmarks/variants.

What Changed?

• Add get_spark_configs() method to base Benchmark class for benchmark-specific Spark configurations
• Common Comet configs (enabled, logging) now defined in Python for jvm/native modes
• Add shuffle benchmark variants with and without native parquet writes:
  • shuffle-hash-native-write: hash shuffle with Comet native parquet writes enabled
  • shuffle-hash-spark-write: hash shuffle with native writes disabled (uses Spark writer)
  • shuffle-roundrobin-native-write: round-robin shuffle with native writes enabled
  • shuffle-roundrobin-spark-write: round-robin shuffle with native writes disabled
• Add --print-configs CLI option to output benchmark-specific configs
• Refactor run_all_benchmarks.sh to use helper function and remove duplicated configs
• Exclude benchmarks/pyspark/** from CI test workflows to avoid triggering tests for benchmark-only changes

Test plan

• Run python run_benchmark.py --list-benchmarks to verify new benchmarks are registered
• Run python run_benchmark.py --print-configs --benchmark shuffle-hash-native-write --mode native to verify config output
• Run ./run_all_benchmarks.sh to verify benchmarks execute correctly

🤖 Generated with Claude Code

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.02%. Comparing base (f09f8af) to head (fec2c9c).
⚠️ Report is 859 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #3226      +/-   ##
============================================
+ Coverage     56.12%   60.02%   +3.89%     
- Complexity      976     1429     +453     
============================================
  Files           119      170      +51     
  Lines         11743    15746    +4003     
  Branches       2251     2602     +351     
============================================
+ Hits           6591     9451    +2860     
- Misses         4012     4976     +964     
- Partials       1140     1319     +179     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.