MDEV-37608: Increase default binlog_row_event_max_size to 64k

[KhaledSayed04]

Summary:
This PR updates the default value of binlog_row_event_max_size from 8,192 bytes (8k) to 65,536 bytes (64k).

Technical Rationale:
The legacy 8k default causes extreme fragmentation in the binary log when handling modern workloads with large row payloads. By increasing this limit to 64k, the engine can aggregate more rows into fewer Write_rows events.

My benchmarks show that for a transaction involving 1,000 rows (5KB each), the 64k default reduces the number of binary log events from 1,000 to 77—a 92.3% reduction in header and metadata overhead. While 128k offers slightly better aggregation (39 events), 64k represents the optimal balance for memory allocation and network MTU efficiency.

---

Benchmark Results

Test Case: 1,000 rows inserted in a single transaction (5KB payload per row).

Buffer Size (Default)	Write_rows Events	Duration	Efficiency Gain
8192 (Legacy)	1,000	0.297s	-
32768	167	0.294s	~83.3% Fewer Headers
65536 (Proposed)	77	0.287s	~92.3% Fewer Headers
131072	39	0.284s	~96.1% Fewer Headers

---

Test Environment

• CPU: AMD Ryzen 5 PRO 4650G (6 Cores / 12 Threads)
• RAM: 16GB (15Gi Available)
• Storage: Kingston SNVS500G NVMe SSD (ROTA: 0)
• OS: Linux Mint 22.1 (Kernel 6.8.0-54-generic)

---

Verification

• Functional: Passed mtr binlog.binlog_row_binlog.
• Reproducibility: The results can be reproduced using the attached multi-row insert script.

Click to view Benchmark Script

#!/bin/bash

# Configuration
SIZES=(8192 32768 65536 131072)
SOCKET="/tmp/mariadb_bench.sock"
DATADIR="$(pwd)/sql/data"
BINLOG_DIR="$DATADIR"

# Prepare the Payload (1000 rows, 5KB each ~ 5MB total)
echo "Preparing the heavy multi-row payload..."
VALUES_PART=$(for i in {1..999}; do echo -n "(REPEAT('M', 5000)), "; done)
VALUES_PART+="(REPEAT('M', 5000))"
SQL_QUERY="CREATE DATABASE IF NOT EXISTS bench; USE bench; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (id INT PRIMARY KEY AUTO_INCREMENT, d LONGTEXT) ENGINE=InnoDB; INSERT INTO t1 (d) VALUES $VALUES_PART;"

pkill -9 mariadbd
rm -rf "$DATADIR" && mkdir -p "$DATADIR"

echo -e "\n| Size | Events | Duration |"
echo "| :--- | :--- | :--- |"

for VAL in "${SIZES[@]}"; do
    # Start server with specific size
    ./sql/mariadbd --datadir="$DATADIR" --console --skip-grant-tables \
    --binlog-format=ROW --log-bin=mysql-bin --port=3307 \
    --socket="$SOCKET" --binlog_row_event_max_size=$VAL > server.log 2>&1 &
    
    until ./client/mariadb --socket="$SOCKET" -e "SELECT 1" > /dev/null 2>&1; do
        sleep 1
    done

    # Run workload and capture time
    # We use 'bash -c' to ensure 'time' captures only the client execution
    TIME_RESULT=$( { time ./client/mariadb --socket="$SOCKET" -e "$SQL_QUERY" ; } 2>&1 )
    DURATION=$(echo "$TIME_RESULT" | grep "real" | awk '{print $2}')

    # Count Write_rows events
    LOG=$(./client/mariadb --socket="$SOCKET" -e "SHOW MASTER LOGS;" -sN | tail -1 | awk '{print $1}')
    COUNT=$(./client/mariadb-binlog "$BINLOG_DIR/$LOG" | grep "Write_rows" | wc -l)

    # Output result
    echo "| $VAL | $COUNT | $DURATION |"

    # Shutdown for next run
    pkill -9 mariadbd
    sleep 2
done

[CLAassistant]

All committers have signed the CLA.

[gkodinov]

Thank you for your contribution! This is a preliminary review.

Good job on the performance evaluation!

Several things to fix here:

• Please use a single commit
• Make sure the commit message complies to CODING_STANDARDS.md
• Re-record mysql--help to reflect the new value
• (optional) Consider adding a specialized test for this variable, similar to e.g. binlog_cache_size_basic.test

[KhaledSayed04]

Hi @gkodinov,

I've implemented the default change to 64k and added the basic system variable tests.
However, the CI (specifically the Embedded Debug builds) is showing regressions.

These failures occur because of some environment conditions that I couldn't catch. I have ran the failing tests locally and they passed.

Regarding expectations based on the legacy 8k event boundaries for some other tests, I have investigated two ways to resolve this:

• Scaling the Tests: Updating the .test and .inc files to use larger payloads (e.g., 64k+). This is thorough but results in large diffs and might be brittle.
• Pinning the Suites: Adding suite-level my.cnf or individual .opt files to keep these specific tests running at the legacy 8k limit. This preserves the original test logic while allowing the 64k default to take effect everywhere else.

Given that this task was marked as 'beginner-friendly,' the second option (pinning) seems more localized, but I'm happy to perform the full refactor if you prefer the first.

[gkodinov]

Yes, I think pinning should be OK.

[gkodinov]

The embedded failure occurs because of the following:

main.vector 'aria'                       w1 [ fail ]
        Test ended at 2026-02-27 11:26:29
CURRENT_TEST: main.vector
Warning: /home/buildbot/amd64-debian-12-debug-embedded/build/libmysqld/examples/mariadb-test-embedded: unknown option '--loose-disable-ssl-verify-server-cert'
Warning: /home/buildbot/amd64-debian-12-debug-embedded/build/libmysqld/examples/mariadb-test-embedded: unknown variable 'loose-ssl-ca=/home/buildbot/amd64-debian-12-debug-embedded/build/mysql-test/std_data/cacert.pem'
Warning: /home/buildbot/amd64-debian-12-debug-embedded/build/libmysqld/examples/mariadb-test-embedded: unknown variable 'loose-ssl-cert=/home/buildbot/amd64-debian-12-debug-embedded/build/mysql-test/std_data/client-cert.pem'
Warning: /home/buildbot/amd64-debian-12-debug-embedded/build/libmysqld/examples/mariadb-test-embedded: unknown variable 'loose-ssl-key=/home/buildbot/amd64-debian-12-debug-embedded/build/mysql-test/std_data/client-key.pem'
Warning: /home/buildbot/amd64-debian-12-debug-embedded/build/libmysqld/examples/mariadb-test-embedded: unknown option '--loose-skip-ssl'
Got ERROR: "/home/buildbot/amd64-debian-12-debug-embedded/build/libmysqld/examples/mariadb-test-embedded: unknown variable 'binlog-row-event-max-size=8192'" errno: 2000
mysqltest: Can't initialize MariaDB server

Note how the rest of the variables in the .opt file that are unknown are prefixed with --loose? Although they're not really dynamic. It's because of the embedded case where these are (apparently) not sourced.

[gkodinov]

We're very close. Let's just fix the failing buildbot tests. The rest is fine.

[gkodinov]

prefix with --loose here because it's run in embedded mode.

[KhaledSayed04]

@gkodinov
I've identified that the 64k change causes widespread regressions in legacy suites (Vector, Replication, and Windows Bootstrap) because they rely on specific 8k boundaries.

These tests don't fail all at once when running CI to group them all. They appear one by one.
I have a cleaner solution than adding dozens of .opt files: I can add loose-binlog-row-event-max-size=8192 to mysql-test/include/default_mysqld.cnf. This pins the entire test suite to the legacy limit while allowing the server to adopt 64k globally.

Is modifying default_mysqld.cnf the preferred way to handle such a large-scale test environment mismatch, or should I stick to individual .opt files for the failing tests?

[KhaledSayed04]

To build on my previous comment, I’ve performed a deeper static analysis to identify why some tests (like the Windows bootstrap) fail in CI despite passing in my local environment.

The 64k default creates a "physics shift" in event packing and memory overhead that the legacy test suite is highly sensitive to. I’ve identified the following candidate affected tests that likely require pinning to maintain stability across different architectures (Linux, Windows, etc.):

• Explicit Position-Tracking (47 tests): These tests use commands like SHOW BINLOG EVENTS or master_pos_wait, making them hard-coded to specific byte offsets.

grep -rlE "show binlog events|show master status|master_pos_wait" mysql-test/ | grep "\.test$" | wc -l

• Storage & Row-Ordering (27 tests): These rely on Aria physical packing behavior (e.g., main.vector_aria), which changes with larger event sizes.

grep -rlE "engine=aria" mysql-test/main/ | grep "\.test$" | wc -l

• Buffer & Protocol Constraints (61 tests): These tests manually lower max_allowed_packet or net_buffer_length to values (often to 8k or 16k) that clash with a 64k default. A 64k event default may cause "Packet too large" errors or protocol mismatches that these tests aren't configured to handle.

grep -rlE "max_allowed_packet|net_buffer_length|binlog_cache_size" mysql-test/ | grep "\.test$" | wc -l

• Memory Overhead (123 tests): These track memory-related variables or internal limits; the 64k per-thread cache default increases the base initialization footprint.

grep -rlE "memory" mysql-test/main/ | grep "\.test$" | wc -l

Given that nearly 200 tests are potentially affected by these implicit environment dependencies, I think the single-line addition to mysql-test/include/default_mysqld.cnf is the scalable way to adopt the 64k engine default without drowning the PR in .opt files.

I understand that individual .opt files make requirements explicit. However, given that these 200+ candidates are 'implicitly' dependent on the old default, Path A (global pin) provides a cleaner baseline.

A possible compromise: We could use default_mysqld.cnf for the general stability of the suite, and I can add specific .opt files only for the primary 'Replication' and 'Vector' tests that are most critical for documenting these boundaries. This avoids PR bloat while maintaining documentation where it matters most.

Looking forward to your guidance on which approach aligns best with the MariaDB testing philosophy.

[bnestere]

Hi @KhaledSayed04 !

It is good to update tests for new configurations, as this makes it obvious what problems users may run into when upgrading. Future tests will also more closely resemble our users behaviors. I'd suggest the following:

• Explicit Position-Tracking (47 tests): These tests use commands like SHOW BINLOG EVENTS or master_pos_wait, making them hard-coded to specific byte offsets.
• Storage & Row-Ordering (27 tests): These rely on Aria physical packing behavior (e.g., main.vector_aria), which changes with larger event sizes.
  ...
• Memory Overhead (123 tests): These track memory-related variables or internal limits; the 64k per-thread cache default increases the base initialization footprint.

The above three cases don't seem to rely on binlog_row_event_max_size for any actual behavior, it is just reporting numbers have changed. I'd suggest re-recording the tests to account for the new offset.

• Buffer & Protocol Constraints (61 tests): These tests manually lower max_allowed_packet or net_buffer_length to values (often to 8k or 16k) that clash with a 64k default. A 64k event default may cause "Packet too large" errors or protocol mismatches that these tests aren't configured to handle.

These tests seem to rely on behavior from binlog_row_event_max_size. I think changing/adding a .opt/.cnf file is unfortunately the only way to go about it for these tests.

Otherwise it looks good! If you can fix the tests by March 10, we can get this patch into the upcoming 13.0 preview release.