HDDS-9940. Use MappedByteBuffer in ChunkManagerDummyImpl

[Russole]

What changes were proposed in this pull request?

• Updated ChunkManagerDummyImpl to avoid allocating heap buffers on reads (previously ByteBuffer.allocate(len)), by creating a small backing file (default 1MB) and reading data from a reusable MappedByteBuffer.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-9940

How was this patch tested?

All CI checks passed.
https://github.com/Russole/ozone/actions/runs/21101732901

[Gargi-jais11]

Thanks @Russole for the patch.
Left some review comments below.

[Gargi-jais11]

Here, the code uses ch.truncate(newSize) to expand the file when ch.size() < newSize, but truncate() only reduces file size - it cannot expand files.

According to Java API:> "If the given size is greater than or equal to the file's current size then the file is not modified."
When the file is smaller than newSize, truncate() does nothing, leaving the file unchanged. This causes silent failures when read requests exceed the current mapped buffer size.

1. Scenario 1: File is Smaller, Need Smaller Size (< 1MB)

Example:

• Current file: 500 KB
• Request: Read 256 KB
• newSize = max(1MB, 256KB) = 1MB (because of DEFAULT_MAP_SIZE)

Current: ch.size() = 500 KB
Need:    newSize = 1 MB
Check:   Is 500 KB < 1 MB? → YES 
Action:  ch.truncate(1 MB)
Result:  ❌ truncate() does NOTHING (can't expand)
         File stays 500 KB, but we need 1 MB!

Problem: File doesn't expand to 1 MB, mapping fails or maps wrong size.

2. Scenario 2: File is Larger, Need Smaller Size (< 1MB)

Example:

• Current file: 2 MB
• Request: Read 256 KB
• newSize = max(1MB, 256KB) = 1MB

Current: ch.size() = 2 MB
Need:    newSize = 1 MB
Check:   Is 2 MB < 1 MB? → NO ❌
Action:  Skip truncate (condition false)
Result:  File is already big enough (2 MB > 1 MB)
         But we're mapping only 1 MB, so it works!

Status: Works (but wastes space - file is 2 MB but we only use 1 MB)

We need to take care of these scenarios.

if (ch.size() < newSize) {
  // LOGIC:  Need to EXPAND file 
} else if (ch.size() > newSize) {
  // Need to SHRINK file
  ch.truncate(newSize);  // This works for shrinking!
}

[Gargi-jais11]

Add some test coverage in TestChunkManagerDummyImpl for below scenarios:

• Test when read size is exactly at current mapped size boundary
• Test when read size larger than current mapped size (triggers resize)
• Test Concurrent reads from multiple threads

[Russole]

Thanks @Gargi-jais11 for the detailed review. I’ll make the necessary updates based on the comments.

[jojochuang]

realistically ChunkManagerDummyImpl is for perf test only not production code. IMO doesn't require super rigorous test corner case coverage. (Though it's nothing if AI generates the test code)

[Gargi-jais11]

realistically ChunkManagerDummyImpl is for perf test only not production code. IMO doesn't require super rigorous test corner case coverage. (Though it's nothing if AI generates the test code)

Okay if that's the case then we can skip testing this. @Russole Just fix the above review comment.

[Russole]

Thanks @Gargi-jais11 for the clarification. I’ll focus on fixing the review comment mentioned above and update the patch accordingly.

[Russole]

Hi @szetszwo, just sharing this PR for visibility as it follows up on the related JIRA. Thanks!

[szetszwo]

@Russole , thanks for working on this!

We could

• create the tmp file in the begining,
• initialize the MappedByteBuffer and make it readonly,
• duplicate the buffer and set limit for each readChunk(..) call.

Please see the comments inlined and also https://issues.apache.org/jira/secure/attachment/13080734/9649_review.patch

[szetszwo]

Let's keeping using limitReadSize(..). It limits the len <= OZONE_SCM_CHUNK_MAX_SIZE.

Then, we may create the backing file using the size OZONE_SCM_CHUNK_MAX_SIZE.

[szetszwo]

... A later attempt to write bytes at such a position will cause the file to be grown to accommodate the new bytes; the values of any bytes between the previous end-of-file and the newly-written bytes are unspecified.

From the FileChannel javadoc above, we should write zeros for the zero-filled buffer..

[szetszwo]

+1 the change looks good.

[adoroszlai]

Thanks @Russole for the patch, @Gargi-jais11, @szetszwo for the review.