[Real-time Table Replication X clusters][2/n] Create a Controller Job for historical segments backfill

[wirybeaver]

• Create an async controller zk jobs to deeply backfill historical segments prior to the inducted consumer watermarks. The deep copy means replicating segments to a standalone remote storage path.
• Introduced TableReplicator to fetch segment ZK metadata from the source, register a controller job, and submit 4 worker tasks (can be fine tuned in the future) that call a SegmentCopier for each segment; errors immediately trigger job updates and completed segments are batched for ZK updates.
• Added ZkBasedTableReplicationObserver and TableReplicationProgressStats to keep in-memory progress (remaining count and failed segment list) and persist job metadata to ZK (including a SEGMENTS_TO_BE_COPIED field) every 100 completions or immediately on error.
• Implemented SegmentCopier interface with RealtimeSegmentCopier (deep-copy mode) that copies segments between the same PinotFS scheme and uploads the segment URI to the destination controller; if a segment lacks a deep-store download URL the code logs the problem and records failures.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 65.61086% with 76 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.21%. Comparing base (9de73ec) to head (5167f4f).
⚠️ Report is 12 commits behind head on master.

Files with missing lines	Patch %	Lines
.../helix/core/replication/RealtimeSegmentCopier.java	57.57%	23 Missing and 5 partials ⚠️
...roller/helix/core/replication/TableReplicator.java	60.41%	17 Missing and 2 partials ⚠️
...ntroller/helix/core/PinotHelixResourceManager.java	42.30%	13 Missing and 2 partials ⚠️
...oller/api/resources/PinotTableRestletResource.java	0.00%	9 Missing ⚠️
...e/replication/ZkBasedTableReplicationObserver.java	90.62%	2 Missing and 1 partial ⚠️
...ller/helix/core/replication/NoOpSegmentCopier.java	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #17521      +/-   ##
============================================
- Coverage     63.24%   63.21%   -0.03%     
  Complexity     1454     1454              
============================================
  Files          3176     3182       +6     
  Lines        191002   191219     +217     
  Branches      29204    29223      +19     
============================================
+ Hits         120791   120888      +97     
- Misses        60796    60913     +117     
- Partials       9415     9418       +3     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (ø)
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-11	63.17% <65.61%> (-0.03%)	⬇️
java-21	63.19% <65.61%> (+0.02%)	⬆️
temurin	63.21% <65.61%> (-0.03%)	⬇️
unittests	63.21% <65.61%> (-0.03%)	⬇️
unittests1	55.58% <0.00%> (+0.01%)	⬆️
unittests2	34.11% <65.61%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.