HDDS-14416. Handle delete ratis replica in DELETED/DELETING container scenario.

[ashishkumar50]

What changes were proposed in this pull request?

When container state is DELETED/DELETING in SCM, and if any DN reports replica which is CLOSED and non-empty. And if replica bcs-id is less or equal to container sequence id, in this case SCM should trigger delete replica to DN. This will reduce orphan containers in the cluster.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14416

How was this patch tested?

Unit tests

[rich7420]

@ashishkumar50 thanks for the patch!

[rich7420]

It seems the three test methods (testDeletedContainerWithStaleClosedReplicaRatis, testDeletedContainerWithLowerBcsidStaleReplicaRatis, and testDeletedECContainerWithStaleClosedReplicaShouldNotForceDelete) share duplicated code for creating ContainerReportHandler, building ContainerReportsProto, sending messages via reportHandler.onMessage(), and verifying DeleteContainerCommand. Could we try to extract a helper method like sendReportAndVerifyDeleteCommand() or something that handles the common setup and verification logic?

[priyeshkaratha]

Thanks @ashishkumar50 for working on this. Changes LGTM

[ashishkumar50]

It seems the three test methods (testDeletedContainerWithStaleClosedReplicaRatis, testDeletedContainerWithLowerBcsidStaleReplicaRatis, and testDeletedECContainerWithStaleClosedReplicaShouldNotForceDelete) share duplicated code for creating ContainerReportHandler, building ContainerReportsProto, sending messages via reportHandler.onMessage(), and verifying DeleteContainerCommand. Could we try to extract a helper method like sendReportAndVerifyDeleteCommand() or something that handles the common setup and verification logic?

Extracted common code.

[rich7420]

@ashishkr200 thanks for the update!

[ashishkumar50]

@errose28 @sumitagrawl, Can you please help to review this PR.
If container state is DELETED/DELETING in SCM and if CLOSED replica is reported we are doing deleteReplica if replica bcsid <= container bcsid.
As we had discussed offline for DELETED case it is safe to delete CLOSED replica.
I think even for DELETING state it is safe to delete replica, if replica bcsid <= container bcsid.
(RM moves container state from CLOSED to DELETING if all 3 replicas are empty and then move to DELETED).
If any replica which is non-empty and reported when container state is DELETING with replica bcsid <= container bcsid, in this case replica can be deleted and will not cause any data loss.
Please confirm if we can proceed with both DELETED and DELETING state or we can just consider DELETED state.

[sumitagrawl]

LGTM