Motivation
-
I've observed that when the Controller detects a down slave node, it does not push this status update to the rest of the cluster. Consequently, clients are unable to receive topology updates through the server and must rely on their own health checking and traffic shedding mechanisms. This is problematic for simpler clients that lack sophisticated detection logic.
-
Currently, Kvrocks lacks an interface to proactively drain traffic and put an instance into maintenance mode. This often results in lossy routine maintenance. I propose implementing a feature to manually change the instance state to achieve a graceful traffic switch-off before maintenance.
Solution
Baesd on apache/kvrocks#3385
Upon detecting a replica failure, the downtime information is propagated to all other healthy nodes via CLUSTERX SETNODES. This enables clients to refresh their topology and promptly exclude the failed node from the traffic pool.
Are you willing to submit a PR?
I'm willing to submit a PR!