fix: allow for providers to safely shutdown#1744
fix: allow for providers to safely shutdown#1744nicklasl wants to merge 5 commits intoopen-feature:mainfrom
Conversation
bb09ece to
08e5fc7
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1744 +/- ##
============================================
+ Coverage 92.51% 93.64% +1.13%
- Complexity 642 648 +6
============================================
Files 58 58
Lines 1536 1559 +23
Branches 170 173 +3
============================================
+ Hits 1421 1460 +39
+ Misses 70 56 -14
+ Partials 45 43 -2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
7271a12 to
dc340c1
Compare
|
I added a commit to additionally prevent race conditions during shutdown:
|
|
chrfwow
left a comment
There was a problem hiding this comment.
The implementation looks nice, but the tests may have some race conditions
| setFeatureProvider(provider); | ||
|
|
||
| Thread shutdownThread = new Thread(() -> { | ||
| providerRepository.shutdown(); |
There was a problem hiding this comment.
How long does the Mocked Provider take to shutdown? Are we sure we interrupt during this time?
Maybe this is a case for a VMLens test
| Thread t1 = new Thread(() -> providerRepository.shutdown()); | ||
| Thread t2 = new Thread(() -> providerRepository.shutdown()); | ||
| Thread t3 = new Thread(() -> providerRepository.shutdown()); |
There was a problem hiding this comment.
We cannot be sure that any of these threads execute in parallel. This should be a VMLens test
There was a problem hiding this comment.
Thanks, I'll look into making them VMLens tests instead, just need to find some time :)
f612687 to
5fe4cbc
Compare
|
@chrfwow I've been working on the shutdown behavior fix for ProviderRepository a bit on the side and ran into a VMLens issue I could use help with. I wrote a CT test (ProviderRepositoryCT) to verify concurrent shutdown behavior, but VMLens crashes when ThreadPoolExecutor.shutdown() is called inside AllInterleavings: Claude tells me that VMLens instruments I've commented out the tests for now but I'm wondering if you've encountered this before? Any ideas for testing concurrent shutdown with VMLens, or should we rely on the regular unit tests? |
4c6709a to
eefdb14
Compare
|
|
||
| Stream.concat(Stream.of(this.defaultStateManger.get()), this.stateManagers.values().stream()) | ||
| .distinct() | ||
| .forEach(this::shutdownProvider); | ||
| this.stateManagers.clear(); | ||
| taskExecutor.shutdown(); |
There was a problem hiding this comment.
I wonder if we have to wrap this with synchronized (registerStateManagerLock) {, otherwise we might lose additions to this.stateManagers
src/test/java/dev/openfeature/sdk/vmlens/ProviderRepositoryCT.java
Outdated
Show resolved
Hide resolved
src/test/java/dev/openfeature/sdk/vmlens/ProviderRepositoryCT.java
Outdated
Show resolved
Hide resolved
|
@nicklasl I opened an issue for the NPE in VmLens: vmlens/vmlens#58 |
e3325f9 to
2ac107b
Compare
aeeb0fd to
cf255c9
Compare
Signed-off-by: Nicklas Lundin <nicklasl@spotify.com>
Signed-off-by: Nicklas Lundin <nicklasl@spotify.com>
VMLens crashes with NPE when ThreadPoolExecutor.shutdown() is called inside AllInterleavings block. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Nicklas Lundin <nicklasl@spotify.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Nicklas Lundin <nicklasl@spotify.com>
Add tests for RejectedExecutionException handling, exception in provider shutdown fallback, concurrent shutdown calls, and interruption during shutdown. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Nicklas Lundin <nicklasl@spotify.com>
cf255c9 to
efb3475
Compare
|



Summary
Improves the shutdown behaviour of the
ProviderRepositoryto ensure safe termination of the task executor. The previous implementation calledshutdown()on the executor but did not wait for termination, potentially leading to abrupt shutdown, aborted provider shutdown sequences or hanging threads.Changes
ProviderRepository.shutdown()with a 3-second timeoutRejectedExecutionExceptionwhen tasks are submitted to a shut down executorImplementation Details
The shutdown process now:
shutdown()on the task executor to prevent new tasksawaitTermination()shutdownNow()to force terminationInterruptedExceptionby callingshutdownNow()and restoring the interrupt flagAtomicBooleanflagRejectedExecutionExceptionby running provider shutdown synchronously when executor is closedTest Coverage
Added comprehensive test coverage for shutdown behavior:
[x] Successful shutdown when executor terminates within timeout
[x] Forced shutdown when executor does not terminate within timeout
[x] Graceful handling of interruption during shutdown
[x] Protection against indefinite hangs during shutdown
[x] Concurrent shutdown calls (idempotent behavior)
[x] Shutdown during provider initialization
[x] Provider replacement during shutdown
[x] Prevention of new provider registration after shutdown starts
Related Issues
#1745
This change aligns with the broader effort to improve shutdown behaviour across the SDK, similar to PR #873 which addresses the same issue on Eventing.