Skip to content

Add OpenSSL 4.0 compatibility - handle ENGINE API removal#513

Draft
Copilot wants to merge 5 commits intomasterfrom
copilot/fix-daily-coverity-build
Draft

Add OpenSSL 4.0 compatibility - handle ENGINE API removal#513
Copilot wants to merge 5 commits intomasterfrom
copilot/fix-daily-coverity-build

Conversation

Copy link

Copilot AI commented Feb 2, 2026

OpenSSL 4.0 removed the ENGINE API (deprecated since 3.0), causing undefined references during linking:

undefined reference to `ENGINE_get_first'
undefined reference to `ENGINE_by_id'

Changes

Test files (test_digest.c, test_ciphers.c)

  • Guard ENGINE API usage with #if !defined(OPENSSL_NO_ENGINE) && OPENSSL_VERSION_MAJOR < 4
  • Provide stub engine_is_available() returning 0 when ENGINE unavailable
  • Skip ENGINE-specific tests in warn_all_untested() when not supported

CMake (CMakeLists.txt)

  • Add HAVE_ENGINE compile check to detect ENGINE API availability
  • Conditionally build test_keyexpimp only when ENGINE available (test requires ENGINE, cannot run without it)
  • Handle missing enginesdir for OpenSSL 4.0 with fallback path

test_keyexpimp.c

  • Add #error directive when ENGINE unavailable (test fundamentally requires ENGINE)

Example

#if !defined(OPENSSL_NO_ENGINE) && OPENSSL_VERSION_MAJOR < 4
int engine_is_available(const char *name) {
    ENGINE *e = ENGINE_get_first();
    int found = 0;
    while (e != NULL) {
        if (strcmp(ENGINE_get_id(e), name) == 0) {
            found = 1;
            break;
        }
        e = ENGINE_get_next(e);
    }
    ENGINE_free(e);
    return found;
}
#else
int engine_is_available(const char *name) {
    (void)name;
    return 0;
}
#endif

Bug fixes

  • Fixed engine_is_available() in test_ciphers.c returning 0 unconditionally (logic error in existing code)
  • Fixed use-after-free: both functions were checking e != NULL after ENGINE_free(e)
Original prompt

This section details on the original issue you should resolve

<issue_title>failed daily coverity build</issue_title>
<issue_description>we need to investigate and fix the following issue

+ PREFIX=/home/runner/opt
+ PATH=/home/runner/opt/bin:/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
+ mkdir build
+ cd build
+ cmake -DTLS13_PATCHED_OPENSSL=0 -DOPENSSL_ROOT_DIR=/home/runner/opt -DOPENSSL_ENGINES_DIR=/home/runner/opt/engines ..
-- The C compiler identification is GNU 13.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found OpenSSL: /home/runner/opt/lib/libcrypto.so (found suitable version "4.0.0", minimum required is "3.4")
-- Setting build type to 'RelWithDebInfo' as none was specified.
-- Found OpenSSL application: /home/runner/opt/bin/openssl
-- Looking for clock_gettime
-- Looking for clock_gettime - found
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- LITTLE_ENDIAN
-- Performing Test ADDCARRY_U64
-- Performing Test ADDCARRY_U64 - Success
-- Performing Test RELAXED_ALIGNMENT
-- Performing Test RELAXED_ALIGNMENT - Success
CMake Warning (dev) at CMakeLists.txt:479 (install):
  Target lib_gost_engine has PUBLIC_HEADER files but no PUBLIC_HEADER
  DESTINATION.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done (1.2s)
-- Generating done (0.1s)
-- Build files have been written to: /home/runner/work/engine/engine/build
+ make
[  1%] Building C object CMakeFiles/test_digest.dir/test_digest.c.o
In file included from /home/runner/work/engine/engine/test_digest.c:33:
/home/runner/work/engine/engine/gost_lcl.h:34:5: warning: #warning "Gost-engine is built against not fully supported version of OpenSSL. EVP_CTRL_SET_TLSTREE_PARAMS definition in OpenSSL is expected." [-Wcpp]
   34 | #   warning "Gost-engine is built against not fully supported version of OpenSSL. \
      |     ^~~~~~~
[  2%] Linking C executable bin/test_digest
/usr/bin/ld: CMakeFiles/test_digest.dir/test_digest.c.o: in function `engine_is_available':
/home/runner/work/engine/engine/test_digest.c:949:(.text+0x62e): undefined reference to `ENGINE_get_first'
/usr/bin/ld: /home/runner/work/engine/engine/test_digest.c:954:(.text+0x644): undefined reference to `ENGINE_get_next'
/usr/bin/ld: /home/runner/work/engine/engine/test_digest.c:952:(.text+0x654): undefined reference to `ENGINE_get_id'
/usr/bin/ld: /home/runner/work/engine/engine/test_digest.c:956:(.text+0x678): undefined reference to `ENGINE_free'
/usr/bin/ld: CMakeFiles/test_digest.dir/test_digest.c.o: in function `warn_all_untested':
/home/runner/work/engine/engine/test_digest.c:980:(.text+0x6c7): undefined reference to `ENGINE_by_id'
/usr/bin/ld: /home/runner/work/engine/engine/test_digest.c:981:(.text+0x6db): undefined reference to `ENGINE_init'
/usr/bin/ld: /home/runner/work/engine/engine/test_digest.c:984:(.text+0x6eb): undefined reference to `ENGINE_get_digests'
/usr/bin/ld: /home/runner/work/engine/engine/test_digest.c:989:(.text+0x727): undefined reference to `ENGINE_get_digest'
/usr/bin/ld: /home/runner/work/engine/engine/test_digest.c:990:(.text+0x73e): undefined reference to `ENGINE_finish'
/usr/bin/ld: /home/runner/work/engine/engine/test_digest.c:991:(.text+0x746): undefined reference to `ENGINE_free'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/test_digest.dir/build.make:102: bin/test_digest] Error 1
make[1]: *** [CMakeFiles/Makefile2:172: CMakeFiles/test_digest.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
```</issue_description>

## Comments on the Issue (you are @copilot in this section)

<comments>
</comments>


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Fix failed daily Coverity build issue Add OpenSSL 4.0 compatibility - handle ENGINE API removal Feb 2, 2026
Copilot AI requested a review from chipitsine February 2, 2026 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

failed daily coverity build

2 participants