GH-48894: [Python][C++] Set minimum nodejs on conda env to 16 for Azurite to work and use base Azure::Core::RequestFailedException instead of final Azure::Storage::StorageException #48895

raulcd · 2026-01-19T11:37:23Z

Rationale for this change

nodejs 12 is currently being installed on conda. CI jobs are failing and or segfaulting due to azurite failing with old versions.

2026-01-13T18:32:39.6961900Z #15 [ 9/11] RUN /arrow/ci/scripts/install_azurite.sh
2026-01-13T18:32:39.9624124Z #15 0.417 Node.js version = v12.4.0
2026-01-13T18:32:42.2087322Z #15 2.663 npm WARN deprecated [email protected]: Rimraf versions prior to v4 are no longer supported
2026-01-13T18:32:42.3917601Z #15 2.846 npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
2026-01-13T18:32:51.4870197Z #15 11.94 npm WARN deprecated [email protected]: Glob versions prior to v9 are no longer supported
2026-01-13T18:32:51.7035681Z #15 12.01 npm WARN deprecated [email protected]: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
2026-01-13T18:33:02.1406491Z #15 22.59 /opt/conda/envs/arrow/bin/azurite -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/azurite.js
2026-01-13T18:33:02.3841290Z #15 22.60 /opt/conda/envs/arrow/bin/azurite-queue -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/queue/main.js
2026-01-13T18:33:02.3842792Z #15 22.60 /opt/conda/envs/arrow/bin/azurite-blob -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/blob/main.js
2026-01-13T18:33:02.3844216Z #15 22.60 /opt/conda/envs/arrow/bin/azurite-table -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/table/main.js
2026-01-13T18:33:02.3846002Z #15 22.66 npm WARN [email protected] requires a peer of applicationinsights-native-metrics@* but none is installed. You must install peer dependencies yourself.
2026-01-13T18:33:02.3847278Z #15 22.66 
2026-01-13T18:33:02.3847564Z #15 22.66 + [email protected]
2026-01-13T18:33:02.3848038Z #15 22.66 added 376 packages from 296 contributors in 20.644s
2026-01-13T18:33:02.3848830Z #15 22.69 /opt/conda/envs/arrow/bin/azurite
2026-01-13T18:33:02.8929329Z #15 23.35 /opt/conda/envs/arrow/lib/node_modules/azurite/node_modules/fs-extra/lib/util/async.js:14
2026-01-13T18:33:02.8930231Z #15 23.35         (err) => err ?? new Error('unknown error')
2026-01-13T18:33:02.8930740Z #15 23.35                       ^

The job on PyArrow was segfaulting due to an Exception being thrown but not catch. In general we were using Azure::Storage::StorageException but Azure::Core::Http::TransportException could also be thrown on some cases.
Bot are final but inherit from Azure::Core::RequestFailedException.

What changes are included in this PR?

Pin minimum nodejs version to 16 so the failure doesn't happen again.
Update catching Azure::Storage::StorageException to Azure::Core::RequestFailedException so Azure::Core::Http::TransportException is also catch.

Are these changes tested?

Yes on CI.

Are there any user-facing changes?

No

GitHub Issue: [Python] Conda jobs segfault on Azure tests due to old nodejs version being installed #48894

…urite to work

raulcd · 2026-01-19T11:37:53Z

@github-actions crossbow submit test-conda-*

github-actions · 2026-01-19T11:38:12Z

⚠️ GitHub issue #48894 has been automatically assigned in GitHub to PR creator.

github-actions · 2026-01-19T11:40:46Z

Revision: 3fabe1d

Submitted crossbow builds: ursacomputing/crossbow @ actions-cd25c6296c

Task	Status
test-conda-cpp
test-conda-cpp-valgrind
test-conda-python-3.10
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-1.3.4-numpy-1.21.2
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.12-pandas-latest-numpy-1.26
test-conda-python-3.12-pandas-latest-numpy-latest
test-conda-python-3.13
test-conda-python-3.13-pandas-nightly-numpy-nightly
test-conda-python-3.13-pandas-upstream_devel-numpy-nightly
test-conda-python-3.14
test-conda-python-emscripten

raulcd · 2026-01-19T13:21:13Z

Moving this back to draft to investigate the segfault before fixing the nodejs version.

…age::StorageException so we also catch Azure::Core::Http::TransportException

raulcd · 2026-01-19T17:20:03Z

@github-actions crossbow submit -g cpp

github-actions · 2026-01-19T17:22:33Z

Revision: 3ccfdb7

Submitted crossbow builds: ursacomputing/crossbow @ actions-d7c2a57735

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-12-cpp-amd64
test-debian-12-cpp-i386
test-debian-experimental-cpp-gcc-15
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

raulcd · 2026-01-20T08:43:22Z

@pitrou is this what you expected? About factoring out the exception-catching code we can follow on a different issue if you want, I am unsure what that would require.
I initially started this PR to fix our current CI failures only.

pitrou · 2026-01-20T09:11:26Z

@pitrou is this what you expected?

Yes, it is. Did you check that it averted the crashes by turning them into proper errors?

About factoring out the exception-catching code we can follow on a different issue if you want, I am unsure what that would require.

Looking at the code, I'm not sure there's much to do about it, actually.

raulcd · 2026-01-20T09:20:35Z

@github-actions crossbow submit test-conda-python-3.13

github-actions · 2026-01-20T09:22:46Z

Revision: b9b1ab6

Submitted crossbow builds: ursacomputing/crossbow @ actions-35c166620c

Task	Status
test-conda-python-3.13

This reverts commit b9b1ab6.

raulcd · 2026-01-20T09:32:27Z

@github-actions crossbow submit test-conda-python-3.13

github-actions · 2026-01-20T09:34:41Z

Revision: a86bb98

Submitted crossbow builds: ursacomputing/crossbow @ actions-6ceafde0b6

Task	Status
test-conda-python-3.13

raulcd · 2026-01-20T09:40:25Z

Did you check that it averted the crashes by turning them into proper errors?

Yes, I did locally but I've pushed a job without the nodejs bump so we can see it there. Pytest finishes running all tests, it doesn't crash anymore but tests fail as expected:
https://github.com/ursacomputing/crossbow/actions/runs/21166071337/job/60871016940#step:6:6596

opt/conda/envs/arrow/lib/python3.13/site-packages/pyarrow/tests/test_fs.py:329: in azurefs
    fs.create_dir(container)
pyarrow/_fs.pyx:638: in pyarrow._fs.FileSystem.create_dir
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   OSError: Failed to create a container: pyarrow-filesystem: http://127.0.0.1:35357/devstoreaccount1/pyarrow-filesystem Azure Error: [] Fail to get a new connection for: http://127.0.0.1:35357. Could not connect to server

apacheGH-48894: [Python] Set minimum nodejs on conda env to 16 for Az…

3fabe1d

…urite to work

github-actions bot added the awaiting committer review Awaiting committer review label Jan 19, 2026

raulcd marked this pull request as ready for review January 19, 2026 12:11

raulcd requested review from assignUser, jonkeane and kou as code owners January 19, 2026 12:11

raulcd mentioned this pull request Jan 19, 2026

Fix CI conda-forge/arrow-cpp-feedstock#1913

Open

raulcd marked this pull request as draft January 19, 2026 13:20

Catch base Azure::Core::RequestFailedException instead of Azure::Stor…

3ccfdb7

…age::StorageException so we also catch Azure::Core::Http::TransportException

github-actions bot added the Component: C++ label Jan 19, 2026

raulcd changed the title ~~GH-48894: [Python] Set minimum nodejs on conda env to 16 for Azurite to work~~ GH-48894: [Python][C++] Set minimum nodejs on conda env to 16 for Azurite to work and use base Azure::Core::RequestFailedException instead of final Azure::Storage::StorageException Jan 20, 2026

raulcd marked this pull request as ready for review January 20, 2026 08:40

raulcd requested a review from pitrou January 20, 2026 08:43

Remove pin for nodejs to see new Error (not crash) on CI

b9b1ab6

Revert "Remove pin for nodejs to see new Error (not crash) on CI"

a86bb98

This reverts commit b9b1ab6.

pitrou approved these changes Jan 20, 2026

View reviewed changes

GH-48894: [Python][C++] Set minimum nodejs on conda env to 16 for Azurite to work and use base Azure::Core::RequestFailedException instead of final Azure::Storage::StorageException #48895

Are you sure you want to change the base?

GH-48894: [Python][C++] Set minimum nodejs on conda env to 16 for Azurite to work and use base Azure::Core::RequestFailedException instead of final Azure::Storage::StorageException #48895

Conversation

raulcd commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

raulcd commented Jan 19, 2026

Uh oh!

github-actions bot commented Jan 19, 2026

Uh oh!

github-actions bot commented Jan 19, 2026

Uh oh!

raulcd commented Jan 19, 2026

Uh oh!

raulcd commented Jan 19, 2026

Uh oh!

github-actions bot commented Jan 19, 2026

Uh oh!

raulcd commented Jan 20, 2026

Uh oh!

pitrou commented Jan 20, 2026

Uh oh!

raulcd commented Jan 20, 2026

Uh oh!

github-actions bot commented Jan 20, 2026

Uh oh!

raulcd commented Jan 20, 2026

Uh oh!

github-actions bot commented Jan 20, 2026

Uh oh!

raulcd commented Jan 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

raulcd commented Jan 19, 2026 •

edited

Loading