Extreme memory usage (>800G and rising) in Spl-IsoQuant v2.2.0 during StereoSeq nanopore barcode calling (428M+ barcodes)

[chenghuihui2018]

I am processing StereoSeq nanopore raw data with Spl-IsoQuant v2.2.0, and encountered severe memory exhaustion during the barcode calling step. The process completes barcode indexing but then consumes massive memory (already 800GB and still increasing), making it impossible to proceed with my current computing resources.

Key Details (Logs + Environment)
Critical runtime logs (process stuck after barcode indexing):

2025-12-16 06:00:48,194 - INFO - Running Spl-IsoQuant version 2.2.0
2025-12-16 06:00:48,196 - WARNING - Output folder already exists, some files may be overwritten.
2025-12-16 06:00:48,274 - INFO - Novel unspliced transcripts will not be reported, set --report_novel_unspliced true to discover them
2025-12-16 06:00:48,274 - INFO - === Spl-IsoQuant pipeline started ===
2025-12-16 06:00:48,274 - INFO - Python version: 3.11.14 | packaged by conda-forge | (main, Oct 22 2025, 22:46:25) [GCC 14.3.0]
2025-12-16 06:00:48,274 - INFO - gffutils version: 0.13
2025-12-16 06:00:48,274 - INFO - pysam version: 0.23.3
2025-12-16 06:00:48,274 - INFO - pyfaidx version: 0.9.0.3
2025-12-16 06:00:48,298 - INFO - Detecting barcodes
2025-12-16 06:00:48,326 - INFO - Processing /xxxx/sample1.fq.gz
2025-12-16 06:00:48,340 - INFO - Using barcodes from /xxxx/barcodelist
2025-12-16 10:50:24,536 - INFO - Indexed 428882153 barcodes
2025-12-16 10:50:33,129 - INFO - Barcode caller created

After the Barcode caller created log line, the process does not proceed further, and memory usage climbs continuously to 800GB+ (no OOM error yet, but my node’s memory is nearly exhausted).

Questions:
Are there Spl-IsoQuant-specific parameters to optimize memory usage for massive barcode sets (e.g., chunked barcode processing, lightweight indexing, or barcode filtering)?
Is there a way to preprocess the 428M barcode list to reduce memory load without losing valid barcodes for StereoSeq data?
What is the recommended memory configuration for running Spl-IsoQuant with a ~400M barcode library (StereoSeq full-chip data)?

[andrewprzh]

Dear @chenghuihui2018

Indexing 238M barcodes may take quite a lot of RAM. However, 800G looks like an overkill.
I ran Stereo-Seq full data but the peak RAM was ~120G for 8 threads.
Could you send me also the command line you used?

Best
Andrey

[chenghuihui2018]

Dear Andrey,
Thank you so much for your prompt reply and insights! I really appreciate your help with this issue.

1. Full command line I used

python splisoquant.py
--reference ./mouse/fasta/genome.fa
--genedb ./mouse/genes/genes.gtf
--fastq ./ONT_STO/sample1.fq.gz
--barcode_whitelist ./barcodelist
--data_type nanopore
--mode stereoseq
--output ./
--complete_genedb
-p sto

2. Background: Original error before code modification
   Before encountering the memory issue, I hit an OverflowError related to uint32 bounds when processing the 428M+ barcodes (full StereoSeq chip barcode list). The critical error log is as follows:

2025-12-15 12:16:51,826 - CRITICAL - Spl-IsoQuant failed with the following error...
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "lib/python3.11/concurrent/futures/process.py", line 261, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "spl-IsoQuant/detect_barcodes.py", line 320, in process_in_parallel
barcode_detector = BARCODE_CALLING_MODESargs.mode
File "spl-IsoQuant/src/barcode_calling/barcode_callers.py", line 696, in init
self.barcode_indexer = SharedMemoryArray2BitKmerIndexer(bit_barcodes, kmer_size=10, ...)
File "spl-IsoQuant/src/barcode_calling/shared_mem_index.py", line 78, in init
self.index_ranges[index_i] = i
~~~~~~~~~~~~~~~~~^^^^^^^^^
OverflowError: Python integer 4294972967 out of bounds for uint32
"""
OverflowError: Python integer 4294972967 out of bounds for uint32

This error occurred because the number of barcodes (~428M) exceeded the maximum value of uint32 (4294967295), so I had to modify the code to bypass this issue.

3. Details of my code modification (src/barcode_calling/shared_mem_index.py)
   To fix the uint32 overflow, I made the following changes:
   Dynamically determine self.KMER_DTYPE (for barcode index) and self.INDEX_DTYPE (for index_ranges offsets) after calculating index_size and total_sequences:
   KMER_DTYPE is set to fit the total number of barcodes (total_sequences).
   INDEX_DTYPE is set to fit the offset values of index_ranges (index_size).
   Added dtype names (kmer_dtype_name, index_dtype_name) to SharedMemoryIndexInfo to ensure consistent dtype interpretation of shared memory between main/child processes.
   Added backward compatibility to SharedMemoryIndexInfo.setstate (fall back to uint32 if the old state tuple length is 7).
   Added exception handling in the del method to avoid errors during destructor execution.

4. Current issue after modification
   After the above changes, the uint32 overflow error was resolved, and the pipeline proceeded to Barcode caller created (as shown in the logs I shared earlier). However:
   The process stops progressing after Barcode caller created.
   Critical environment detail: My computing node only has 256GB of RAM (not 800GB), but the system monitor shows memory usage climbing to 800GB+ (likely including swap space) — the process has been stuck in this state for over 10 hours with no progress.
   No OOM error is thrown yet, but the pipeline is completely unsustainable and unresponsive.

My key questions to you:
Could my code modifications (especially dynamic dtype changes for shared memory index) be the root cause of the extreme memory usage (and swap exhaustion) on my 256GB node?
Is there an official/recommended fix for the uint32 overflow issue with large barcode libraries (~400M+ barcodes) in Spl-IsoQuant v2.2.0?
Are there any parameters or optimizations I can apply to reduce memory consumption for such a large barcode set (beyond what I’ve tried)?
What is the expected runtime for each stage of Spl-IsoQuant (barcode indexing, barcode calling, etc.) when processing ~400M barcodes + StereoSeq nanopore data on a 256GB RAM node? Is 10+ hours of stuck progress abnormal?
Thank you again for your time and guidance! I’m happy to provide any additional logs, code snippets, system monitor outputs, or environment details if needed.

Best regards,
huihui