LoomExperiment round-trip error after logNormCounts

Generated: 2026-07-17 11:15:21

Context

This notebook illustrates a round-tripping bug related to LoomExperiment and scuttle packages.

It aims to facilitate a discussion on responsibility to determine where the fix should be applied.

Libraries

library(LoomExperiment)
library(Matrix)
library(scuttle)

Simulate data

Simulate a sparse count matrix (100 features x 10 barcodes).

Note: The error does not occur for a regular matrix.

set.seed(1)
simulated_counts <- rsparsematrix(
    nrow = 100, ncol = 10, density = 1,
    rand.x = function(n) rpois(n, lambda = 5)
)

Create a SingleCellLoomExperiment from that count matrix.

Note: Reproducing the bug does not require populating other slots (e.g., rowData, colData).

sce <- SingleCellExperiment(
    assays = list(counts = simulated_counts)
)
scle <- as(sce, "SingleCellLoomExperiment")
rownames(scle) <- paste0("feature", seq(nrow(scle)))
scle

class: SingleCellLoomExperiment 
dim: 100 10 
metadata(0):
assays(1): counts
rownames(100): feature1 feature2 ... feature99 feature100
rowData names(0):
colnames: NULL
colData names(0):
reducedDimNames(0):
mainExpName: NULL
altExpNames(0):
rowGraphs(0): NULL
colGraphs(0): NULL

Saving and reloading the original object works

if (!dir.exists("outputs/")) {
    dir.create("outputs")
}
if (!file.exists("outputs/scle.loom")) {
    export(scle, "outputs/scle.loom", format = "loom")
}

import("outputs/scle.loom", format = "loom")

class: LoomExperiment 
dim: 100 10 
metadata(4): CreatedWith LOOM_SPEC_VERSION LoomExperiment-class
  MatrixName
assays(1): counts
rownames(100): feature1 feature2 ... feature99 feature100
rowData names(0):
colnames: NULL
colData names(0):
rowGraphs(0): NULL
colGraphs(0): NULL

Apply scuttle::logNormCounts

scle_lognorm <- scuttle::logNormCounts(scle)
scle_lognorm

class: SingleCellLoomExperiment 
dim: 100 10 
metadata(0):
assays(2): counts logcounts
rownames(100): feature1 feature2 ... feature99 feature100
rowData names(0):
colnames: NULL
colData names(1): sizeFactor
reducedDimNames(0):
mainExpName: NULL
altExpNames(0):
rowGraphs(0): NULL
colGraphs(0): NULL

The new object can be saved but not reloaded

if (!file.exists("outputs/scle_lognorm.loom")) {
    export(scle_lognorm, "outputs/scle_lognorm.loom", format = "loom")
}

tryCatch(
    import("outputs/scle_lognorm.loom", format = "loom"),
    error = function(e) conditionMessage(e)
)

[1] "error in evaluating the argument 'x' in selecting a method for function 't': HDF5. Dataset. Can't open object."

Temporary workaround

The code chunk below illustrates the “set dimnames of individual assays to NULL” workaround that got me around the issue in the Clustering 3K PBMCs with Bioconductor notebook.

library(LoomExperiment)
library(scuttle)
sce <- import('outputs/scle.loom', format = "loom", type = "SingleCellLoomExperiment")
sce <- scuttle::logNormCounts(x = sce)
dimnames(sce@assays@data$counts) <- list(NULL, NULL)
dimnames(sce@assays@data$logcounts) <- list(NULL, NULL)
if (!file.exists("outputs/scle_lognorm_patched.loom")) {
    export(sce, "outputs/scle_lognorm_patched.loom", format = "loom")
}
import("outputs/scle_lognorm_patched.loom", format = "loom")

class: LoomExperiment 
dim: 100 10 
metadata(4): CreatedWith LOOM_SPEC_VERSION LoomExperiment-class
  MatrixName
assays(2): counts logcounts
rownames(100): feature1 feature2 ... feature99 feature100
rowData names(0):
colnames: NULL
colData names(1): sizeFactor
rowGraphs(0): NULL
colGraphs(0): NULL

Discussion

Quoting from Claude Code:

import() doesn’t eagerly load the assay matrix — it wraps the on-disk HDF5 dataset in a lazy DelayedArray. Loom files store matrices transposed (cells × genes on disk vs. genes × cells in R), so somewhere inside import() it calls t(...) on that delayed array to flip it back.

To dispatch the S4 method for t(), R has to evaluate the argument first — which forces an actual read from the HDF5 file. That read fails with HDF5. Dataset. Can't open object, HDF5’s error for “you asked me to open a dataset/group that doesn’t exist.” So the crash isn’t in your code logic — it’s import() trying to open something inside the file that isn’t there.

Every earlier loom file has symmetric (both NULL) dimnames on the assay matrix, so on import there’s nothing to open.

However, after scuttle::logNormCounts() runs, the resulting logcounts/counts assay matrices end up with dimnames set on only one side (length 32738 = genes), while the other side remains NULL.

When export() writes that asymmetric dimnames pair, it writes only the “2” entry to disk.

During re-import(), the reader apparently doesn’t handle a partial dimnames group gracefully—it still tries to open both “1” and “2”, and fails when “1” doesn’t exist.

Obviously, this needs to be manually confirmed and properly debugged by a human, but it gives a plausible explanation with a successful workaround.