Single-cell clustering and batch correction with omicverse

Overview

This skill distills the single-cell tutorials [tcluster.ipynb](../../omicverseguide/docs/Tutorials-single/tcluster.ipynb) and [tsinglebatch.ipynb](../../omicverseguide/docs/Tutorials-single/tsinglebatch.ipynb). Use it when a user wants to preprocess an AnnData object, explore clustering alternatives (Leiden, Louvain, scICE, GMM, topic/cNMF models), and evaluate or harmonise batches with omicverse utilities.

Instructions

1. Import libraries and set plotting defaults

- Load omicverse as ov, scanpy as sc, and plotting helpers (scvelo as scv when using dentate gyrus demo data). - Apply ov.plotset() or ov.utils.ovplot_set() so figures adopt omicverse styling before embedding plots.

1. Load data and annotate batches

- For demo clustering, fetch scv.datasets.dentategyrus(); for integration, read provided .h5ad files via ov.read() and set adata.obs['batch'] identifiers for each cohort. - Confirm inputs are sparse numeric matrices; convert with adata.X = adata.X.astype(np.int64) when required for QC steps.

1. Run quality control

- Execute ov.pp.qc(adata, tresh={'mitoperc': 0.2, 'nUMIs': 500, 'detectedgenes': 250}, batchkey='batch') to drop low-quality cells and inspect summary statistics per batch. - Save intermediate filtered objects (adata.writeh5ad(...)) so users can resume from clean checkpoints.

1. Preprocess and select features

- Call ov.pp.preprocess(adata, mode='shiftlog|pearson', nHVGs=3000, batchkey=None) to normalise, log-transform, and flag highly variable genes; assign adata.raw = adata and subset to adata.var.highlyvariablefeatures for downstream modelling. - Scale expression (ov.pp.scale(adata)) and compute PCA scores with ov.pp.pca(adata, layer='scaled', npcs=50). Encourage reviewing variance explained via ov.utils.plotpcavarianceratio(adata).

1. Construct neighbourhood graph and baseline clustering

- Build neighbour graph using sc.pp.neighbors(adata, nneighbors=15, npcs=50, userep='scaled|original|Xpca') or ov.pp.neighbors(...). - Generate Leiden or Louvain labels through ov.utils.cluster(adata, method='leiden'|'louvain', resolution=1), ov.single.leiden(adata, resolution=1.0), or ov.pp.leiden(adata, resolution=1); remind users that resolution tunes granularity. - IMPORTANT - Dependency checks: Always verify prerequisites before clustering or plotting: ```python # Before clustering: check neighbors graph exists if 'neighbors' not in adata.uns: if 'Xpca' in adata.obsm: ov.pp.neighbors(adata, nneighbors=15, userep='Xpca') else: raise ValueError("PCA must be computed before neighbors graph")

# Before plotting by cluster: check clustering was performed if 'leiden' not in adata.obs: ov.single.leiden(adata, resolution=1.0) `` - Visualise embeddings with ov.pl.embedding(adata, basis='X_umap', color=['clusters','leiden'], frameon='small', wspace=0.5) and confirm cluster separation. Always check that columns in color= parameter exist in adata.obs` before plotting.

1. Explore advanced clustering strategies

- scICE consensus: instantiate model = ov.utils.cluster(adata, method='scICE', userep='scaled|original|Xpca', resolutionrange=(4,20), nboot=50, nsteps=11) and inspect stability via model.plotic(figsize=(6,4)) before selecting model.bestk groups. - Gaussian mixtures: run ov.utils.cluster(..., method='GMM', ncomponents=21, covariancetype='full', tol=1e-9, maxiter=1000) for model-based assignments. - Topic modelling: fit LDAobj = ov.utils.LDAtopic(...), review LDAobj.plottopiccontributions(6), derive cluster calls with LDAobj.predicted(k) and optionally refine using LDAobj.getresultsrfc(...). - cNMF programs: initialise cnmfobj = ov.single.cNMF(... components=np.arange(5,11), niter=20, numhighvargenes=2000, outputdir=...), factorise (factorize, combine), select K via kselectionplot, and propagate usage scores back with cnmfobj.getresults(...) and cnmfobj.getresults_rfc(...).

1. Evaluate clustering quality

- Compare predicted labels against known references with adjustedrandscore(adata.obs['clusters'], adata.obs['leiden']) and report metrics for each method (Leiden, Louvain, GMM, LDA variants, cNMF models) to justify chosen parameters.

1. Embed with multiple layouts

- Use ov.utils.mde(...) to create MDE projections from different latent spaces (adata.obsm["scaled|original|Xpca"], harmonised embeddings, topic compositions) and plot via ov.pl.embedding(..., color=['batch','celltype']) or ov.pl.embedding for consistent review of cluster/batch mixing.

1. Perform batch correction and integration

- Apply ov.single.batchcorrection(adata, batchkey='batch', methods='harmony'|'combat'|'scanorama'|'scVI'|'CellANOVA', npcs=50, ...) sequentially to generate harmonised embeddings stored in adata.obsm (Xharmony, Xcombat, Xscanorama, XscVI, Xcellanova). For scVI, mention latent size (nlatent=30) and genelikelihood="nb"; for CellANOVA define control pools via controldict. - After each correction, project to 2D with ov.utils.mde and visualise batch vs celltype to check mixing and conservation.

1. Benchmark integration performance

- Persist final object (adata.writeh5ad('neurips2021batchall.h5ad', compression='gzip')) and reload when necessary. - Use scibmetrics.benchmark.Benchmarker with embeddings list (["Xpca", "Xcombat", "Xharmony", "Xcellanova", "Xscanorama", "Xmiratopic", "Xmirafeature", "XscVI"]) to compute batch-vs-biology trade-offs via bm.benchmark() and summarise with bm.plotresultstable(minmaxscale=False).

1. General troubleshooting

- Ensure adata.raw captures the unscaled log-normalised matrix before subsetting to HVGs. - Confirm userep='scaled|original|Xpca' strings exist in adata.obsm prior to clustering; rerun preprocessing if missing. - Monitor memory when running cNMF or scVI; adjust n_iter, components, or latent dimensions for smaller datasets. - Pipeline dependency errors: When you encounter errors like "Could not find 'leiden' in adata.obs", always check and add prerequisites: - Before leiden/louvain clustering → ensure 'neighbors' in adata.uns - Before plotting by clustering → ensure the cluster column exists in adata.obs - Before UMAP/embedding → ensure PCA or another dimensionality reduction is complete - Code generation pattern: When generating multi-step code, use defensive checks rather than assuming prior steps completed successfully. This prevents cascading failures when users run steps out of order or in separate sessions.

Examples

• "Normalise dentate gyrus cells, compare Leiden, scICE, and GMM clusters, and report ARI scores versus provided clusters."
• "Batch-correct three NeurIPS datasets with Harmony and scVI, produce MDE embeddings coloured by batch and cell_type, and benchmark the embeddings."
• "Fit topic and cNMF models on a preprocessed AnnData object, retrieve classifier-refined cluster calls, and visualise the resulting programs on UMAP."

References

• Clustering walkthrough: [tcluster.ipynb](../../omicverseguide/docs/Tutorials-single/t_cluster.ipynb)
• Batch integration walkthrough: [tsinglebatch.ipynb](../../omicverseguide/docs/Tutorials-single/tsingle_batch.ipynb)
• Quick copy/paste commands: [reference.md](reference.md)