18 Single-cell Transcriptomics: Concepts

18.1 The problem

Bulk RNA-seq measures the average transcriptome of a population. That average is a real quantity and it is often the wrong one: a two-fold change in a bulk sample is equally consistent with every cell doubling a gene’s expression and with half the cells appearing that were not there before. Single-cell methods exist to tell those apart.

The price is that every measurement gets much smaller. A bulk library spends its reads on one sample; a single-cell library spends the same reads across thousands of cells, so each cell is characterized by a few thousand molecules out of the hundreds of thousands it contained. Everything difficult about this subject follows from that one fact.

A single-cell dataset is not a collection of small bulk samples. It is one sparse, extremely under-sampled observation per cell, and the analysis is mostly a set of strategies for borrowing information between cells to say anything about any of them. Which cells are allowed to lend to which is exactly what clustering decides, which is why that step is load-bearing in a way it does not look.

18.2 One barcode, one cell, one molecule

The measurement rests on two identifiers attached before amplification.

A cell barcode is a sequence shared by every fragment from the same droplet. It is what makes the experiment single-cell at all: the library is one tube, and the cells exist only as groups of reads sharing a barcode.

A unique molecular identifier, or UMI, is a random sequence attached to each original transcript molecule. It exists because PCR amplifies unevenly by orders of magnitude, so counting reads counts amplification success. Counting distinct UMIs instead counts molecules, which is the quantity that was actually there.

Both identifiers are short, and sequencing errors turn one into another. A one-base error in a UMI splits one molecule into two; a one-base error in a barcode moves reads between cells. The methods that work do not treat observed identifiers as ground truth — they collapse them under an error model, typically by building a graph of identifiers within an edit distance of each other and merging by abundance (Smith et al. 2017).

Barcodes are not cells until something decides they are

A droplet library produces far more barcodes than cells, most of them containing only ambient material. Separating real cells from empty droplets is a modelling decision made by the quantification tool, usually from the shape of the distribution of counts per barcode, and it is where two tools most visibly disagree about the same data. A “number of cells” is an output of that decision, not a property of the experiment.

18.3 What a zero means

Most entries in a single-cell count matrix are zero, and the temptation is to treat that as a defect to be repaired. It is worth being precise about what a zero can be.

It can mean the gene was not expressed in that cell. It can mean the gene was expressed and no molecule of it was captured, which is likely for any gene present at a few copies given that capture efficiency is well below one. The first is biology and the second is sampling, and nothing in the data distinguishes them for an individual entry.

What can be established is whether there are more zeros than sampling alone would produce. For droplet data with UMIs, the answer is largely no: the observed counts are consistent with a straightforward count distribution, and the excess-zero phenomenon reported in earlier full-length protocols is not a general property of the technology (Svensson 2020). This matters because it removes the motivation for a whole class of zero-inflated models and for aggressive imputation.

Sparsity here is under-sampling, not corruption. That reframing has a practical consequence: the right response to a sparse matrix is a model that accounts for how little was sampled, not a procedure that fills the zeros in. Imputation invents structure that the downstream clustering will then discover, and it is very hard to tell that outcome from a result.

18.4 A droplet is not a cell

Two artifacts come from the chemistry rather than from the sequencing, and both are properties of a working experiment rather than signs of a failed one.

Doublets are two cells captured under one barcode. They are unavoidable at any useful loading concentration — the rate rises with the number of cells recovered — and the result is a barcode whose transcriptome is a mixture. A doublet of two different cell types reads as a cell expressing both programs, which is indistinguishable in form from a genuine transitional or doubly positive population, and that is precisely the kind of finding people get excited about.

Every detection method available works the same way, because the problem allows little else: build artificial doublets by summing observed cells, embed them alongside the real ones, and score each real cell by how much of its neighbourhood is artificial. The approach has a structural blind spot. A doublet of two cells of the same type looks like one cell of that type with higher counts, so it is invisible to the method and mostly harmless to the conclusions.

Ambient RNA is material from cells that lysed before capture, distributed through the suspension and therefore present in every droplet. It makes each cell look slightly like the average of the sample, which systematically weakens the very distinctions the experiment is for. It is estimable exactly because it is everywhere: the empty droplets contain nothing else, so they measure the background directly.

Both corrections can be run too hard

Ambient correction removes counts that a model attributes to background, and a marker gene genuinely expressed at low level in a rare population looks like background. Doublet removal discards cells whose neighbourhood is unusual, and a genuinely intermediate population is unusual. Both are worth doing and both can delete the finding. Running the analysis with and without is cheap; not knowing which side of that comparison your result lives on is not.

18.5 Normalization, and why it is still argued about

Cells differ enormously in total counts, for reasons that include cell size and RNA content — which are biology — and capture efficiency, which is not. Comparing cells requires putting them on a common scale, and this is the step on which the field has least consensus.

The default is to scale each cell by its total count and take a logarithm. The scaling assumes total count is a nuisance, which is only partly true. The logarithm stabilizes variance so that highly expressed genes do not dominate every distance calculation, and it requires adding a pseudocount to handle zeros, whose size quietly changes how much weight the near-zero entries carry.

Alternatives exist — pooling counts across similar cells before estimating per-cell factors, so that a cell whose own total is too small to be a reliable divisor borrows from its neighbours; and residual-based transforms derived from an explicit count model rather than from a log. Careful comparison finds no transform dominant, and finds the simple default competitive with more elaborate ones for the purposes it is usually put to (Ahlmann-Eltze and Huber 2023).

Normalization is not a preprocessing detail with a right answer waiting to be looked up. It determines what “similar” means, and everything downstream — the neighbour graph, the clustering, the embedding — is a function of that definition. Two analyses of the same data that normalize differently are not two runs of one analysis.

18.6 Clustering is a hypothesis, and gets read as a result

The standard path from a normalized matrix to an answer is: select variable genes, reduce dimensions, build a nearest-neighbour graph, partition it, and draw the result in two dimensions. Every step has parameters, and the partition step has one — resolution — that directly sets how many clusters appear.

There is no number of cell types in a dataset waiting to be found. Clustering algorithms of this kind return a partition for whatever resolution they are given, and they return one whether or not the data contain discrete groups at all. Continuous variation — a differentiation gradient, an activation spectrum — is partitioned as readily as discrete variation, and the output looks the same.

The two-dimensional embedding compounds this. Projecting hundreds of dimensions into two cannot preserve the structure that was there, and the distortions are not random: apparent distances between clusters, the density of points within them and the presence of gaps are all substantially artifacts of the projection and its parameters (Chari and Pachter 2023). The picture is a navigational aid. It is not evidence, and quantitative claims read off it — this population is closer to that one, this gap is real — are claims about the algorithm.

The resolution parameter is a decision about the answer

Raising the resolution until a population of interest separates, then reporting that it separated, is circular. The parameter is a choice, so state it, and support a claim that a cluster is real with something the clustering did not produce — a marker known in advance, a replicate, an orthogonal assay, or a prospective test on data held out from the choice.

18.7 Batch and biology have the same shape

Two datasets from different runs, donors, chemistries or laboratories differ, and integration removes that difference so shared cell types coincide. The methods are effective and the concept has a hard limit worth stating plainly.

No integration method can distinguish technical variation from biological variation. It removes variation associated with the labels you supply. If a donor label happens to track disease status, correcting for donor removes the disease effect with the same enthusiasm, and the output looks like a successful integration either way — well-mixed batches are what the objective rewards.

That makes integration quality a genuinely two-sided measurement: how well batches mix, and how well distinct biological populations stay distinct. The methods trade these against each other and independent benchmarking finds none dominant across datasets, with the balance depending on how strong the batch effect is relative to the biology (Luecken et al. 2022).

The binding constraint is the experimental design, not the algorithm. If condition is confounded with batch — all cases processed on Monday, all controls on Tuesday — no integration method can recover the comparison, because the two effects are the same vector in the data. This is the single-cell instance of a general point: computation cannot separate what the design combined.

18.8 What to take forward

Each cell is severely under-sampled, and every method here is a strategy for borrowing information between cells.
Barcodes and UMIs are error-prone identifiers collapsed under a model; the number of cells is a decision, not an observation.
A zero is unresolvable between “absent” and “not captured” at the level of one entry, but the data are not zero-inflated beyond ordinary sampling, so the answer is a count model rather than imputation.
Doublets and ambient RNA are properties of working droplet chemistry, both correctable, and both correctable too far.
Normalization defines similarity, no transform is dominant, and it is not a detail that can be deferred.
Clustering returns a partition at whatever resolution you ask for, with or without discrete structure in the data, and the two-dimensional picture distorts distances and gaps.
Integration removes variation associated with the labels it is given and cannot tell batch from biology; confounded designs stay confounded.

A broader survey of current practice across the whole workflow, with much more detail than this chapter carries, is available as a community review (Heumos et al. 2023).

Section 19.1 names what to run.