4 Quality Control and Preprocessing: Concepts

Concepts

4.1 The problem

Between the instrument and the analysis there is a step everyone performs and few think about: looking at the reads, and deciding what to do about what you see.

It is worth thinking about, because both failure modes are expensive. Skipping it means a contaminated, swapped or failed library propagates through every downstream stage and is discovered — if at all — as a biological anomaly. Overdoing it means removing real signal in pursuit of tidy-looking data, which is worse, because it leaves no trace and produces results that look fine.

The useful framing is that QC answers one question and preprocessing answers a different one. QC asks: is this library what I think it is? That question is about the experiment, and its answer is a decision about whether to proceed. Preprocessing asks: which parts of these reads are not biological? That question is about the sequence, and its answer changes the data. Conflating them produces the two classic mistakes — trimming until the plots look good, and treating a green report as a certificate of correctness.

4.2 Errors are not uniform, and their shape is the signal

Every platform makes mistakes with a characteristic structure, and knowing the structure is what makes a QC plot readable.

Short-read sequencing-by-synthesis has an error rate that rises along the read as the cluster loses synchrony, so quality declining toward the 3’ end is expected rather than alarming. Its errors are predominantly substitutions. Long-read platforms have historically had much higher raw error rates concentrated in insertions and deletions, particularly in homopolymers, where the difficulty is counting identical bases rather than identifying them.

That difference propagates into everything downstream. It is why indel-aware alignment matters more for long reads, why variant callers ship per-platform models (Section 10.1), and why an assembler’s assumptions about its input are the first thing to check (Section 8.2).

Quality scores are estimates, not measurements

A Phred score is the instrument’s own claim about how often it is wrong, and it is calibrated on data that may not resemble yours. Basecallers are also retrained, so scores from one software version are not comparable to another’s. Treat quality as a strong hint about relative reliability within a run, not as a probability you can do arithmetic with across runs.

4.3 What QC can and cannot tell you

QC reports are diagnostic, not evaluative, and the distinction is where most misreading happens.

QC is good at detecting gross failures: a library that is mostly adapter, a run that failed partway, a sample that is not the organism claimed, a catastrophic duplication rate. These are unambiguous and the correct response is usually to stop.

QC cannot tell you whether your experiment will work. It has no access to the biology, the design, or the question. A library can pass every check and be useless because it was the wrong tissue, the wrong timepoint, or underpowered.

Pass and fail flags are not about your experiment

The thresholds in a standard QC report encode expectations for a generic whole-genome shotgun library. Amplicon data fails duplication and diversity checks by construction. Small-RNA libraries fail length checks. Bisulfite data fails base-composition checks spectacularly. RNA-seq shows duplication that is expression, not artifact. A red flag on the right kind of library is information about the check, not about the data.

4.4 Adapters are contamination; low quality usually is not

Two different things get called “trimming”, and the case for them is not the same.

Adapter removal is uncontroversial. Adapter sequence is not from the organism. It appears when the fragment is shorter than the read, so the sequencer runs off the end of the insert and into the adapter, and it is concentrated in exactly the short fragments you might care about. Left in place, it forces aligners to explain sequence with no genomic origin — the failure described in Section 6.1. Remove it.

Quality trimming is a different proposition. Cutting bases because their scores are low was standard advice from an era when downstream tools could not handle error, and it has aged poorly. Modern aligners soft-clip; modern callers weight evidence by quality rather than assuming it away. Aggressive quality trimming removes real bases that those tools would have used correctly, shortens reads so they map less uniquely, and — most insidiously — introduces length biases that correlate with sequence composition.

The asymmetry to remember: an adapter base is definitely not biological, and a low-quality base probably is. The first is contamination and removing it is free. The second is uncertain evidence, and removing it discards information a model could have used, while keeping it costs you nothing if the model understands quality. Trim adapters always; trim on quality only when you can say which downstream step required it.

4.5 Duplicates come from the library, not the sequencer

Two reads with identical sequence and position are usually not two independent observations. They are usually the same original molecule, amplified during library preparation and sequenced twice.

That matters because almost every downstream method treats reads as independent evidence. A variant supported by five duplicates of one molecule has the support of one molecule, not five. Counting them separately inflates confidence in whatever that molecule happened to say, including its errors.

The standard response is to identify duplicates by their alignment coordinates and mark them, so downstream tools can ignore them. This works and has a clear limitation: at high coverage, or with short fragments, two genuinely independent molecules can share a start and end position by chance, and coordinate-based marking cannot tell the difference.

Unique molecular identifiers resolve this by tagging each original molecule with a random barcode before amplification. Reads sharing coordinates and a tag are duplicates; sharing coordinates alone is not enough. This is the difference between inferring duplication and observing it.

When duplicates are the data

Duplicate removal assumes duplication is an artifact. For RNA-seq that assumption is wrong — a highly expressed transcript produces many genuinely independent fragments from the same positions, and removing them discards exactly the signal you are measuring. The same applies to amplicon sequencing and to any targeted assay. Marking duplicates is near-universal advice that does not apply universally.

4.6 Contamination and the sample you did not intend

The failure QC is most valuable for catching is the one that is invisible downstream: this is not the sample you think it is.

Sample swaps happen. Index hopping puts reads from one library into another’s output. Reagents carry microbial DNA, which dominates when input is low. Cross-species contamination in a shared facility is routine.

None of this shows up as bad quality scores — the contaminating reads are perfectly good reads of the wrong thing. The way to see it is to ask where reads actually map, across several candidate genomes, rather than assuming. A small subsample answers the question cheaply, and it is worth doing on every library rather than on the ones that already look suspicious.

4.7 What to take forward

QC asks whether the library is what you think; preprocessing asks which bases are not biological. They are different questions with different consequences.
Error profiles are platform-specific and structured, and that structure explains choices made in every later part of this book.
QC detects gross failure. It cannot evaluate an experiment, and its pass/fail flags encode assumptions about a generic library.
Adapter removal is free; quality trimming costs real information and needs a reason.
Duplicates are a library-prep artifact, coordinate-based marking is an inference, and for expression data the assumption behind it is wrong.
The most valuable thing QC catches is contamination and sample identity, which no quality score reveals.

Section 5.1 names what to run.