# Demultiplex Raw Sequences

When pooling samples and sequencing them in parallel on an Illumina sequencer, you will be given large multiplexed FASTQ files in return. These files contain sequences for all of your samples and need to be demultiplexed using barcodes to separate the sequences for each sample into their own files (a forward and reverse file for each sample). These barcodes should have been added during the sample DNA preparation in a laboratory. The demultiplexing strategy will vary based on the haplotagging technology you are using (read Haplotagging Types).

harpy demultiplex METHOD OPTIONS... R1_FQ R2_FQ I1_FQ I2_FQ

harpy demultiplex gen1 --threads 20 --schema demux.schema Plate_1_S001_R*.fastq.gz Plate_1_S001_I*.fastq.gz

In addition to the common runtime options , the demultiplex gen1 module is configured using these command-line arguments:

It's not uncommon that some sequences cannot be demultiplexed due to sequencing errors at the ID location. Use --keep-unknown-samples/-u to have Harpy still separate those reads from the original multiplex. Those reads will be labelled _unknown_sample.R*.fq.gz

It's likewise not uncommon that sequencing errors make it so that the sequences don't match the list of known barcode segments. Use --keep-unknown-barcodes/-b to have Harpy separate those reads out from the original multiplex as _unknown_barcodes.R*.fq.gz.

Using --qx-rx, you can opt-in to retain the QX:Z (barcode PHRED scores) and RX:Z (nucleotide barcode) tags in the sequence headers. These tags aren't used by any subsequent analyses, but may be useful for your own diagnostics.

Sample01    C01
Sample02    C02
Sample03    C03
Sample04    C04

Sample01    D01
Sample02    D02
Sample03    D03
Sample03    D21

Sample01    C01
Sample02    C02
Sample03    C02

Sample01    C01
Sample02    D02
Sample03    C03

Barcode correction and migration into the read headers is performed using demult_fastq (Harpy renames it to demuxGen1), which is distributed by the team behind haplotagging. Demultiplexing the pooled FASTQ files into individual samples is performed in parallel and using the beloved workhorse grep.

graph LR
    subgraph Inputs
        direction TB
        A[multiplexed FASTQ]:::clean---BX
        BX[index reads FASTQ]:::clean---SCH
        SCH[Sample Schema]:::clean
    end
    Inputs-->B([demultiplex samples]):::clean
    B-->D([quality metrics]):::clean
    style Inputs fill:#f0f0f0,stroke:#e8e8e8,stroke-width:2px
    classDef clean fill:#f5f6f9,stroke:#b7c9ef,stroke-width:2px

The default output directory is Demultiplex with the folder structure below. Sample1 and Sample2 are generic sample names for demonstration purposes. The resulting folder also includes a workflow directory (not shown) with workflow-relevant runtime files and information.

Demultiplex/
├── Sample1.F.fq.gz
├── Sample1.R.fq.gz
├── Sample2.F.fq.gz
├── Sample2.R.fq.gz
└── reports
    └── demultiplex.QC.html

item	description
*.F.fq.gz	Forward-reads from multiplexed input --file belonging to samples from the samplesheet
*.R.fq.gz	Reverse-reads from multiplexed input --file belonging to samples from the samplesheet
reports/demultiplex.QC.html	phased vcf annotated with phased blocks

FASTQC metrics

This is the summary report Harpy generates for this workflow. You may right-click the image and open it in a new tab if you wish to see the example in better detail.

Harpy v2 introduced a new demultiplexer under the hood called dmox, which is singificantly faster, lighter on memory, and has better maintenance than the previous solution. Iago Bonnici of Montpellier Bioinformatics Biodiversity (MBB) saw the need for better demultiplexing performance and took it upon themselves to donate their time to write a brand-new purpose-built demultiplexer for the Meier/Chan haplotagging bead design. Beyond just being way more performant, this new demultiplexer has more features, has more output options, and is flexible for haplotagging bead designs where the sample ID is not the C-segment. If you're happy with the performance of the new demultiplexing workflow, please let Iago/MBB know!