Bioinformatics file formats, explained

FASTA

Plain text for DNA, RNA or protein sequences. Each record starts with a header line beginning with >; the first word is the identifier and the rest is an optional description. Sequence lines follow, usually wrapped at 60 to 80 characters.

>seq1 example nucleotide sequence
ATGGCGTACGTTAGCCTAGGCTAACGTAGCTAGGCTAACGATCGATCGTAGCTAGCTAGGATC
GATCGGCTAGCTAACG
>seq2 a second record
MKTAYIAKQRQISFVKSHFSRQ

• Common extensions: .fa, .fasta, .fna (nucleotides) and .faa (amino acids).
• Index a large FASTA with samtools faidx to extract regions quickly.
• Lowercase bases in genome FASTA files often mark soft-masked repeats; they are still valid sequence.
• Watch for duplicate identifiers and Windows line endings, which break many tools.

FASTQ and quality scores

Raw reads with a quality score for every base, in four-line records: a header beginning with @, the sequence, a separator line beginning with +, and the quality string, which has one character per base.

@read_001 sample=A lane=1
GATTTGGGGTTCAAAGCAGTATCGATCAAATAGT
+
IIIIIIIIIIIIIIIIIIIIIII?????55555+

Each quality character encodes a Phred score: Q = ASCII code − 33 (the Phred+33 or "Sanger" encoding used by all modern instruments). Q is defined as −10·log₁₀(P), where P is the probability that the base call is wrong.

• Paired-end data come as two files (often _R1 and _R2) whose records must stay in the same order.
• FASTQ files are usually gzipped: .fastq.gz or .fq.gz.
• Files from Illumina pipelines before version 1.8 may use Phred+64 instead.
• Decode any quality string with the Phred quality converter.

SAM, BAM and CRAM

Aligned reads. SAM is tab-separated text; BAM is the same information compressed in binary; CRAM compresses further by storing differences from the reference, so you need that same reference to read it. Header lines begin with @: @HD (version and sort order), @SQ (reference sequences), @RG (read groups) and @PG (programs used).

@HD	VN:1.6	SO:coordinate
@SQ	SN:ref	LN:45
r001	99	ref	7	30	8M2I4M1D3M	=	37	39	TTAGATAAAGGATACTG	*

That record (from the SAM specification) is read r001, aligned to ref at position 7 with flag 99 and CIGAR 8M2I4M1D3M. Every alignment line has eleven mandatory columns:

Optional fields follow as TAG:TYPE:VALUE, for example NM:i:2 (edit distance), AS:i: (alignment score) and RG:Z: (read group).

FLAG bits

CIGAR operations

• Sort BAM files by coordinate and index them (.bai or .csi) before region queries or viewing in IGV.
• POS is 1-based in SAM text; inside binary BAM it is stored 0-based, and tools convert for you.

VCF and BCF

Genetic variants and genotypes. Meta-information lines start with ## and define the INFO, FORMAT and FILTER fields used below them; a single header line starting with #CHROM names the columns.

##fileformat=VCFv4.2
##contig=<ID=chr1,length=248956422>
##INFO=<ID=DP,Number=1,Type=Integer,Description="Total depth">
##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
#CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO	FORMAT	sample1
chr1	10468	.	C	T	62	PASS	DP=35	GT	0/1
chr1	20345	.	GTC	G	48	PASS	DP=27	GT	1/1

Genotypes: 0/0 homozygous reference, 0/1 heterozygous, 1/1 homozygous alternate, 1/2 two different alternate alleles and ./. missing. A | instead of / means the genotype is phased.

• Indels include the reference base before the event, so the deletion above removes TC.
• Compress with bgzip and index with bcftools index or tabix for region queries; BCF is the binary equivalent.
• Normalise with bcftools norm before comparing call sets. See the bcftools cheat sheet.

BED

Genomic intervals, one per line, tab-separated. Three columns are required: chromosome, start and end. The start is 0-based and the end is exclusive, so the length of an interval is simply end − start.

chr1	999	2000	peak_1	500	+
chr1	4999	5100	peak_2	320	-

The first line covers 1-based positions 1,000 to 2,000 (1,001 bases). Optional columns 4 to 12 are name, score (0–1000), strand, thickStart, thickEnd, itemRgb, blockCount, blockSizes and blockStarts; the 12-column form (BED12) describes exon structures.

• Sort before merging or finding the closest feature: sort -k1,1 -k2,2n.
• Chromosome names must match your reference (chr1 versus 1).
• See the bedtools cheat sheet for overlaps and coverage.

GFF3 and GTF

Gene annotation: genes, transcripts, exons and coding sequences, with nine tab-separated columns: seqid, source, type, start, end, score, strand, phase and attributes. Coordinates are 1-based and inclusive. Phase (0, 1 or 2) applies to CDS features and gives the number of bases to skip to reach the next codon.

##gff-version 3
chr1	example	gene	1000	9000	.	+	.	ID=gene1;Name=EX1
chr1	example	mRNA	1050	9000	.	+	.	ID=mrna1;Parent=gene1
chr1	example	exon	1050	1500	.	+	.	Parent=mrna1
chr1	example	CDS	1201	1500	.	+	0	Parent=mrna1

GTF (also called GFF2.5) uses the same columns but writes attributes as key "value"; pairs and requires gene_id and transcript_id:

chr1	example	exon	1050	1500	.	+	.	gene_id "gene1"; transcript_id "mrna1";

• Use the annotation release that matches your reference assembly, and record its version.
• Aligners such as STAR and counters such as featureCounts usually expect GTF.

Coordinate systems

Off-by-one errors are among the most common bugs in bioinformatics. The same first 100 bases of chr1 are written differently in each format:

To convert a 1-based inclusive region to BED, subtract 1 from the start and keep the end. To go back, add 1 to the BED start.

IUPAC nucleotide codes

The standard genetic code

Rows give the first base of the codon and columns the second; each cell lists the four possible third bases. ATG (methionine) is the usual start codon; TAA, TAG and TGA are stop codons. This is NCBI translation table 1.

Amino acids

Two rarer amino acids, selenocysteine (Sec, U) and pyrrolysine (Pyl, O), are encoded by recoded stop codons in some organisms.