File Formats

BioLang provides built-in readers and writers for all major bioinformatics file formats. The read_* readers load reusable tables or lists. Their short counterparts (fasta, fastq, bed, gff, vcf) return lazy streams for constant-memory processing. Writers accept streams or collections and handle buffering automatically.

FASTA

FASTA files contain one or more sequences, each preceded by a header line starting with >. BioLang provides two ways to read FASTA:

# Table mode (eager, reusable) — best for small/medium files
let records = read_fasta("data/sequences.fasta")
records |> filter(|r| r.length > 100)    # works
records |> map(|r| r.id)                 # still works — it's a Table

# Stream mode (lazy, one-time) — best for large files
records = fasta("data/sequences.fasta")
records |> map(|r| r.id) |> collect()    # works

# Each record has: id, description, seq, length
read_fasta("data/sequences.fasta")
  |> map(|r| print(r.id, seq_len(r.seq)))

# Filter and write to a new file
read_fasta("data/sequences.fasta")
  |> filter(|r| seq_len(r.seq) > 10000)
  |> write_fasta("long_contigs.fa")

# Read gzipped FASTA (auto-detected by extension)
read_fasta("data/sequences.fasta")
  |> map(|r| gc_content(r.seq))
  |> collect()

FASTQ

FASTQ extends FASTA with per-base quality scores. Records have id, seq, and quality fields. Quality scores are Phred+33 encoded:

# Table mode (eager, reusable)
let reads = read_fastq("data/reads.fastq")
reads |> filter(|r| r.length > 50)       # reusable

# Quality filtering
read_fastq("data/reads.fastq")
  |> filter(|r| mean_phred(r.quality) >= 30)
  |> write_fastq("filtered.fq.gz")

# Compute per-read statistics
read_fastq("data/reads.fastq")
  |> map(|r| {
    id: r.id,
    gc: gc_content(r.seq),
    mean_qual: mean_phred(r.quality)
  })
  |> to_table()
  |> write_csv("read_stats.csv")

BED

BED files describe genomic regions. BioLang supports BED3 through BED12. Each record maps to a table row with named columns:

# Read a BED file into a table
let regions = read_bed("data/regions.bed")

# Columns: chrom, start, end, name, score, strand (BED6)
regions
  |> filter(|r| r.score > 500)
  |> mutate("width", |r| r.end - r.start)
  |> print()

# Convert BED records to intervals for set operations
let intervals = read_bed("data/regions.bed")
  |> map(|r| interval(r.chrom, r.start, r.end, r.strand))
  |> collect()

# Write intervals back to BED
intervals |> write_bed("output.bed")

GFF / GTF

GFF3 and GTF are tab-delimited annotation formats. The attributes column remains a string; use parse_attr() to parse all attributes or retrieve one key:

# Read a GFF3 file
let annotations = read_gff("data/annotations.gff")

# Fields: seqid, source, type, start, end, score, strand, phase, attributes
annotations
  |> filter(|r| r.type == "gene")
  |> map(|r| {
    gene_id: parse_attr(r.attributes, "ID"),
    name: parse_attr(r.attributes, "Name"),
    chrom: r.seqid,
    start: r.start,
    end: r.end
  })
  |> to_table()
  |> print()

# Read GTF
read_gff("data/annotations.gff")
  |> filter(|r| r.type == "exon")
  |> map(|r| parse_attr(r.attributes, "gene_name"))
  |> unique()
  |> len()
  |> print()

VCF

VCF files contain variant calls. read_vcf() returns a reusable list of typed Variant values. INFO data is parsed lazily when v.info is accessed:

# Read a VCF file
let variants = read_vcf("data/variants.vcf")

# Fields: chrom, pos, id, ref, alt, qual, filter, info
variants
  |> filter(|v| v.qual > 30 && v.filter == "PASS")
  |> print()

# Access variant fields
read_vcf("data/variants.vcf")
  |> map(|v| {
    chrom: v.chrom,
    pos: v.pos,
    ref_allele: v.ref,
    alt_allele: v.alt,
    qual: v.qual
  })
  |> to_table()
  |> write_csv("variant_summary.csv")

# Write filtered VCF
read_vcf("data/variants.vcf")
  |> filter(|v| v.chrom == "chr1")
  |> write_vcf("chr1_variants.vcf")

SAM / BAM

SAM and BAM files contain aligned reads. BAM files are the binary compressed form. BioLang reads both transparently:

# Read a BAM file (returns a table of alignments)
let alignments = read_bam("sorted.bam")

# Fields: qname, flag, rname, pos, mapq, cigar, rnext, pnext, tlen, seq, qual
alignments
  |> filter(|a| a.mapq >= 30)
  |> map(|a| { name: a.qname, chrom: a.rname, pos: a.pos })
  |> to_table()
  |> print()

# Read SAM (text format)
let sam_records = read_sam("alignments.sam")
print(nrow(sam_records), "alignments")

MAF (Multiple Alignment Format)

MAF files store multiple sequence alignments, commonly used in comparative genomics. Each alignment block contains sequences from different species aligned to each other:

# Read a MAF file
let records = read_maf("multiz30way.maf")

# MAF records are returned as a table
records |> print()

Format-Specific Readers

Each format has its own reader function. Use the appropriate one for your file:

# Each format has a dedicated reader
let variants = read_vcf("data/variants.vcf")
let reads = read_fastq("data/reads.fastq")
let alignments = read_bam("data/aligned.bam")
let regions = read_bed("data/regions.bed")
let annotations = read_gff("data/annotations.gff")
let sequences = read_fasta("data/sequences.fasta")

Format Mismatch Detection

All read functions check the file's content before parsing and return a clear error if the format doesn't match. This catches common mistakes like passing a FASTA file to fastq():

# Wrong function for the file format
read_fastq("data/sequences.fasta")
# TypeError: read_fastq(): file appears to be FASTA format
#   (starts with >), not FASTQ. Use read_fasta() instead

# Works both ways
read_fasta("data/reads.fastq")
# TypeError: fasta(): file appears to be FASTQ format
#   (starts with @), not FASTA. Use fastq() instead

# Also detects binary vs text mismatches
sam("aligned.bam")
# TypeError: sam(): file appears to be BAM (binary) format,
#   not SAM (text). Use bam() instead

# Stream — consumed once (good for large files)
let records = fasta("data/sequences.fasta")
records |> map(|r| r.id) |> collect()   # ["seq1", "seq2"]

# Table — reusable (good for small/medium files)
records = read_fasta("data/sequences.fasta")
records |> map(|r| r.id)     # works
records |> map(|r| r.seq)    # still works — it's a Table

# Other short readers are also lazy:
vcf("data/variants.vcf")
bed("data/regions.bed")
gff("data/annotations.gff")

Writing Files

Every reader has a corresponding writer. Writers accept streams, collections, or tables and handle format-specific details like compression:

# Write functions match read functions
write_fasta(records, "output.fa")
write_fastq(records, "output.fq.gz")    # .gz = auto-compress
write_bed(intervals, "regions.bed")
write_vcf(variants, "filtered.vcf.gz")

# Pipe-friendly syntax (stream |> write)
read_fasta("data/sequences.fasta")
  |> filter(|r| seq_len(r.seq) > 500)
  |> write_fasta("filtered.fa")

# Construct a variant: variant(chrom, pos, ref, alt)
let v = variant("chr17", 7674220, "G", "A")

# Access variant fields
println(v.chrom, ":", v.pos, v.ref, ">", v.alt)

# VCF reader returns Variant values
let variants = read_vcf("data/variants.vcf")
for v in variants {
  if v.filter == "PASS" && v.qual > 30 {
    println(v.chrom, v.pos, v.ref, ">", v.alt)
  }
}

# Type checking
println(type(v))            # "Variant"
println(is_snp(v))          # true
println(variant_type(v))    # "SNP"