Benchmarks — BioLang

Full Results

All 30 tasks across 6 categories. Click tabs to switch platforms.

Linux (WSL2) — Intel i9-12900K, 16 GB RAM — BioLang 0.3.0, Python 3.12.3, R 4.3.3

Task	BioLang	Python	R	Speedup vs Py
Sequence I/O
FASTA Small (30 KB)	0.138s	0.926s	1.243s	6.7x
FASTA Medium (4.6 MB)	0.170s	0.991s	1.371s	5.8x
FASTA Statistics	0.995s	1.590s	1.764s	1.6x
FASTA Large (51 MB)	0.821s	1.649s	2.103s	2.0x
FASTA gzipped (1.3 MB)	0.141s	0.930s	1.327s	6.6x
FASTQ QC	1.960s	3.551s	4.404s	1.8x
Real-world Genomes
E. coli Genome	0.176s	1.081s	1.354s	6.1x
Human Chr22	1.126s	1.673s	2.092s	1.5x
GC Content (51 MB)	0.830s	2.771s	2.358s	3.3x
K-mer Analysis
K-mer Counting	6.551s	21.01s	—	3.2x
Chr22 21-mers	10.72s	28.73s	—	2.7x
Protein K-mers	0.191s	1.331s	1.298s	7.0x
Intervals & Overlaps
ENCODE Peak Overlap	0.363s	2.574s	—	7.1x
Write Filtered FASTA	0.821s	1.679s	4.924s	2.0x
Variant & Data
VCF Filtering	0.349s	0.166s	6.312s	Py 2.1x faster
ClinVar Variants	0.661s	0.265s	1.208s	Py 2.5x faster
CSV Join + Group-by	0.281s	0.156s	0.312s	Py 1.8x faster
GFF3 Ensembl chr22	0.453s	0.171s	—	Py 2.6x faster
BED Overlap (synthetic)	0.160s	0.067s	1.279s	Py 2.4x faster
Pipelines
FASTQ QC Pipeline	2.349s	5.059s	—	2.2x
Variant Analysis Pipeline	0.268s	0.177s	—	Py 1.5x faster
ClinVar Variant Pipeline	0.591s	0.261s	—	Py 2.3x faster
Multi-Sample Aggregation	0.245s	0.090s	—	Py 2.7x faster
RNA-seq DE Analysis	0.114s	0.054s	—	Py 2.1x faster
Variant Annotation	0.309s	0.122s	—	Py 2.5x faster
ClinVar + Ensembl Annotation	0.409s	0.183s	—	Py 2.2x faster

Amber rows: Python wins on VCF/CSV/GFF text parsing and pipeline orchestration where Python's regex, csv, and dict modules are highly optimized C extensions. BioLang excels at I/O-heavy and compute-heavy tasks (FASTA/FASTQ parsing, k-mer counting, interval overlaps).

Task	BioLang	Python	R	Speedup vs Py
Sequence I/O
FASTA Statistics	1.02s	1.04s	—	~1.0x
FASTQ QC	2.02s	3.04s	—	1.5x
FASTQ (26 MB)	1.02s	2.05s	—	2.0x
Reverse Complement	1.02s	1.03s	—	~1.0x
Real-world Genomes
E. coli Genome	1.02s	1.04s	—	~1.0x
Human Chr22	1.01s	1.04s	—	~1.0x
GC Content (51 MB)	1.02s	1.04s	—	~1.0x
K-mer Analysis
K-mer Counting	6.04s	18.14s	—	3.0x
Chr22 21-mers	9.08s	24.24s	—	2.7x
Protein K-mers	1.01s	1.03s	—	~1.0x
Intervals & Overlaps
ENCODE Peak Overlap	1.03s	3.03s	—	2.9x
BED Overlap (synthetic)	1.01s	1.03s	—	~1.0x
Variant & Data
VCF Filtering	1.02s	1.03s	—	~1.0x
ClinVar Variants	1.01s	1.03s	—	~1.0x
CSV Join + Group-by	1.02s	1.03s	—	~1.0x
Pipelines
FASTQ QC Pipeline	2.03s	5.06s	—	2.5x
Variant Analysis Pipeline	1.02s	1.03s	—	~1.0x
ClinVar Variant Pipeline	1.02s	1.02s	—	~1.0x
Multi-Sample Aggregation	1.01s	1.05s	—	~1.0x
ClinVar + Ensembl Annotation	1.01s	1.04s	—	~1.0x

K-mer counting uses canonical (strand-agnostic) 21-mers in BioLang vs forward-only in Python — BioLang does strictly more work.

Methodology

Hardware

CPU: Intel Core i9-12900K (16C/24T)
Linux: WSL2, 16 GB RAM
Windows: Windows 11 Pro, 32 GB RAM
Storage: NVMe SSD

Measurement

Runs: Best of 3, wall-clock time
BioLang: v0.3.0 (tree-walking interpreter)
Python: 3.12.3 (Linux) / 3.12.4 (Windows)
R: 4.3.3 (Linux only)

Data Sources

Synthetic: Generated FASTA/FASTQ/VCF/BED/CSV
Real-world: NCBI (E. coli, Human Chr22), UniProt, ClinVar, ENCODE, Ensembl GFF3
Sizes: 30 KB to 51 MB per file

Notes

K-mers: BioLang uses canonical (strand-agnostic) 21-mers; Python uses forward-only. BioLang does strictly more work.
Python wins: VCF/CSV/GFF parsing and pipeline orchestration use Python's highly optimized C extensions (regex, csv, dict).
Windows: ~1s process creation overhead compresses speedup ratios for sub-second tasks.

Correctness Validation

Two validation suites — synthetic and real-world — cross-validate 9 tasks against Python (BioPython) and R (Bioconductor) to ensure BioLang produces identical or numerically equivalent results.

Generated test data — controlled inputs for deterministic output comparison

Task	Validated Against	Tolerance
GC Content	Python, R	Float ±1e-6
K-mer Counting	Python	Exact integer
VCF Filtering	Python, R	Exact integer
Reverse Complement	Python, R	Exact string
Translate	Python, R	Exact string
CSV Group-by	Python, R	Float ±1e-6
GFF Features	Python, R	Exact integer
Sequence Stats	Python, R	Float ±1e-6
BED Intervals	Python, R	Exact integer

Task	Data Source	Validated Against	Tolerance
GC Content	S. cerevisiae genome (16 chroms)	Python, R	Float ±1e-6
K-mer Counting	E. coli K-12 genome (50 KB)	Python	Exact integer
VCF Filtering	ClinVar (5,000 variants)	Python, R	Exact integer
Reverse Complement	S. cerevisiae (5 chroms, 200bp)	Python, R	Exact string
Translate	S. cerevisiae (3 chroms, 99bp)	Python, R	Exact string
CSV Group-by	ClinVar variants CSV	Python, R	Float ±1e-6
GFF Features	E. coli K-12 GFF3 annotation	Python, R	Exact integer
Sequence Stats	S. cerevisiae genome	Python, R	Float ±1e-6
BED Intervals	E. coli gene coords (from GFF)	Python, R	Exact integer

How it works

Each validated task outputs JSON from all three languages. A recursive diff compares the outputs field-by-field, applying the specified tolerance for floating-point values and exact comparison for integers and strings.

Validation scripts and real-world data download tool live in the benchmarks/correctness/ directory of the repository.

Reproduce the Benchmarks

Clone the repo and run the benchmark suite yourself.

# Clone and enter the benchmarks directory

$ git clone https://github.com/oriclabs/biolang.git

$ cd biolang/benchmarks

# Install BioLang (if not already)

$ cargo install biolang

# Generate test data

$ python generate_data.py

# Run all benchmarks (Linux)

$ bash run_all.sh

# Run correctness validation (synthetic data)

$ bash correctness/validate.sh

# Run real-world validation (E. coli, yeast, ClinVar)

$ python correctness/download_real_data.py

$ bash correctness/validate_real.sh

Raw results, scripts, and per-task analysis available at github.com/oriclabs/biolang/tree/main/benchmarks

BioLang Performance