How fast each building block is against the tools people would otherwise use,
and what each swappable backend costs, measured on genomeblocks 2.0.0.
Every comparison first checks that the engines return the same answer;
timings are medians of 3–5 runs after a warm-up.
Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz · 64 cores · 811 GB RAM ·
Python 3.12.12 · synthetic hg38-shaped data (seeded) · benchmarks ran one at a time and use at
most 8 worker processes
intervals: fastest enginegenomeblocksmerge
bigwig: fastest enginepybigtoolssignal 10,000 x 1 track x 200 bins
motifs: fastest enginelightmotifscan 1,000 windows x 100 motifs
fasta: fastest enginememorysequences 10,000 x 500 bp
tables: fastest enginepolarsLoci.make 1,000,000 peaks (keep=True)
graph: fastest enginegraph-toolbuild the engine's graph
One overlap lookup8.8 µson 100k indexed peaks: 167× faster than pyranges, 607× than bioframe
bigWig → heatmap matrix98×faster than deepTools computeMatrix on one core (13k vs 137 regions/s)
Enrichment vs 500 peak files11 msper 20k-peak query including Fisher tests
Motif scanning131×faster than MEME FIMO, 187× faster than Biopython
Hi-C pair counting1.6M/sread pairs into 50 kb windows: 2.9× the rate of cooler cload
The same call through every installed engine (backend=); every row below returns the same answer as the default, so the choice is only about speed and what is installed.
intervals · A & B at n = 1,000,000; the default engine in colour.
intervals · merge at n = 1,000,000; the default engine in colour.
intervals · nearest (distances) at n = 100,000; the default engine in colour.
bigwig · signal 10,000 x 1 track x 200 bins at n = 10,000; the default engine in colour.
motifs · scan 1,000 windows x 100 motifs at n = 1,000; the default engine in colour.
fasta · sequences 10,000 x 500 bp at n = 10,000; the default engine in colour.
tables · Loci.make 1,000,000 peaks (keep=True) at n = 1,000,000; the default engine in colour.
tables · Genes.make (GTF) at n = 70,028; the default engine in colour.
graph · components (114,367 edges) at n = 114,367; the default engine in colour.
graph · pagerank (114,367 edges) at n = 114,367; the default engine in colour.
graph · build the engine's graph at n = 114,367; the default engine in colour.
Loci
A single overlap query against an indexed set is the cost of every interactive question (“what overlaps this peak?”): 8.8 µs per lookup through the cached point index (8.8 µs with genomeblocks (cgranges), 9.6 µs with genomeblocks (ncls), 14 µs with genomeblocks (numpy point index)), against 1.5 ms for pyranges, which has to slice a dataframe per call.
One overlap lookup against 100,000 indexed peaks (median per query; log scale).
A & B on 1M intervals per set (rows of A overlapping B); every engine returns the same rows.
merge on 1M intervals per set (sort + merge of A ∪ B); every engine returns the same rows.
On whole-set operations at 1M intervals the numpy kernel does A & B in 87 ms and merge in 135 ms, against 336 ms and 164 ms for pyranges; the Backends section times the other interval engines behind the same call.
All Loci timings
op
engine
n = 1k
n = 10k
n = 100k
n = 1M
intersect
genomeblocks
268 µs
901 µs
7.4 ms
87 ms
intersect
genomeblocks (cgranges)
1.1 ms
9.8 ms
99 ms
1.6 s
intersect
genomeblocks (cgranges, warm index)
1.1 ms
10 ms
99 ms
1.4 s
intersect
pyranges
49 ms
44 ms
62 ms
336 ms
intersect
bioframe
7.3 ms
12 ms
61 ms
1.0 s
intersect
bedtools (CLI)
16 ms
36 ms
169 ms
1.8 s
intersect
intervaltree
24 ms
464 ms
22 s
–
intersect
naive double loop
36 ms
2.2 s
–
–
merge
genomeblocks
366 µs
2.0 ms
18 ms
135 ms
merge
pyranges
28 ms
30 ms
52 ms
164 ms
merge
bioframe
16 ms
23 ms
102 ms
699 ms
merge
bedtools (CLI)
27 ms
92 ms
543 ms
5.1 s
make
genomeblocks
2.2 ms
6.6 ms
56 ms
394 ms
make
pyranges
5.7 ms
13 ms
91 ms
928 ms
make
pandas.read_csv
1.5 ms
4.2 ms
29 ms
253 ms
Signal
Filling a heatmap matrix (200 bins, ±3 kb) is one native call per region and track: 13k regions/s on one core, 98× deepTools computeMatrix and 386× pyBigWig’s binned stats(). Every engine returns the same matrix (deepTools differs only at bin edges, r ≥ 0.99).
Regions per second into a 200-bin matrix from one bigWig, one core (higher is better; log scale).
Throughput vs number of regions (regions/s, log–log). The native path speeds up with batch size as fixed costs amortise; per-base readers stay flat.
Many tracks scale with processes: 16 bigWigs × 5,000 regions take 5.3 s sequentially and 1.4 s with workers=8. Threads do not help (pybigtools serialises Python threads), which is why signal() uses processes.
16 bigWigs × 5,000 regions: worker processes vs threads vs deepTools -p (wall time; log scale).
Atlas
Testing a 20,000-peak query against 500 peak files is one sparse row-sum plus a vectorised Fisher test: 11 ms per query, including building the result table.
One query against 500 peak files (wall time; log scale). GIGGLE was not run (it is built from source; set GIGGLE= to include it).Index build, load, accuracy
step
bin size
workers
time
index size
build
200 bp
3
10 s
190 MB
build
1,000 bp
1
7.8 s
68 MB
build
1,000 bp
3
3.5 s
68 MB
build
5,000 bp
3
2.8 s
43 MB
load (Atlas.load (npz))
266 ms
load (fresh python: import + load + search)
4.2 s
accuracy vs exact overlaps
200 bp
Spearman 0.999
top-25 shared 21
accuracy vs exact overlaps
1,000 bp
Spearman 0.999
top-25 shared 20
accuracy vs exact overlaps
5,000 bp
Spearman 0.993
top-25 shared 18
bootstrap (10 shuffles)
8.7 ms per shuffle
bootstrap (100 shuffles)
7.5 ms per shuffle
Motifs
Counting hits of 100 JASPAR motifs in 1,000 windows of 500 bp: 58 ms with genomeblocks on MOODS (the windows joined into one block, the library scanned in one pass), 131× faster than FIMO. The 993 hits are identical to those of lightmotif per window, MOODS, numpy and Biopython; FIMO reports 996.
Motif scanning engines on identical PSSMs and windows (wall time; log scale).
The whole JASPAR library (1,019 motifs) on one core (wall time; log scale). Against MOODS: genomeblocks is 1.3× faster at 1,000 windows, genomeblocks is 1.6× faster at 5,000 windows. FIMO was timed at 1,000 windows only.Worker processes
workers
time
speed-up
1
1.4 s
1.0×
2
872 ms
1.6×
4
641 ms
2.2×
8
530 ms
2.7×
Hi-C pairs
Streaming 5M read pairs into 50 kb windows × partner chromosome runs at 1.6M pairs/s, close to the speed of just parsing the file, and 2.9× the rate of cooler cload.
Read pairs per second (higher is better; log scale). The naive loop was timed on the first 200k pairs.
Genes
A GENCODE-shaped GTF with 20,000 genes, 70,028 transcripts and 876,947 lines.
Gene-model steps (wall time; log scale). Labelling uses the cached annotation index; the counts are CREs labelled.
Architecture
The pipeline on 100k CREs and 50k loops (114,367 edges), step by step, then the graph algorithms on every installed graph engine and the parquet round trip:
Architecture pipeline steps (wall time; log scale). Graph algorithms run once per installed engine, with identical results; the default engine in colour.
HiChIP short-range track
From allValidPairs to a coverage bigWig: 17 s and 1.0 GB peak memory with genomeblocks.hichip, against 2.3 min and 1.9 GB for the awk | sort | bedtools recipe, with the same ends and a byte-identical bedGraph.
Steps
implementation
step
time
peak memory
shell
extract ends (awk)
6.1 s
20 MB
shell
fragments + sort + genomecov
2.0 min
1,903 MB
shell
bedGraph -> bigWig
8.9 s
–
genomeblocks
ends
4.5 s
–
genomeblocks
write BED
897 ms
–
genomeblocks
coverage (bedGraph)
3.6 s
–
genomeblocks
bigWig
7.8 s
–
Import cost
import genomeblocks is lazy, and from genomeblocks import Loci costs 201 ms on top of starting Python (37 ms): matplotlib, pandas, scipy and graph-tool load only when a function needs them.
Import cost above python -c pass, fresh process each (cumulative statements; log scale).
Reproduce
cd benchmarks
python make_data.py # synthetic hg38-shaped data, ~1.5 GBPY=python ./run_all.sh # every bench, one at a time
python build_site.py # this page: docs/benchmarks/index.md
External baselines are found on PATH (bedtools, deepTools computeMatrix,
cooler) or via FIMO=. The data are synthetic and seeded, laid out on real hg38
chromosome sizes; motifs are the JASPAR CORE vertebrate collection.
