vep-rs

A fast, memory-efficient reimplementation of the Ensembl Variant Effect Predictor (VEP) in Rust. It takes VEP’s inputs and flags, writes VEP’s consequence vocabulary in VEP’s output formats, and annotates population-scale callsets two orders of magnitude faster than the Perl implementation.

In this README, “Perl VEP” is the reference implementation, ensemblorg/ensembl-vep release 116.2. The accompanying paper calls the same engine “VEP”.

Why vep-rs instead of Ensembl VEP or fastVEP

The speed figures cover all of four population datasets (1,179,639,224 variants) under one protocol: the sum of per-chromosome medians over 20 machines at 16 threads. The concordance figures cover those four plus the two genome-wide ClinVar releases, six whole-genome datasets in all (1,188,425,523 variants, 11,838,755,797 annotation tuples, 11,838,744,378 matched), against Ensembl VEP 116.2. The Current release section has the tables and the method.

The adjusted F1 sets aside only the documented cases where Ensembl VEP’s own output is wrong, or where the two engines represent a record differently. Each case is traced to the mechanism in Ensembl VEP’s source and reproducible from the record (docs/intended-divergences.md). Outside those cases every tuple matches, so vep-rs’s remaining disagreements with Ensembl VEP are deliberate, documented corrections rather than defects.

The ten built-in plugin databases are CADD, REVEL, gnomADc, AlphaMissense, dbscSNV, LoFTEE, LoFtool and pLI on both GRCh37 and GRCh38, plus GWAS and SpliceAI on GRCh38. Dynamic-library plugins load from --dir_plugins (docs/plugins.md). Parquet output needs the duckdb CLI on PATH.

What “drop-in” means

vep-rs takes the same input files and flag names as Perl VEP and writes the same output layouts. Two things differ:

• The transcript cache. --json_cache names a directory built by vep-cache-builder or converted from a Perl VEP cache (docs/cache-setup.md). --cache, --offline, --dir_cache, --cache_version and --species are accepted but don’t locate a cache, and vep-rs never contacts a database.
• Some flags are accepted but not implemented. Among them are --regulatory, --custom, --refseq, --merged, --coding_only, --pick_order and the statistics flags (vep-rs writes no summary file). docs/cli-reference.md marks every such flag.

Current release: v0.4.0

These are the released version’s own figures, measured under the paper’s protocol against Ensembl VEP 116.2. The figures the paper reports, measured at its pinned build, are in docs/published-figures.md and stay as published. How each number was produced is under Method.

Whole-genome concordance

Against Ensembl VEP 116.2. A population dataset is scored per chromosome and its shards pooled; each ClinVar release is one genome-wide file. The per-chromosome tables are in the provenance records.

Whole-genome wall time

Seconds: a population dataset as the sum of its per-chromosome medians, each ClinVar release as its whole-file median. The figure in parentheses beside a comparator’s time is that time divided by vep-rs’s, taken only where both were measured under this protocol. The two right-hand columns are vep-rs on its largest chromosome, or on the whole file for ClinVar.

• vep-rs and fastVEP ran the population datasets on c8gd.8xlarge (ARM Graviton4) and c8id.8xlarge (x86 Intel), 32 vCPU with local NVMe.
• Ensembl VEP 115.2’s figures are its reference run of the same inputs under the same protocol, on r8gd.8xlarge and r8id.8xlarge (the same processors with 256 GiB), carried forward from the earlier reference generation. The concordance tables compare against Ensembl VEP 116.2’s output.
• The ClinVar rows are the released version’s whole-file medians on the paper’s instance types (r8gd.8xlarge, r8id.8xlarge); the comparators’ times on them are carried forward from an earlier measurement and labelled with the engine version they were measured at, and no ratio is taken against them.

Structural variants

The structural-variant per-VCF sets (16 files per assembly, the files’ counts pooled), against Ensembl VEP 116.2. vep-rs v0.3.2’s rows are its re-scoring against the same reference.

Structural-variant wall time

The released version on the paper’s instance types (r8gd.8xlarge, r8id.8xlarge), 16 files per assembly run as 16 processes: a start-up-bound regime, excluded from every speed-up above.

Method

• Terms. A variant is one VCF record. A tuple is one output row reduced to its Location, Allele, Feature, Feature_type and Consequence set. F1 pools a dataset’s chromosomes by summed counts. HGVS notation is outside every F1 above.
• Concordance. One run per dataset of a build of the released version, against Ensembl VEP 116.2’s output on the same inputs, with the same comparator, cache and reference FASTA. F1 is deterministic per binary and independent of architecture.
• Adjusted F1. The tuples it sets aside are the documented classes of docs/intended-divergences.md. Every tuple outside those classes is matched.
• fastVEP v0.4.0 was scored on the same Ensembl VEP output, with the same comparator, cache and reference and the same classes set aside. Its rows state its own tuple counts. Its adjusted figure differs from its raw one where the documented classes reach its whole-genome rows (1000 Genomes Phase 3 on GRCh37; 1000 Genomes high-coverage on GRCh38; ClinVar on GRCh37; ClinVar on GRCh38; All six on both assemblies). It was scored on the ClinVar cells and not on the structural-variant cells, whose tables carry no fastVEP v0.4.0 row.
• Previous release. vep-rs v0.3.2’s rows are its re-scoring against the same Ensembl VEP 116.2 reference on the cells this release’s record pairs (previous_release_concordance), measured the same day under the same protocol; tuple counts move against v0.3.2 on ClinVar GRCh37, ClinVar GRCh38, SV per-VCF (16 files) GRCh37, SV per-VCF (16 files) GRCh38; its published figures were scored against an earlier reference and are not compared across references.
• Wall time. The median of 20 independent machine measurements per cell on both architectures, with --fork 16 on sites-only inputs. Each measurement runs on a fresh instance after a discarded warmup, whose output is flushed and deleted before the timed run.

Provenance records

Every value above is recorded in these files:

• docs/concordance-provenance/2026-10-16-release-v0.4.0.json: the ClinVar and structural-variant cells, the per-column field agreement, the compiler and build inputs of the measured binaries, the reference sets, the same-day comparison against v0.3.2, and the whole-genome concordance per chromosome.
• docs/concordance-provenance/2026-10-16-population-v0.4.0.json: every chromosome’s wall-time median, P5 and P95 with the 20 wall times behind them, for both engines, and fastVEP’s concordance per chromosome.
• docs/concordance-provenance/2026-10-15-comparator-fastvep-v0.4.0.json: fastVEP v0.4.0’s ClinVar figures, scored on the same Ensembl VEP output with the same comparator, cache and reference, the same classes set aside.
• docs/concordance-provenance/2026-09-20-n20-parallel-clones.json: the comparators’ ClinVar wall times, carried forward and labelled with the engine version they were measured at.
• docs/concordance-provenance/2026-10-06-release-v0.3.2.json: vep-rs v0.3.2’s own record, with the figures it published.
• docs/concordance-provenance/2026-10-06-population-v0.3.2.json: vep-rs v0.3.2’s whole-genome measurement.

Reproducing the figures

• scripts/concordance/run_clone_measurement.sh re-measures the ClinVar and structural-variant cells on your own machines.
• scripts/concordance/run_concordance.sh times both engines on a directory of your own VCFs.
• scripts/README.md is the runbook for reproducing the concordance numbers.

Installation

Pre-built binaries

Binaries for Linux x86_64, Linux aarch64 and macOS aarch64 are on the Releases page. The x86_64 binary needs AVX2; see CPU floor.

Build from source

Building needs Rust 1.88+ (rustup):

git clone https://github.com/natera-open-source/vep-rs.git
cd vep-rs
cargo build --release
# Binary at target/release/vep

CPU floor

.cargo/config.toml compiles every build for a fixed CPU baseline, and release binaries are built the same way:

On an older CPU, a binary built or downloaded this way aborts with an illegal-instruction fault rather than running slowly. To build for the machine you are on, override the flag for that build:

RUSTFLAGS="-C target-cpu=native" cargo build --release

RUSTFLAGS replaces the file’s setting; it does not add to it. You can also edit the rustflags line for your target in .cargo/config.toml.

Container image

Each published release from 0.2.0 on is also an image on the GitHub Container Registry. It holds the release’s own x86_64 Linux binaries, vep-cache-builder, vep-cache-converter, and the duckdb CLI that Parquet output needs.

docker run --rm --user "$(id -u):$(id -g)" \
  -v "$PWD/tests/golden/116/GRCh37:/corpus:ro" -v "$PWD:/data" \
  ghcr.io/natera-open-source/vep-rs:0.4.0 \
  vep --json_cache /corpus/json_cache --assembly GRCh37 -i /corpus/variants.vcf -o /data/output.txt

• Tags are X.Y.Z, X.Y (the newest patch of that line) and latest.

• The image is linux/amd64 only and has the same x86-64-v3 CPU floor as the release binary.

• On Apple silicon, run it with --platform linux/amd64 under Rosetta. That works, but it is a translated binary, so don’t use it for timing.

• Each image carries a build-provenance attestation:

  gh attestation verify oci://ghcr.io/natera-open-source/vep-rs:0.4.0 --repo natera-open-source/vep-rs
  

Quick start

1. Try the bundled corpus

This needs no download. It runs the bundled 1,008-record excerpt of the benchmark datasets against the pruned GRCh37 cache that ships in tests/golden/:

target/release/vep \
  --json_cache tests/golden/116/GRCh37/json_cache \
  --assembly GRCh37 \
  -i tests/golden/116/GRCh37/variants.vcf -o output.txt

2. Get a transcript cache

For real data you need a JSON transcript cache. Build one natively with vep-cache-builder (GRCh38 shown; docs/cache-setup.md explains each input and gives the GRCh37 command):

# Ensembl release 116, GRCh38
wget https://ftp.ensembl.org/pub/release-116/gtf/homo_sapiens/Homo_sapiens.GRCh38.116.gtf.gz
wget https://ftp.ensembl.org/pub/release-116/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz
gunzip Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz
samtools faidx Homo_sapiens.GRCh38.dna.primary_assembly.fa

cargo build --release -p vep-cache-builder
target/release/vep-cache-builder \
  --species homo_sapiens --assembly GRCh38 --release 116 \
  --gtf Homo_sapiens.GRCh38.116.gtf.gz \
  --genome-fasta Homo_sapiens.GRCh38.dna.primary_assembly.fa \
  --output-dir $HOME/.vep/json_cache/homo_sapiens/116_GRCh38

Or convert an existing Perl VEP cache with scripts/data/storable_to_json.pl (scripts/README.md).

3. Annotate

Run the same VEP command you run today, with --json_cache naming the cache directory. That is the directory holding info.json and transcripts/, so 116_GRCh38 itself, not its parent:

target/release/vep \
  --json_cache $HOME/.vep/json_cache/homo_sapiens/116_GRCh38 \
  --assembly GRCh38 \
  -i input.vcf -o output.txt

Usage

# With annotation plugins, on data staged by scripts/data/setup_plugin_data.sh
target/release/vep \
  --json_cache $HOME/.vep/json_cache/homo_sapiens/116_GRCh38 \
  --assembly GRCh38 \
  --plugin CADD,snv=$HOME/.vep/plugin_data/grch38/full/cadd/whole_genome_SNVs.tsv.gz \
  --plugin REVEL,file=$HOME/.vep/plugin_data/grch38/full/revel/revel.tsv.gz \
  -i input.vcf -o output.txt --force_overwrite

# JSON output with HGVS notation
target/release/vep \
  --json_cache $HOME/.vep/json_cache/homo_sapiens/116_GRCh38 \
  --assembly GRCh38 --json \
  --fasta Homo_sapiens.GRCh38.dna.primary_assembly.fa \
  --hgvs --everything \
  -i input.vcf -o output.json

• Output formats. VEP default (the tab-delimited text format with the Extra column), --vcf (annotations in the CSQ INFO field), --json (one object per variant) and --tab match Perl VEP’s layouts. --parquet is a vep-rs extension written through the duckdb CLI (1.2 or newer on PATH). Field-level specs are in docs/output-formats.md.
• Plugins. --plugin <Name>,<params> resolves one of the eleven built-in plugins first (ten with supported data), and otherwise loads a C-ABI dynamic library from --dir_plugins. Support is per reference genome: --assembly selects the right column and refuses a data file whose own assembly contradicts it. The support matrix, the data files and how to write a plugin are in docs/plugins.md.
• HGVS. --fasta takes an uncompressed, faidx-indexed genome and gives --hgvs the reference context for the 3’ shifting of indels.
• All flags. docs/cli-reference.md is the complete flag reference, including the flags accepted for Perl VEP compatibility but not implemented.

Documentation

Crates

One row per crate under crates/, in directory order; each links the crate’s own README.

Contributing

Contributions are welcome. See CONTRIBUTING.md for build, test and pull-request guidelines, and CODE_OF_CONDUCT.md for community expectations. To report a security vulnerability, follow SECURITY.md rather than filing a public issue.

Authors

vep-rs was created and is maintained by:

• Matthew Porter, Natera, Inc.
• Robert Borkowski, Natera, Inc.

Citing vep-rs

If you use vep-rs in published work, cite the paper:

Porter M, Borkowski R. vep-rs: high-throughput Rust variant annotation with population-scale concordance to Ensembl VEP. bioRxiv (2026). doi:10.64898/2026.09.22.753614.

The journal DOI will be added here on acceptance. The paper’s figures are in docs/published-figures.md and re-derive from manuscript/data/*.csv in this tree (manuscript/data/README.md). The current release’s own measurements are in Current release.

The software itself is archived at Zenodo, and that archive is the citation for the code and the released data files:

Porter M, Borkowski R. vep-rs, version 0.1.0. Zenodo (2026). doi:10.5281/zenodo.22837897

CITATION.cff at the repository root carries the software citation in machine-readable form, with the all-versions DOI (10.5281/zenodo.22837896) and the archived version’s own. GitHub’s “Cite this repository” button and citation managers pick it up directly. The article is added there on acceptance.

Acknowledgments

vep-rs is an independent Rust implementation of the variant effect prediction algorithm. It is not affiliated with or endorsed by EMBL-EBI or the Ensembl project.

The algorithm is described in:

McLaren W, Gil L, Hunt SE, et al. The Ensembl Variant Effect Predictor. Genome Biology 17, 122 (2016). doi:10.1186/s13059-016-0974-4

License

Apache-2.0. See LICENSE.

Disclaimer

vep-rs is distributed under the Apache License 2.0 on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using vep-rs and assume any risks associated with that use. See sections 7 and 8 of the LICENSE for the complete disclaimer of warranty and limitation of liability.

vep-rs is released for research and informational use. It has not been validated, cleared, or approved by any regulatory authority for clinical use, and its output must not be relied upon for patient care. Any clinical or diagnostic application of this software is the sole responsibility of the party undertaking it.