Public Results 2022
Select Results Here
Phylogeny-Based Definition Benchmarks
- Species Tree Discordance Benchmark
- Generalized Species Tree Discordance Benchmark
Reference Orthology Based Benchmarks
- Agreement with Reference Gene Phylogenies: SwissTree
- Agreement with Reference Gene Phylogenies: TreeFam-A
- Agreement with Reference Orthologs: VGNC
Function-Based Definition Benchmarks
- Gene Ontology conservation benchmark
- Enzyme Classification (EC) conservation benchmark
Summary table
Uploaded data
This benchmark samples orthologs following a comb species tree from eukaryota species, infers gene trees from them and compare them with the expected species tree. Details on this benchmark are available in Altenhoff and Dessimoz, 2019 Phylogenetic and Functional Assessment of Orthologs Inference Projects and Methods. PLoS Computational Biology, 5(1): e1000262.
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | Number of predicted pairwise ortholog relations | Number of completed tree samples (out of 50k trials) | Avg Robinson-Foulds distance (genetree, exp. speciestree) | Fraction of incorrect trees |
| BBH_(SW_alignments) | 246288 | 9912 +- 141 | 0.0464 +- 0.0029 | 0.1007 +- 0.0059 |
| Domainoid+ | 381087 | 13464 +- 134 | 0.0452 +- 0.0024 | 0.1024 +- 0.0051 |
| Hieranoid_2 | 290730 | 10296 +- 141 | 0.0340 +- 0.0025 | 0.0736 +- 0.0050 |
| OMA_Groups | 209443 | 7745 +- 136 | 0.0351 +- 0.0030 | 0.0727 +- 0.0058 |
| OMA_HOGs | 323400 | 7395 +- 135 | 0.0407 +- 0.0032 | 0.0863 +- 0.0064 |
| OMA_Pairs | 331885 | 10339 +- 141 | 0.0396 +- 0.0027 | 0.0851 +- 0.0054 |
This benchmark samples orthologs following a comb species tree from fungi species, infers gene trees from them and compare them with the expected species tree. Details on this benchmark are available in Altenhoff and Dessimoz, 2019 Phylogenetic and Functional Assessment of Orthologs Inference Projects and Methods. PLoS Computational Biology, 5(1): e1000262.
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | Number of predicted pairwise ortholog relations | Number of completed tree samples (out of 50k trials) | Fraction of incorrect trees | Avg Robinson-Foulds distance (genetree, exp. speciestree) |
| BBH_(SW_alignments) | 99661 | 9399 +- 166 | 0.6602 +- 0.0096 | 0.3247 +- 0.0055 |
| Domainoid+ | 119096 | 9982 +- 170 | 0.6629 +- 0.0093 | 0.3354 +- 0.0055 |
| Hieranoid_2 | 101064 | 8627 +- 161 | 0.633 +- 0.010 | 0.3109 +- 0.0058 |
| OMA_Groups | 60519 | 5115 +- 131 | 0.576 +- 0.014 | 0.2849 +- 0.0076 |
| OMA_HOGs | 80356 | 5716 +- 137 | 0.603 +- 0.013 | 0.2969 +- 0.0072 |
| OMA_Pairs | 85552 | 6685 +- 146 | 0.616 +- 0.012 | 0.3043 +- 0.0066 |
This benchmark samples orthologs following a comb species tree from bacteria species, infers gene trees from them and compare them with the expected species tree. Details on this benchmark are available in Altenhoff and Dessimoz, 2019 Phylogenetic and Functional Assessment of Orthologs Inference Projects and Methods. PLoS Computational Biology, 5(1): e1000262.
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | Number of completed tree samples (out of 50k trials) | Number of predicted pairwise ortholog relations | Avg Robinson-Foulds distance (genetree, exp. speciestree) | Fraction of incorrect trees |
| BBH_(SW_alignments) | 985 +- 58 | 14026 | 0.569 +- 0.018 | 0.914 +- 0.018 |
| Domainoid+ | 1220 +- 64 | 20812 | 0.593 +- 0.016 | 0.933 +- 0.014 |
| Hieranoid_2 | 1028 +- 60 | 16730 | 0.571 +- 0.018 | 0.911 +- 0.017 |
| OMA_Groups | 577 +- 46 | 10717 | 0.516 +- 0.024 | 0.875 +- 0.027 |
| OMA_HOGs | 1126 +- 62 | 30557 | 0.584 +- 0.018 | 0.909 +- 0.017 |
| OMA_Pairs | 871 +- 55 | 20025 | 0.551 +- 0.020 | 0.898 +- 0.020 |
This benchmark samples orthologs from randomly sub-sampled species trees from the luca clade, infers a gene tree from each sampled group and compares them with the expected species tree. Details on this benchmark are available in Altenhoff A et al., 2020 The Quest for Orthologs benchmark service and consensus calls in 2020. Nucl. Acids Res., 48:W1, W538–W545.
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | Number of predicted pairwise ortholog relations | Number of completed tree samples (out of 50k trials) | Fraction of incorrect trees | Avg Robinson-Foulds distance (genetree, exp. speciestree) |
| BBH_(SW_alignments) | 5154632 | 1209 +- 67 | 0.864 +- 0.019 | 0.264 +- 0.010 |
| Domainoid+ | 12375494 | 1921 +- 84 | 0.863 +- 0.015 | 0.2703 +- 0.0082 |
| Hieranoid_2 | 8111432 | 832 +- 56 | 0.843 +- 0.025 | 0.256 +- 0.012 |
| OMA_Groups | 3192599 | 202 +- 28 | 0.876 +- 0.046 | 0.248 +- 0.023 |
| OMA_HOGs | 50934088 | 438 +- 41 | 0.847 +- 0.034 | 0.253 +- 0.017 |
| OMA_Pairs | 7482099 | 550 +- 46 | 0.860 +- 0.029 | 0.255 +- 0.014 |
This benchmark samples orthologs from randomly sub-sampled species trees from the eukaryota clade, infers a gene tree from each sampled group and compares them with the expected species tree. Details on this benchmark are available in Altenhoff A et al., 2020 The Quest for Orthologs benchmark service and consensus calls in 2020. Nucl. Acids Res., 48:W1, W538–W545.
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | Number of completed tree samples (out of 50k trials) | Number of predicted pairwise ortholog relations | Fraction of incorrect trees | Avg Robinson-Foulds distance (genetree, exp. speciestree) |
| BBH_(SW_alignments) | 3646 +- 114 | 4251345 | 0.814 +- 0.014 | 0.2322 +- 0.0062 |
| Domainoid+ | 5162 +- 133 | 10196468 | 0.819 +- 0.014 | 0.2368 +- 0.0063 |
| Hieranoid_2 | 3643 +- 114 | 7004479 | 0.800 +- 0.014 | 0.2220 +- 0.0061 |
| OMA_Groups | 1063 +- 63 | 2772338 | 0.766 +- 0.026 | 0.205 +- 0.010 |
| OMA_HOGs | 2981 +- 104 | 50074914 | 0.820 +- 0.014 | 0.2367 +- 0.0062 |
| OMA_Pairs | 2369 +- 93 | 6593688 | 0.802 +- 0.016 | 0.2256 +- 0.0070 |
This benchmark samples orthologs from randomly sub-sampled species trees from the vertebrata clade, infers a gene tree from each sampled group and compares them with the expected species tree. Details on this benchmark are available in Altenhoff A et al., 2020 The Quest for Orthologs benchmark service and consensus calls in 2020. Nucl. Acids Res., 48:W1, W538–W545.
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | Number of predicted pairwise ortholog relations | Number of completed tree samples (out of 50k trials) | Fraction of incorrect trees | Avg Robinson-Foulds distance (genetree, exp. speciestree) |
| BBH_(SW_alignments) | 994403 | 22467 +- 218 | 0.912 +- 0.010 | 0.2409 +- 0.0058 |
| Domainoid+ | 1530986 | 26048 +- 219 | 0.915 +- 0.010 | 0.2443 +- 0.0059 |
| Hieranoid_2 | 1194696 | 19828 +- 214 | 0.913 +- 0.010 | 0.2332 +- 0.0054 |
| OMA_Groups | 894457 | 14826 +- 200 | 0.899 +- 0.011 | 0.2278 +- 0.0054 |
| OMA_HOGs | 1604622 | 18472 +- 212 | 0.906 +- 0.010 | 0.2315 +- 0.0054 |
| OMA_Pairs | 1608461 | 19658 +- 214 | 0.912 +- 0.010 | 0.2355 +- 0.0058 |
This benchmark samples orthologs from randomly sub-sampled species trees from the fungi clade, infers a gene tree from each sampled group and compares them with the expected species tree. Details on this benchmark are available in Altenhoff A et al., 2020 The Quest for Orthologs benchmark service and consensus calls in 2020. Nucl. Acids Res., 48:W1, W538–W545.
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | Number of predicted pairwise ortholog relations | Number of completed tree samples (out of 50k trials) | Fraction of incorrect trees | Avg Robinson-Foulds distance (genetree, exp. speciestree) |
| BBH_(SW_alignments) | 231387 | 11268 +- 183 | 0.680 +- 0.017 | 0.2697 +- 0.0086 |
| Domainoid+ | 282416 | 12319 +- 189 | 0.685 +- 0.017 | 0.2752 +- 0.0087 |
| Hieranoid_2 | 237028 | 10157 +- 176 | 0.668 +- 0.017 | 0.2619 +- 0.0085 |
| OMA_Groups | 139886 | 3941 +- 118 | 0.616 +- 0.017 | 0.2271 +- 0.0079 |
| OMA_HOGs | 192817 | 6556 +- 148 | 0.630 +- 0.017 | 0.2401 +- 0.0082 |
| OMA_Pairs | 202016 | 6581 +- 148 | 0.647 +- 0.017 | 0.2467 +- 0.0083 |
This benchmark compares the predicted orthologs with the orthologs from a set of manually curated reference gene trees. Details on this benchmark are available in Brigitte Boeckmann, Marc Robinson-Rechavi, Ioannis Xenarios, Christophe Dessimoz, 2011 Conceptual Framework and Pilot Study to Benchmark Phylogenomic Databases Based on Reference Gene Trees. Briefings in Bioinformatics, 12:5 (pp. 474-484).
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | True positive rate | Positive predictive value rate |
| BBH_(SW_alignments) | 0.570 +- 0.037 | 0.932 +- 0.036 |
| Domainoid+ | 0.759 +- 0.035 | 0.932 +- 0.030 |
| Hieranoid_2 | 0.726 +- 0.036 | 0.950 +- 0.030 |
| OMA_Groups | 0.496 +- 0.038 | 0.940 +- 0.036 |
| OMA_HOGs | 0.696 +- 0.035 | 0.924 +- 0.030 |
| OMA_Pairs | 0.677 +- 0.035 | 0.929 +- 0.031 |
This benchmark compares the predicted orthologs with the orthologs implied from the reconciled gene trees from tree fam. Details on this benchmark are available in Heng Li et. al, 2006 TreeFam: a curated database of phylogenetic trees of animal gene families. Nucl. Acids Res., 34(suppl 1): D572-D580.
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | True positive rate | Positive predictive value rate |
| BBH_(SW_alignments) | 0.4828 +- 0.0066 | 0.9560 +- 0.0038 |
| Domainoid+ | 0.6715 +- 0.0062 | 0.9231 +- 0.0041 |
| Hieranoid_2 | 0.6206 +- 0.0064 | 0.9300 +- 0.0041 |
| OMA_Groups | 0.3828 +- 0.0064 | 0.9535 +- 0.0044 |
| OMA_HOGs | 0.5799 +- 0.0065 | 0.9231 +- 0.0044 |
| OMA_Pairs | 0.5650 +- 0.0066 | 0.9199 +- 0.0046 |
This benchmark compares the predicted orthologs with some manually curated and asserted orthologs from the VGNC consortium. Details on this benchmark are available in Tweedie S et al., 2021 Genenames.org: the HGNC and VGNC resources in 2021. Nucl. Acids Res., 49:D1, D939-D946.
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | True positive rate | Positive predictive value rate |
| BBH_(SW_alignments) | 0.9790 +- 0.0020 | 0.99974 +- 0.00023 |
| Domainoid+ | 0.9856 +- 0.0017 | 0.99535 +- 0.00096 |
| Hieranoid_2 | 0.9532 +- 0.0030 | 0.99780 +- 0.00067 |
| OMA_Groups | 0.9447 +- 0.0032 | 0.99919 +- 0.00041 |
| OMA_HOGs | 0.9709 +- 0.0024 | 0.9790 +- 0.0020 |
| OMA_Pairs | 0.9716 +- 0.0023 | 0.9801 +- 0.0020 |
This benchmark computes how well the gene ontology annotations are conserved among the predicted orthologs. Details on this benchmark are available in Altenhoff and Dessimoz, 2019 Phylogenetic and Functional Assessment of Orthologs Inference Projects and Methods. PLoS Computational Biology, 5(1): e1000262.
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | Number of predicted pairwise ortholog relations | Avg Schlicker similarity |
| BBH_(SW_alignments) | 88910 | 0.5115 +- 0.0015 |
| Domainoid+ | 169998 | 0.4851 +- 0.0011 |
| Hieranoid_2 | 126270 | 0.4938 +- 0.0013 |
| OMA_Groups | 62952 | 0.5167 +- 0.0018 |
| OMA_HOGs | 362533 | 0.41686 +- 0.00072 |
| OMA_Pairs | 110960 | 0.4964 +- 0.0014 |
This benchmark computes how well the Enzyme Commission numbers are conserved among the predicted orthologs. Details on this benchmark are available in Altenhoff and Dessimoz, 2019 Phylogenetic and Functional Assessment of Orthologs Inference Projects and Methods. PLoS Computational Biology, 5(1): e1000262.
This is an interactive plot of the results. You can enable/disable methods by clicking on the label in the legend, or hover over the data points to get extra information. The measures we use for precession and recall are explained in the documentation section.
| Project | Number of predicted pairwise ortholog relations | Avg Schlicker similarity |
| BBH_(SW_alignments) | 148769 | 0.95149 +- 0.00076 |
| Domainoid+ | 220046 | 0.91590 +- 0.00082 |
| Hieranoid_2 | 169100 | 0.94776 +- 0.00073 |
| OMA_Groups | 99657 | 0.97144 +- 0.00072 |
| OMA_HOGs | 308308 | 0.82920 +- 0.00092 |
| OMA_Pairs | 156000 | 0.95220 +- 0.00073 |