Help Login Create account

Data released on August 27, 2015

Supporting data for "A close look at protein function prediction evaluation protocols".

Ben-Hur, A; Funk, C; Kahanda, I; Ullah, F; Verspoor, K (2015): Supporting data for "A close look at protein function prediction evaluation protocols". GigaScience Database. http://dx.doi.org/10.5524/100153 RIS BibTeX Text

The recently held Critical Assessment of Functional Annotation challenge (CAFA2) required its participants to submit predictions for a large number of target proteins regardless of whether they have previous annotations or not. This is in contrast to the original CAFA challenge in which participants were asked to submit predictions for proteins with no existing annotations. The CAFA2 task is more realistic, in that it more closely mimics the accumulation of annotations over time. In this study we compare these tasks in terms of their difficulty, and determine if cross-validation provides a good estimate of performance. The CAFA2 task is a combination of two sub-tasks: making predictions on annotated proteins and making predictions on previously unannotated proteins. In this study we analyze the performance of several function prediction methods in these two scenarios. Our results show that several methods (GOstruct, binary SVMs, and guilt by association) find it hard to achieve the same level of accuracy on these two tasks compared to cross-validation, and that predicting novel annotations for previously annotated proteins is a harder problem than predicting annotations for uncharacterized proteins. We also find that different methods have different performance characteristics in these tasks, and that cross-validation is not adequate at estimating performance and ranking methods.

Contact Submitter

Related manuscripts:

doi:10.1186/s13742-015-0082-5

Additional information:

https://zenodo.org/record/20553

http://sourceforge.net/projects/strut/

http://pyml.sourceforge.net/

http://bionlp.sourceforge.net/nlp-pipelines/

Keywords:

Genomics Compressed sensing Nonlinear interactions 

Proteomic, Software

http://gigadb.org/images/data/cropped/100153.jpg

Files: (FTP site) Table Settings

Columns:

File Description
Sample ID
File Type
File Format
Size
Release Date
Download Link
File Attributes

File NameSample IDFile TypeFile FormatSizeRelease Date 
OtherTAR959.92 MB2015-08-11
ReadmeTEXT0.89 KB2015-08-11
Displaying 1-2 of 2 File(s).

History:

+

Other datasets you might like: