Close
Help
Need Help?





JOURNAL

Cancer Informatics

1,240,591 Journal Article Views | Journal Analytics

Assessing the Statistical Significance of the Achieved Classification Error of Classifiers Constructed using Serum Peptide Profiles, and a Prescription for Random Sampling Repeated Studies for Massive High-Throughput Genomic and Proteomic Studies

Submit a Paper



Publication Date: 21 Feb 2007

Journal: Cancer Informatics

Citation: Cancer Informatics 2005:1 53-77

James Lyons-Weilera,f, Richard Pelikanb, Herbert J Zeh IIIc,f, David C Whitcombd,f, David E Malehorne,f William L Bigbeee,f Milos Hauskrechtb,f a

Department of Pathology, Cancer Biomarkers Laboratory, Center for Pathology Informatics, Benedum Oncology Informatics Center bDepartment of Computer Science cDepartment of Surgery dDepartments of Medicine, Cell Biology & Physiology, and Human Genetics eClinical Proteomics Facility fUniversity of Pittsburgh Cancer Institute University of Pittsburgh

Abstract: Peptide profiles generated using SELDI/MALDI time of flight mass spectrometry provide a promising source of patientspecific information with high potential impact on the early detection and classification of cancer and other diseases. The new profiling technology comes, however, with numerous challenges and concerns. Particularly important are concerns of reproducibility of classification results and their significance. In this work we describe a computational validation framework, called PACE (Permutation-Achieved Classification Error), that lets us assess, for a given classification model, the significance of the Achieved Classification Error (ACE) on the profile data. The framework compares the performance statistic of the classifier on true data samples and checks if these are consistent with the behavior of the classifier on the same data with randomly reassigned class labels. A statistically significant ACE increases our belief that a discriminative signal was found in the data. The advantage of PACE analysis is that it can be easily combined with any classification model and is relatively easy to interpret. PACE analysis does not protect researchers against confounding in the experimental design, or other sources of systematic or random error.We use PACE analysis to assess significance of classification results we have achieved on a number of published data sets. The results show that many of these datasets indeed possess a signal that leads to a statistically significant ACE.


Downloads

PDF  (1.93 MB PDF FORMAT)

RIS citation   (ENDNOTE, REFERENCE MANAGER, PROCITE, REFWORKS)

BibTex citation   (BIBDESK, LATEX)

XML






What Your Colleagues Say About Cancer Informatics
Cancer Informatics has become an increasingly important source for research in the methodology of cancer genomics and the novel use of informatics technology. I have been impressed by the journal's contents and have been very gratified by the number of accesses to my recent publication. Cancer Informatics has filled an important gap in cancer research journals.
Dr Richard Simon (Chief, Biometric Research Branch, National Cancer Institute, USA )
More Testimonials

Quick Links




Follow Us We make it easy to find new research papers.




SUBJECT HUBS
Author Survey Results
author_survey_results
All authors are surveyed after their articles are published. Authors are asked to rate their experience in a variety of areas, and their responses help us to monitor our performance. Presented here are their responses in some key areas. No 'poor' or 'very poor' responses were received; these are represented in the 'other' category.
See Our Results