p value less than 0.05

I received an email about a Research Integrity conference and checked out the keynote speakers. One of them was Dr. John Ioannidis. This lead to me to what I discovered was one of the most cited research papers out there "Why Most Published Research Findings are False". This article touched off a nerve or two in me, and eventually I will try and read it to at least a level of comprehension, because it is very mathematical. What struck me was the hypothesis that:

Several methodologists have pointed out [9–11] that the high rate of nonreplication (lack of confirmation) of research discoveries is a consequence of the convenient, yet ill-founded strategy of claiming conclusive research findings solely on the basis of a single study assessed by formal statistical significance, typically for a p-value less than 0.05.

I took courses in health research methodology and was taught how to read medical literature and the p value inherent in most of the journal articles always confused me.  I still don't know the significance of the p value, but this I know: it is not good if family doctors, relying on evidence- based medicine to prescribe innovative therapeutic drugs, are relying on these articles' conclusions and p values for their predictive value to help me. They should be relying on the gold standards of medical evidence: systematic reviews and meta-analysis - the highest forms of "unbiased" research. Atlantic magazine has a great article "Lies, Damn Lies, and Medical Science" (in plain English) about Ioannidis and this medical dilemma.

If you want to check out why I might be confused by what a p value is, check out this definition in wikipedia:

In statistical hypothesis testing, the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.^[1] One often "rejects the null hypothesis" when the p-value is less than the significance level α (Greek alpha), which is often 0.05 or 0.01.

Although there is often confusion, the p-value is not the probability of the null hypothesis being true, nor is the p-value the same as the Type I error rate.^[2] A Type I error in statistics is the incorrect rejection of the null hypothesis. In this case the hypothesis was correct but wrongly rejected. In a Type II error, however, the null hypothesis was not rejected despite being incorrect. This results in the failure of rejection of incorrect assumptions.

The best place to learn about all of this is in one of the classics of evidence-based medicine by one of the authors who coined the term "evidence-based medicine", Dr. Gordon Guyatt, who teaches and does research at McMaster University:

Users' Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice, Second Edition (Jama & Archives Journals) by Gordon Guyatt, Drummond Rennie, Maureen Meade and Deborah Cook (May 21, 2008)