Featured Post

Hacking Health in Hamilton Ontario - Let's hear that pitch!

What compelled me to register for a weekend Health Hackathon? Anyway, I could soon be up to my ears in it. A pubmed search on Health Hack...

Showing posts with label Evidence-Based Medicine. Show all posts
Showing posts with label Evidence-Based Medicine. Show all posts

Wednesday, December 19, 2012

p value less than 0.05



I received an email about a Research Integrity conference and checked out the keynote speakers. One of them was Dr. John Ioannidis. This lead to me to what I discovered was one of the most cited research papers out there "Why Most Published Research Findings are False". This article touched off a nerve or two in me, and eventually I will try and read it to at least a level of comprehension, because it is very mathematical. What struck me was the hypothesis that:

Several methodologists have pointed out [9–11] that the high rate of nonreplication (lack of confirmation) of research discoveries is a consequence of the convenient, yet ill-founded strategy of claiming conclusive research findings solely on the basis of a single study assessed by formal statistical significance, typically for a p-value less than 0.05.

I took courses in health research methodology and was taught how to read medical literature and the p value inherent in most of the journal articles always confused me.  I still don't know the significance of the p value, but this I know: it is not good if family doctors, relying on evidence- based medicine to prescribe innovative therapeutic drugs, are relying on these articles' conclusions and p values for their predictive value to help me. They should be relying on the gold standards of medical evidence: systematic reviews and meta-analysis - the highest forms of "unbiased" research. Atlantic magazine has a great article "Lies, Damn Lies, and Medical Science" (in plain English) about Ioannidis and this medical dilemma.

If you want to check out why I might be confused by what a p value is, check out this definition in wikipedia:


In statistical hypothesis testing, the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.[1] One often "rejects the null hypothesis" when the p-value is less than the significance level α (Greek alpha), which is often 0.05 or 0.01.
Although there is often confusion, the p-value is not the probability of the null hypothesis being true, nor is the p-value the same as the Type I error rate.[2] A Type I error in statistics is the incorrect rejection of the null hypothesis. In this case the hypothesis was correct but wrongly rejected. In a Type II error, however, the null hypothesis was not rejected despite being incorrect. This results in the failure of rejection of incorrect assumptions.


The best place to learn about all of this is in one of the classics of evidence-based medicine by one of the authors who coined the term "evidence-based medicine", Dr. Gordon Guyatt, who teaches and does research at McMaster University:

Users' Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice, Second Edition (Jama & Archives Journals) by Gordon Guyatt, Drummond Rennie, Maureen Meade and Deborah Cook (May 21, 2008)



Thursday, December 13, 2012

Another McMaster Study about Health IT: The Renaissance Version!

So, I enjoy reading posts on the Kevin Pho MD newsletter, and I am not even an MD. The ehealth stories are often very interesting, like the one I am posting here. Now this article has 3 main characters:

1. The article by James Salwitz entitled "Why IT is the core of the healthcare renaissance".
2. An article mentioned by Dr. Salwitz by Stephen Soumeri and Ross Koppel on the online Wall Street Journal called "A Major Glitch for Digitized Health-care Reform".
3. A paper by McMaster University researchers entitled "The economics of health information technology in medication management: a systematic review of economic evaluations".

Please draw your own conclusions! My conclusion would be that the McMaster researchers uncovered a need for better economic assessments of healthcare technology, in order to really make a realistic appraisal. And I agree with Dr. Salwitz that looking back 5 decades is too long.

And this isn't the only research that has cast aspersions on the benefits of eHealth technology. A study on PLOS last year, that was almost polemical in tone, blew the doors off of that "The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview"
"http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.1000387



Why IT is the core of the healthcare renaissance


Why IT is the core of the healthcare renaissance
Warning!  I am a practicing doctor who sees real patients using an electronic medical record (EMR).  My sole agenda is to provide the best patient care.  I have no financial stake in information technology (IT).  However, unlike the editorial board at the Wall Street Journal, Mr. Stephen Soumerai of Harvard or Mr. Ross Koppel of the University of Pennsylvania, I have actually used digital patient records for over a decade and I have news for them;  EMRs work.
In a reactionary opinion in the WSJ entitled “A Major Glitch for Digitized Health-Care Records,” the authors expanded to the point of silliness the conclusions of a review of healthcare IT by McMaster University.  The McMaster analysis abstracts data from 36000 studies over five decades of healthcare IT and concludes that computerization has yet to save dollars nor improve health care.  WSJ editorialists proposed that the concept of a common medical database has “already failed” as is “common knowledge.”  While they portend to “fully share the hope” in the success of a computerized healthcare system they express doubt as to “why are we pushing ahead to digitalize.”
The question is so ridiculous as to barely require an answer.  We are pushing ahead to digitalize because the healthcare industry, which is 18% of our GDP, is the last major industry to go electronic.  Despite how critical medicine is to our citizen and nation’s vitality, health care is most often documented with paper and ink.  Can you imagine any other industry where this would be acceptable?  Would you go to a bank where they use a hand-written ledger?  Travel on an airplane without GPS, fly by wire technology or a minimum of three computers?  Do you yearn for rotary phones?  Credit cards left paper money behind decades ago and will soon move on to the next phase, pay by smart-phone. The world is digital and one of the core problems with medical care is its failure to follow.
The average doctor writes his notes on parchment and scribbles orders on contact paper.  He wastes time writing prescriptions by hand that cannot be read and will produce unneeded drug interactions and side effects.  The data on billions of health care events cannot be mined, monitored, analyzed or improved, because it is not digital.   Millions of hours are wasted, billions of dollars vanish and tens of thousands die because of preventable medical complications, the result of massive variation in quality and safety.  We are doomed because we cannot access or evaluate most medical care data; “If you can not measure it, you can not manage it.”
Taking health digital is key to fixing and affording care.  Standard, unified medical records will significantly decrease the risk of providing unneeded or dangerous medical care.  Massive efficiencies will result by reducing duplication, speeding communication and reduction in waste (and fraud).  Critical improvements will follow the use of guidelines to study clinical databases and drive quality. This means that whether one lives in Manhattan, in the mountains of Tennessee or potentially deep in Africa, the finest care will be possible.
How do I know this to be true?  Our practice of seven doctors and three nurse practitioners was an early EMR adapter.  We put in our first basic system in 2000, upgraded three times and have been fully electronic for four years.  This has resulted in marked efficiencies and obvious quality improvement. Encrypted electronic records cannot be lost, are unlikely to be stolen and are always available from anywhere.  Ordering tests is instantaneous, as is reviewing results, organizing treatments, scheduling appointments or communication with outside health providers. On the cost side we reduced non-clinical staff by more than 50% and in an account receivable analysis our billing cycle dropped more than 60% and bad debt fell to low single digits.
Today in the office, I saw four new patients.  Without assistance of clerical staff and without leaving my desk I reviewed their entire surgical, laboratory, pathological, and radiologic records.  By the time, I entered the room to meet each of them a significant part of their medical history was entered into our EMR, based on outside records, so that the care and observations of previous doctors was not forgotten. Half way through each visit documentation of their history and physical exam was complete, leaving more time to talk with each patient.  Tests were ordered, treatments scheduled, disability letters printed, medications e-scripted, instructions created and follow-up appointments setup.  The patients were given codes to electronically access their records from home.  Letters were sent online to referring doctors, as well as any clinicians we were consulting in that patient’s care.  Billing was complete before each patient got to the parking lot.  Such is the power, efficiency and quality of electronic medical records.
This is just the beginning.  Although EMRs now provide assistance with basic medical care, such as scheduling flu shots, identifying drug interactions, and health screening reminders, future systems will use academic information to assist the doctor in making diagnoses and planning treatments.   Seamless with the EMR will be computer augmentation to create differential diagnoses and recommend treatment alternatives.  In oncology alone there are almost 50,000 articles published each year; Artificial intelligence integration with the clinical EMR will help every doctor penetrate that massive database on a continuous basis as it applies to individual patients.
So, why does the McMaster study not show this obvious benefit? It comes to four factors. The first is the “five decades” of study reviewed by the Canadian authors.  Since most doctors did not start adapting EMRs until 3 years ago, that leaves 57 months of irrelevant data.  Second, we have not reached the critical mass to achieve broad system efficiencies, as hospitals and doctors are still figuring out how to incorporate the technology into their daily practice, and less than 50% of health care providers have converted to EMRs.   Third, we do not yet have a universal common database for medical records. This is a complex technological step, which has been achieved in major industries such as banking, but still must be assembled in medicine.  Finally, as was correctly stated in the WSJ op-ed piece, present EMRs are cumbersome, immature, and several generations short of perfection.  However, these are expected problems when implementing disruptive evolving technology in the complex changing health market. Challenge is a weak argument for giving up and bringing back the fountain pen.
I am not certain what the goal of Sommeri and Koppel was in writing this piece, they offer nothing but “hope.”  The WSJ has been a strong supporter of business technology leading the drive towards quality, and has long recognized the positive contribution of IT to industry.   Those of us in the trenches, putting EMRs in place, ironing out the kinks, know that electronic medical records are now and they are the future.  With IT at the core of the healthcare renaissance we can make medicine cost efficient, producing quality second to none.
James C. Salwitz is an oncologist who blogs at Sunrise Rounds.
Image credit: Shutterstock.com

Saturday, September 15, 2012

eHealth Ontario Diabetes Registry is no more

A variation of this article in whatever edition of the Toronto Star I saw, was headline news.  The story line in the current online edition says"Diabetes Registry obsolete, ehealth tells liberals". The headline I saw on the shelves in the grocery store at around 10:00am this morning was something like "eHealth Ontario Axes Diabetes Registry". The twists and turns in this story are many and varied.  I remember the eHealth Ontario procurement and spending scandal in 2009 mostly because I had just starting working towards a graduate degree in eHealth (or Health Informatics) at the time and I was beginning to feel a lot of regret about the name of my degree. The real concern here should be about people who have diabetes and the epidemic this is now becoming globally - not some provincial political hot potato, or football, or whatever metaphor pleases you.

I was studying the clinical efficacy of using personal health records to manage diabetes, looking for systematic reviews of the literature in the Cochrane Collaboration.  They were not many because they are not many clinical trials involving diabetes and electronic medical records (let alone personal health records).  Yes I looked on the clinical trials registry site as well - clinicaltrials.gov   and found a rare few clinical trials, mostly involving large hospital systems that had electronic health records in the United States. I had learned in my courses that clinical trials present the highest form of medical evidence, exept for meta-analysis or systematic reviews, which is a composite study of all the best clinical trials.

It was when I was studying the project management issues in a mobile application for managing type II diabetes, that I came upon the understanding that clinical efficacy should precede business opportunity. See the COMPETE studies at McMaster. In healthcare, the return on investment is improvement in quality of life, but why spend millions on a system because it is believe to be clinically helpful, when the evidence for the actual efficacy is not in yet.  There were experiments and studies that did show promise for this, and maybe they showed promise because they were all small studies and easy to manage.

We were aware that a smartphone or mobile solution for persons with diabetes tracking and managing their condition always seemed to involve a triage person or nurse practitioner who monitored the data flows from the patients.  York University's Department of Health is now started a course of training for Health Coach Professionals, which was part of the subject of my last blog post on a consortium of academics, business and health organizations to develop the open source OSCAR EMR and it's companion PHR call MyOSCAR.  It is this kind of triage person who I think could really help fill the gap between the patient with diabetes using the smartphone or the computer, and the personal health record or electronic medical record where data measurements and journaling provide the tools necessary to self-manage the condition.  A coach is really needed to get people to exercise, eat well, adhere to regimes of all sorts, etc.

Later in 2011 when I was researching the system architecture of personal health records systems in Ontario and their relationship to the national blueprint architecture of the Canada Health Infoway, that main benefit for using them would evolve in the prescription model that physicians currently have with their patients.  A personal health record system could be prescribed to a patient, because of the clinically proven benefits of using them to manage the condition. Again, this involves the evidence based medicine preceding the business model or expediency.  As well, the prescription model maintains the traditional trust between physician and patient, and as far as I know, diabetes is not something you want to try and handle on your own, without professional guidance and help.

When in 2010 we learned eHealth Ontario was going to create a Diabetes Registry for all Ontarians, I for one thought they really did not know what they were getting themselves into.  It is not that I am proved right, it is just that technology projects are often started for the wrong reasons, and pulling the plug on them, before trying to fix the short comings, is the right thing to do.  Wished it could have turned out better for the eHealth Ontario people, CGI, and I suppose the Liberal government, who I know have the patients in Ontario and their health in mind first.

Monday, July 30, 2012

Online Journal Articles and Social Media

http://www.mdpi.com/1660-4601/6/2/492

The academic study of Health Informatics requires a lot of researching of journal articles, mostly online versions through a university electronic subscription. The work of librarians is constantly changing because of digital technology, and journal articles are framed in the context of relevance, statistics, interrelationship.  Of course, one needs to know how to read a journal article, and in my courses at McMaster, we learned how to read journal articles along the principles of evidence based medicine.  Evidence based medicine was started at McMaster, and one of my tutorial leaders, Dr. Brian Haynes, was one of the original founders of this approach.

But if you look at the journal article link above "Emerging Patient-Driven Health Care Models: An Examination of Health Social Networks, Consumer Personalized Medicine and Quantified Self-Tracking", there are many ways to make the data in the article organizable, searchable, and ultimately understandable.  One of those is the link to Connotea, which  reminded me that I setup a Connotea account but haven't used it in several years.  There are so many ways to slice and dice a journal article as a research object, which is why it makes sense to approach research looking for the "nano-slice of the pie of science".

My favourite way of organizing journal articles was through Refworks.  There are three main reasons way Refworks really rocks:
1) you can search the Pubmed or other library catalogues and import articles quickly
2) you can create footnotes and references exported into Word documents easily
3) you can share, add, edit and delete your references with others on the academic team




Wednesday, May 30, 2012

NHS to shut down Personal Health Record Service

The UKs National Health Service is going to shut down it's personal health recored service, called Healthspace.  < Here > is the story. This quote by the clinical informatics director, Dr Charles Gutteridge, citing reasons for shutting it down, is insightful:

"It is too difficult to make an account; it is too difficult to log on; it is just too difficult," he said.

I had previously read journal articles citing this as a failure, or at least there being no real benefit to eHealth technologies like personal health records:
Greenhalgh T, Hinder S, Stramer K, Bratan T, Russell J. Adoption, non-adoption, and abandonment of a personal electronic health record: case study of HealthSpace. BMJ. 2010 Nov 16;341:c5814. doi: 10.1136/bmj.c5814




Wednesday, March 28, 2012

Open Medicine Journal

The Open Medicine Journal is one of the wonders of our times. And so is a recent article on drug policy by some of the leading Public Health officials in the country. Ideology vs. Medicine vs. Morality.

Monday, March 26, 2012

The Panic Virus

Books! I am reading the Panic Virus by Seth Mnookin. Might even see him at a conference next month where he is the key speaker at the Canadian Association of Research Ethics Board. The first 70 pages or so about vaccines and those in the public who advocate AGAiNST them, reminded me of another book I read recently by Michael Bliss on the smallpox epidemic in Montreal in the last century. The role the media plays with science stories is huge, but so is the role parents play who advocate for their kids. This book is not about ehealth or technology. It is mostly about Autism and how bad science created the impression that vaccinations were responsible for causing it.

Tuesday, February 28, 2012

What do predicting wine quality and evidence-based medicine have in common?

The answer is McMaster University and a book by Ian Ayers, called "SuperCrunchers, Why Thinking-by-Numbers Is the New Way to Be Smart" Through regression analysis and crunching the numbers, the quality of wines can be predicted just as well as by expert wine tasters - in advance of the harvest. There is a chapter on how McMaster University medicine developed the science of evidence-based medicine - also by looking at the numbers. This is the dawning of the age of Big Data, as we all should know.