Oxford Medical

 Grading evidence

 In 2001 a group known as SIGN (Scottish Intercollegiate Guidelines Network) created a grading system that was linked to the strength of the evidence including its study design. Following this, GRADE (Grades of Recommended Assessment, Development and Evaluation) working group updated the SIGN grade to look at two important factors:

 Quality and evidence

 Strength of recommendation of the research

 How do you determine the quality of evidence?

This forms the major part of critically appraising a research paper where the research is analysed in the following categories:

1. Hierarchy of study design
2. Study quality
3. Consistency
4. Directness
5. Recommendations

    

   Hierarchy of study design

    The type of study design is important when critically appraising it. There are various grades given to the type of study design which include the following categories:

   1. Meta analysis
   2. Randomised controlled trial
   3. Non-randomised controlled trial
   4. Observational studies/case reports
   5. Expert opinion or peer opinion

   
   Meta analysis is given the very highest grade of study design with expert opinion allocated the very lowest. There are alternative methods for classifying study design which are given the numbers 1 to 5. They may also be subdivided into ‘a’ and ‘b’ for each but this confuses the overall picture and details have not been included.Peer opinion is essentially expert opinion. This is the lowest form of evidence available and is when a group of experts believe a particular intervention has benefits although cannot back this up with hard evidence.Observational studies and case reports are individual publications documenting how single patient episodes have changed and why this may have been. Details on the other three types of study hierarchy are mentioned later in this guide

   ---

   Study quality

   
   Here the study is assessed looking for the presence of randomisation, interpreter blinding, whether the appropriate statistics have been used, and whether the conclusions are supported.

    

   Consistency

   
   Normally there are several published papers looking at the same clinical problem. What is the consistency in the result across the studies? If nine studies have shown no beneficial outcome as a result of an intervention but the tenth shows the opposite, then this study is inconsistent with the previous studies and therefore should be taken cautiously.
   
   

   Directness

   
   The result of research needs to be put in the context of patient outcome. One of the main questions we need to ask ourselves is how does this new intervention benefit patients? Quite clearly if a new intervention reduces mortality by 10% then it is highly significant. However, if this figure was 0.1%, would you change your practice based on this evidence? Probably not.

   ---

   Recommendations

   
   Broadly speaking, the recommendations from a research paper as to a new drug treatment or intervention fall into the following categories:
   
   

   1.  We should be using it
   2. We probably should be using it
   3. We probably should not be using it
   4. We should not use it

    

   Whilst the first and fourth of these recommendations are quite straightforward, it is the grey area that is the second and third recommendation. Unfortunately, most research is not clear-cut and therefore falls into the ‘probably’ category. It is therefore up for individual appraisal by the clinician as to whether his practice should incorporate the results of this research.

   ---

    

   What different types of studies are there?

    

   • Non randomised controlled trial

   
   This is a trial that contains no randomisation of patients and may or may not involve blinding (either single or double).

    

   • Randomised controlled trial

    

   This is essentially the gold standard to study design. Patients and the operator are randomised and blinded in order to test the null hypothesis.
   
   

   • Meta analysis

    

   It is possible that using a randomised controlled trial may not find the answer to a research question solely based on the lack of high enough patient numbers involved in the study. Sometimes the primary measured outcome may change by a very small amount as a result of drug therapy or intervention. Therefore, in order to statistically prove that a change has or has not occurred large numbers of patients may be required in the study. Meta analysis solves this problem by pooling all available published research looking at the same question. By using this approach the numbers of patients may be sufficient for the research question to be answered. As a note there are some sceptics who consider meta analysis to be a statistical fudge. This is because not every study will have used the same methodology and therefore results shouldn’t be taken for granted; however this still remains at the top of the hierarchy of study design.

   ---

    What are the problems with evidence-based medicine?

    

   • Criticisms of evidence-based medicine

   
   There are some concerns such as publication bias about evidence-based medicine. Research that produces negative results may not be published. Research may have been carried out in the correct manner and the correct conclusion drawn but journals have a tendency to publish new and positive studies. It is also impossible to practise evidence-based medicine with every clinical decision that we have to make. There is also the possibility of ‘cookbook medicine’. This is where guidelines and recommendations tell us what we can and can’t do and this has a habit of suppressing clinical freedom and expertise on an individual patient by patient basis.

   ---

    What examples of successful evidence-based medicine do you know?

    

   Examples of successful evidence based medicine include:

    

   1. The use of early thrombolytic therapy in ST segment elevation MI’s
   2. The use of ultrasound guided central venous cannulation

    

   Evidence-based medicine has advantages and disadvantages. It is essential that every clinician is able to critically appraise a research paper so that we maintain clinical standards. Evidence-based medicine is probably the correct path to take in an evolving medical practice.

    

    

   Oxford Medical runs weekly Oxford and London based Consultant Interview Courses. Small group sizes, individually focused to give you the best possible training.Optional video recorded performance analysis and free post course training including over 3 hours of video hot topic tutorials. This is why we are considered the best interview skills training company. Don't forget to prepare for the University Representative who may ask about teaching and medical education. Do you need to improve your medical education section of your CV? Have a look at our Teach the Teacher Courses. We can even provide management training on one of our medical management courses

    

    

   Make sure that you have read a recent journal which is related to your speciality. Select a randomised controlled trial which has good study design and good study quality. Appraise the paper and have facts to hand. You will then be in a position to answer the questions “Tell me about a paper that you have read recently” or “Tell me about a paper that has changed your practice”.