|
|
EBP Critical Appraisal Resources
This page provides information on how
to stay up to date on evidence based practices and how to locate reviews
of evidence based practices.
Systematic Review Organizations
There are many strategies for staying "on-top" of
the treatment research literature. Usually, the question that
comes to mind next is how practitioners can most efficiently keep
abreast of the evidence-based treatments for a particular clinical
diagnosis, particularly given that few people have the time to
regularly and systematically search the academic literature. While
searching and reading the literature yourself is still the most
comprehensive method for staying up-to-date, more and more treatment
providers are relying on systematic reviews of the treatment literature
and classification of interventions by independent organizations.
Identifying the major professional organization for the clinical
diagnosis or condition you're focusing on can be a good place
to start. In addition, there are several organizations or networks
that systematically review and report on EBP for health and mental
health interventions. The advantage to using these organizations
for a source of practice reviews is that they all have a systematic
process for identifying and reviewing research studies, and they
almost always have a diverse group of people working together
to review the literature. Both of these processes make it less
likely that the biases of one or two people will color the review
(although there are no guarantees) compared to the average review
article published in an academic journal. Here are three major
organizations for that conduct such reviews:
-
Cochrane Collaboration: international
effort to promote evidence-based healthcare for the purpose
of enabling patients & doctors to make informed decisions
about treatment and care. www.cochrane.org
- Campbell Collaboration: international
network focused on helping people make well-informed decisions
about the effects of interventions in the social, behavioral
and educational arenas. www.campbellcollaboration.org.
-
Agency for Healthcare Research and Quality
(AHRQ): the health services research arm of the U.S.
Department of Health and Human Services that translates research
findings into better patient care and provides policymakers
& other health care leaders with information needed to make
critical health care decisions. AHRQ funds Evidence-Based Practice
Centers to review and synthesize scientific evidence for conditions
or technologies that are costly, common, or important to the
Medicare or Medicaid programs. The ISTSS review of treatments
for PTSD used the AHRQ classification system in their report.
www.ahcpr.gov
Here is a web resource that will link you to import evidence-based
resources. Over 210 sites provide full text articles.
-
World Wide Web Resources for Social
Workers (WWWRSW) and Information for Practice (IP):
This site provides over 81,585 world wide web links, many focus
on EBP's. The majority of the links are to full text electronic
journals, newsletters, government agencies, educational institutions
and professional organizations. www.nyu.edu/socialwork/ip/.
When reading a review, it's important to examine the procedures
for how research studies were identified for inclusion and exclusion.
Reviews serve as a good starting place, but should never substitute
for your own careful examination of the literature if it's a topic
that's important to you.
Models for Evaluation of
Research
In the section above we briefly reviewed how you can locate systematic
reviews of evidence-based practice. Some individuals may prefer
to do their own literature search for relevant research studies.
There are sets of guidelines to help in determining which treatments
are best for various mental health problems. Examples of models
for evaluation of research are listed below. There are many other
issues that are related to evidence-based practices; for instance,
there are ongoing debates about the most effective way to disseminate
empirically supported psychotherapies for children and adolescents
and how research data can best be translated for practitioner
and consumer use. These and other evidence-based related issues
are briefly reviewed and listed below. Supporting examples or
narratives are provided when ever possible throughout the reference
list.
Guidelines:
Model 1 - APA Model for Evaluation of Research:
The APA has developed a model with which a practitioner or student
can effectively judge the level of evidence that is provided by
an evidence-based report on the effectiveness of a particular
psychotherapeutic intervention (Crits-Christoph, Frank, Chambless,
Brody & Karp, 1995). The APA developed this model so that
students and clinicians are able to judge the degree to which
a particular research project demonstrates the effectiveness of
the intervention. In the APA model there are three levels of evidence:
Well-Established Treatments
-
Highly specified, typically through treatment
manuals, such that procedures could be clearly understood
and replicated by others.
-
Validated in studies in which the characteristics
of the client samples were clearly described.
-
Documented to be effective in either:
At least two group-design
studies of adequate statistical power demonstrating efficacy
through superiority (statistical significance) to pill or
adequate psychological placebo, or through superiority or
equivalence to an already established treatment.
A large series of single-case design studies
using good experimental design and demonstrating superiority
to pill or psychological placebo or to another treatment
-
Demonstrated to be effective
in studies by at least two different investigators.
Probably Efficacious Treatments
-
Highly specified, typically
through treatment manuals, such that procedures could be clearly
understood and replicated by others.
-
Validated in studies in which
the characteristics of the client samples were clearly described.
-
Documented to be effective
in either:
Two studies showing that the treatment
is more efficacious than a waiting-list control-group
One study demonstrating efficacy either
through performance superior to that of a pill or psychological
placebo, or equivalency to an already established treatment
A small series of good single-case design
studies
At least two good studies demonstrating
efficacy but flawed by the heterogeneity of the client samples
-
Treatments that do not meet
criteria for either of the above categories are labeled experimental.
A treatment with this label may be, but is not definitively
effective.
Model 2 - adopted from the Agency of Health Care Policy's:
The Agency of Health Care Policy's developed a classification
of Level of Evidence system. This system classifies each study
into one of six categories in order to denote the strength of
the evidence for the intervention (Foa, Keane, Friedman, 2000).
-
Level A: Evidence is based
upon randomized, well-controlled clinical trials for individuals
with target diagnosis.
-
Level B: Evidence is based
upon well-designed clinical studies, without randomization or
placebo comparison for individuals with target diagnosis.
-
Level C: Evidence is based
on service and naturalistic clinical studies, combined with
clinical observations that are sufficiently compelling to warrant
use of the treatment technique or follow the specific recommendation
for individuals with target diagnosis.
-
Level D: Evidence is based
on long-standing and widespread clinical practice that has not
been subjected to empirical tests for individuals with target
diagnosis.
-
Level E: Evidence is based
on long-standing practice by circumscribed groups of clinicians
that has not been subjected to empirical tests for individuals
with target diagnosis.
-
Level F: Evidence is based
on recently developed treatment that has not been subjected
to clinical or empirical tests for individuals with target diagnosis.
References and Guidelines
that Address Evaluation of Research Studies
Chorpita, B. F., Yim, L. M., Donkervoet, J. C., Arensdorf, A.,
Amundsen, M.J., McGee, C., Serrano, A., Yates, A., Burns, J.,
Morelli, P. (2002). Toward large scale implementation of empirically
supported treatments for children: A review and
observations by the Hawaii empirical basis to services task force.
Clinical psychology 9, 165-190.
Note - the authors use a modified version of
the APA task force guidelines to examine the effectiveness of
child treatments for anxiety, depression, ADHD, conduct and autistic
disorders.
Crits-Cristoph, P., Frank, E.,, Chambless, D.L., Brody, C., Karp,
J (1995). Training in empirically validated treatments: What are
clinical psychology students learning?Professional Psychology:
Research and Practice, 26(5), 514-522.
Foa, E.B., Keane, T., Friedman, M. (Eds) (2000). Effective
treatments for PTSD: Practice guidelines from the International
Society for Traumatic Stress Studies. New York: Guildford
Press.
Nock, M. K (2003). Progress review of the psychosocial treatment
of child conduct problems. Clinical Psychology 10:1-28.
Note - reviews current practices for evaluating
research and describes a new method of review first proposed by
Alan Kazdin. This method is used in this article to review &
evaluate empirical treatments for child conduct problems.
Task Force on Psychological Intervention Guidelines, APA (1995).
Template for developing guidelines: Interventions for mental
disorders and psychological aspects of physical disorders.
Washington, D.C.: APA.
Dissemination of
Evidence-Based Practices
Addis, M.E. (2002). Methods for disseminating research products
and increasing evidence-based practice: Promises, obstacles, and
future directions Clinical Psychology 9, 367-378.
Note - this article discusses ways to disseminate
research and empirically supported treatments (EST). It addresses
why it is important to combine clinical practice and research.
This article also focuses on the promises, obstacles, and future
direction of both dissemination and EST.
Addis, M.E., Wade, W.A., Hatgis, C. (1999). Barriers to dissemination
of evidence-based practices: Addressing practitioners' concerns
about manual-based psychotherapies. Clinical Psychology
6, 430-441.
Note- This article focuses on practitioners
most common concerns related to implementing EBP, they include
- "effects on the therapeutic relationship, unmet client
needs, competence and job satisfaction, treatment credibility,
restriction of clinical innovation, and feasibility of manual-based
treatments." Each concern is addressed using two categories
what we know, and future direction.
Hawley, K.M., Weisz, J (2002). Increasing the relevance of evidence-based
treatment review to practitioners and consumers. Clinical Psychology
9, 225-230.
Note - reviews the many challenges of identifying
and implementing EB treatments in a clinical setting: the use
of treatments that are "possibly efficacious" while
waiting for the research base to mature; how to select a treatment
(i.e. for childhood depression) when there may be six effective
studies using CBT but each differs from the other in significant
focus; and the need to find ways to support related clinician
training.
Brief Overviews of Empirically Supported Treatments
for Children and Adolescents.
Chorpita, B. F., Yim, L. M., Donkervoet, J. C., Arensdorf,
A., Amundsen, M.J., McGee, C., Serrano, A., Yates, A., Burns,
J., Morelli, P. (2002). Toward large scale implementation
of empirically supported treatments for children: A review
and observations by the Hawaii empirical basis to services
task force. Clinical psychology 9, 165-190.
Note - the authors use a modified version
of the APA task force guidelines to examine the effectiveness
of child treatments for anxiety, depression, ADHD, conduct
and autistic disorders.
Nock, M. K (2003). Progress review of the psychosocial treatment
of child conduct problems. Clinical Psychology 10:1-28.
Note - reviews current practices for
evaluating research and describes a new method of review first
proposed by Alan Kazdin. This method is used in this article
to review & evaluate empirical treatments for child conduct
problems.
Burns, B, Hoagwood, K. Mrazek, P. (1999). Effective treatment
for mental disorders in children and adolescents. Clinical
Child and Family Psychology Review, 2(4), 199-254. Results
of an extensive literature search for effective treatments for
prevention, traditional forms of treatment, community-based interventions,
and crisis and support services.
Kazdin, A., E., Weisz, J.R. (1998). Identifying and developing
empirically supported child and adolescent treatments. Journal
of Consulting and Clinical Psychology, 66, 19-36.
McCellan, Jon, M., Werry, J. (2003). Evidence-based treatments
in child and adolescent psychiatry: An inventory. Journal of
Child and Adolescent Psychiatry 42(12) 1388-1400.
Note - Examines effectiveness of psychopharmacology
and cognitive behavioral, interpersonal, family and parenting,
and systemic interventions.
Additional Resources
A Manual for Behavioral Health Administrators and Practitioners
About Understanding and Implementing Evidence-Based Practices
Note - this can be downloaded at: http://www.tacinc.org/Pubs/TKIP.htm
(127 pages) - a CDROM is also available by calling the Technical
Assistance Collaborative, Inc. at (617) 266-5657
This is an invaluable resource for practitioners. The manual
focuses upon the concepts of recovery, resiliency and EB
thinking. It reviews examples of EB and promising practices.
It guides the reader through the selection and implementation
process involved in adopting an EBT to clinical practice
and address quality management and sustainability.
UB faculty training at offsite OMH locations.
|
