Exercise 2: Evidence Needs in Parks and Protected Areas

What is the Process for Applying Research?

Keeping in mind that research informs decisions related to management, there are several things to consider.

  • Understand the question or situation that requires exploring the current research.
  • What is the goal or purpose of consulting the research?
  • Is there a gap between the research available and the question?
  • Determine where research fits in the cycle of planning, proposing, proposing, implementing, and assessing?
  • What is the time frame for incorporating research?
  • Does the organization have the capacity to conduct research, or will it be gathered from existing studies?
  • Where does the research come from? What is available?  How will it be gathered and synthesized?
  • How will the decisions resulting from the research be documented?
  • Monitor, assess, and/or evaluate the situation in accordance with the applied research. How is the impact or effect of the applied research monitored?

Using Critical Thinking

When selecting research papers or articles for use in planning or making decisions, care needs to be taken to ensure that the evidence, data, or information we are gathering is ‘quality’ research. Each piece of research needs to be viewed with a skeptical eye, looking for consistency, plausibility, relevance, degree of detail, and whether the material is current or outdated.

We may assume that the word ‘critical’ means to criticize or find fault, but here we are using it in the sense of being respectfully skeptical and applying a set of questions to test the efficacy of evidence. As Weiss et al., tell us “…people and systems change, and as they do, audiences need to maintain a healthy skepticism about the validity of even the most robust research evidence” (p. 31)

Weiss et al., (2008) summarize critical thinking in evaluating materials for research as:

A widespread assumption is that research and evaluation should influence the making of public policy. In recent years, this belief has flourished under the label of evidence-based policy. It holds that the findings of research should help policy makers develop wise policy that is based in strong understanding of current conditions. It calls on policy makers to learn what research has to say before they dive into the swamps of the policy process. (p. 29)

Testing the evidence

Our ‘test’ of the evidence, data, report, or article will consist of five main areas to question.  Each of these areas in turn generates questions we can as of the evidence.  The overarching question we ask ourselves is “How do I know that this piece of evidence is solid, reliable, and of good quality?”

1. organizational support for evidence-based decision-making

2. Availability and accessibility of research findings/ evidence/data/information

3. Good evaluation of research/data/evidence using critical thinking

4. Belief in value of using evidence to support decisions

5. Clear processes for applying evidence to decision-making

solid,
justifiable,
defendable decision

Evaluating evidence for decision-making

quality

  • Is this research published in a peer-reviewed journal? Who published it? Is it a relevant publication to the decision? 
  • Has this material been cited by others? Recently?  …as a good example, bad example, foundational info?  Are the claims supported or challenged by other authors?
  • How convincing is what the researcher is saying? What arguments support the main point?
  • Have limitations and biases been identified and acknowledged?
  • Does the researcher include counterevidence of their claim? Have they included all viewpoints?  Are there opposing views to consider?

accuracy

  • Is the evidence credible? Are the facts accurate?  What methodology was used?
  • Is the information credible? Is it verifiable?
  • Was the data triangulated?

author

  • Is the author known or credible in their field? How can I tell?  What are their credentials?
  • Have they published a substantial amount of work? Are they known?  Are they well documented?  What are their affiliations? 
  • What could be influencing their conclusions? Who provided funding for their research?
  • Can I trust their research?

purpose

  • What is the researcher trying to do/achieve? What is their position on this topic?  What is the ‘so what’?
  • Is the researcher or author stating fact or opinion? If facts, are they backed up?
  • What is the main point? What is the purpose of this material?  What questions are being addressed?

currency

  • What is the date on this research / evidence? Is it current our ‘classic’?
  • How frequently is this publication published?

other
questions

  • How was the research funded?
  • What assumptions did the researcher make, if any?
  • Does it seem like there is an Inclination to favour certain evidence over other evidence because it is ‘home grown’ or produced in-house?

 

Not every question will be relevant to every evaluation, but this is a good starting point to look at the materials you gather for decision-making.  Barends et al., (2014, p. 11) describes the best available evidence should be gathered, then tested for trustworthiness.