A super blog by Jack Hassard describing yet-another-think-tank-report on Science Education, points out that if anyone in an argument says “all the scientific research shows” then it is worth ensuring that what they are pointing to really hits the standard they are claiming.
This is not a new point, but what Hassard does give is a handy “Is this scientific?” checklist based on the criteria supported by the American Education Research Association.
To be considered ‘credible & valid’ education research should include:
- The development of a logical, evidence-based chain of reasoning;
- Methods appropriate to the questions posed;
- Observational or experimental designs and instruments that provide reliable and generalizable findings;
- Data and analysis adequate to support findings;
- Description of procedures and results clearly and in detail, including specification of the population to which the findings can be generalized;
- Adherence to professional norms of peer review;
- Dissemination of findings to contribute to scientific knowledge;
- Access to data for reanalysis, replication, and the opportunity to build on findings.
My caveat to this list is that if you are falling short somewhere with your evidence – do not panic. It still matters. What I saw each day in my classroom with my eyes is evidence and I will use it when I am arguing with people as a way of demonstrating a point. But what I cannot do is claim that my anecdote is ‘scientific’ nor that it is is ‘credible and valid research’. Stories are useful and we should still debate their merits and meaning. Reports of people’s opinions, the gathering and critique of data, the pointing out of logical inaccuracies in a policy: all of these are useful. But they are not science.