Research and evaluation: a beginner’s guide

Coaten R, Pasiecznik Parsons M (2024) Journal of Dementia Care 32(6)21-22.

Richard Coaten and Maria Pasiecznik Parsons throw light on the often intimidating topics of research and evaluation, explain the difference and give  helpful advice to practitioners on evaluating their work.

Research and evaluation are two words that can be very off-putting to creative health practitioners, especially those new to the subject who imagine it is full of jargon and complex ideas. If you are an activities coordinator or care home manager please do read on as what follows is designed to be useful, easy to read and digest and not a deep dive into the subject. It has some helpful ideas at the bottom so please do persevere!

Research and evaluation are distinct yet complementary methods of inquiry that have different purposes and methods. In this context they are used to better understand the complexities involved in creative health interventions with people living with dementia. In a nutshell, research aims to generate new knowledge and contribute to academic discourse and prove something while evaluation is more focused on assessing the effectiveness of an intervention or, more often, a specific programme, in other words to improve something. The results or outputs of both research and evaluation are about learning and used to help develop the evidence base for, in this instance, creative arts, leading to more effective interventions, programmes and services.

Research: complementary approaches

Quantitative and qualitative research approaches and methods are used in both research and evaluation, but increasingly mixed methods are favoured in “striking a balance between scientific rigour and the flexibility and creativity essential to arts” (Fancourt 2017 p.198) and (really) capturing their benefits. The growth of using creative arts as a medium for maintaining and improving the health and wellbeing of people living with dementia is due, in no small part, to the demand for evidence of its effectiveness (Schneider 2018, 2023).

Simply put, research involves deciding on a focus or problem, developing the research question(s) related to it, choosing a strategy or methodology that helps answer the question(s). A review of the literature is essential to be able to situate the work in a body of knowledge where shortcomings or a gap has been found. Following this, a specific approach to the conduct of the research and tools (methodology) is identified to answer the question(s)/measure change. It also involves arranging the practical elements, collecting the data, preparing and carrying out analysis, reporting what has been discovered (dissemination) and its relevance to the field (Robson 1993).

Crucially, and this cannot be overstated, people with lived experience of the condition and their carers should be fully involved in both processes. ‘Hearing’ the voices of people directly affected has become a core consideration in developing research, policy, and care practice both in the United Kingdom (UK) and worldwide. For example, the Alzheimer’s Society Research Network includes lay members – people living with dementia and carers – who are involved in reviewing grant applications to the charity. Qualitative research methods such as action research, ethnographic and emancipatory methods are being widely used in “field research” with people living with dementia in neighbourhoods and care homes (Ludwin & Capstick 2017). Also, in generic studies where family carers and their relatives act as co-researchers (Birt et al 2023) they provide rich insights into how people living with the condition maintain their personhood in social settings.

Evaluation: assessing effectiveness

Evaluation on the other hand is a systematic process of assessing the effectiveness of an intervention, a project, programme, or policy involving the collection and analysis of information to determine whether or not intended aims and objectives were achieved. These can include participant outcomes such as Quality of Life and Wellbeing, Falls Reduction as well as broader aims such as determining the influence of staff culture on care practice. Beyond judging whether whatever was carried out achieved its aims and objectives, an evaluation seeks to discover the key factors that contributed to its achievement or that were implicated in its failure to achieve desired results. This is in order to inform decisions and/or make recommendations. Findings are typically reported to programme commissioners who are usually the funders and stakeholders and who request information of this type.

Learning and questioning

At the heart of both processes is learning, and at the heart of learning is a desire to question in order to better understand, to be able to reflect on our work whether as care co-ordinator, arts practitioner or creative arts therapist, in order to develop it. Here are some suggestions to help us learn in order that we don’t get stuck in a rut in what we do, finding ourselves doing the same old things in the same old way:

Keep a journal to help you plan, run, evaluate and reflect on your work. There are a number of resources that can help you develop evaluation skills to plan and carry out evaluation of your work – visit the resources below – start simply and build your confidence and resource bank.

Record what you do in a journal, for example write out a session plan (or plans if you are running a number of sessions) including aims and objectives and how you are going to evaluate what you do. Add a narrative of your experience, pen pictures of participants and their responses and reflect on your experience. Use the reflective cycle (of which there are several – see below) to do this in a systematic way. If you are going to work with a group over time, involve participants in co-designing the session and gather feedback and biographical information that can help you tailor what you do to their interests and needs. Collect and record basic statistical information about participants’ and their attendance. Journal contents may also be used to populate research and/or evaluation write ups. Remember that whatever you do in terms of research and evaluation is a contribution to your own learning and development, as it is to the field when you are able to get your work published!

Visit CHWA to access a range of useful resources including research and evaluation https://www.culturehealthandwellbeing.org.uk/resources/research-and-evaluation.

How might you evaluate what you do? Try https://www. culturehealthandwellbeing.org.uk/i-want-evaluate-my-work

Publication

Research is typically published as an article in an academic journal, often online where it might be available freely via Google Scholar or Pub Med, though some online academic journals have payment firewalls and only offer an abstract of the articles. Nevertheless, abstracts are usually short clear summaries of the article proper that include key recommendations or results from a research study and are a good start if engaging in research.

Student research produced in UK and global universities can be accessed via their research repositories https://v2.sherpa.ac.uk/opendoar/ . For example, Canterbury Christchurch University staff and students upload their research to https://repository.canterbury.ac.uk/

Resources

The Centre for Cultural Value has a comprehensive website that offers resources for research and evaluation inhttps://www.culturehive.co.uk/research-and-evaluation-practice/ and a FREE online course https://www.culturalvalue.org.uk/evaluation-arts-culture-heritage-online-course/

The Centre for Cultural Value evaluation course can also be accessed along with hundreds of free courses on many different aspects of research and evaluation at Future Learn https://www.futurelearn.com/

Join your local university library as a public member.

Join Dementia Community and subscribe to the Journal of Dementia Care. The journal always includes research and evaluation articles, often about creative arts, and subscribers have access to the whole archive of issues back to 1993. Events are posted on the website including free webinars featuring researchers.

Research and evaluation is presented at conferences such as Dementia Community’s UK Dementia Congress, the British Society of Gerontology, and Alzheimer Europe.

Check out Dementia and Imagination – A short guide for artists and other people who plan to deliver arts-based activities with people who are living with dementia, that provides a set of useful ideas and recommendations distilled from a robust research project setting out some foundations for developing visual arts projects with and for, people affected by dementia. https://www.artsforhealth.org/resources/dementia-and-imagination.pdf

References

Culture Health and Wellbeing Alliance (2024) Research and Evaluation. Accessed online 22/5/24 at https://www.culturehealthandwellbeing.org.uk/resources/research-and-evaluation
Birt, L., Charlesworth, G., Moniz-Cook E., Leung, P., Higgs, P., Orrell M., & Poland F. (2023) The Dynamic Nature of Being a Person: An Ethnographic Study of People Living With Dementia in Their Communities. The Gerontologist, Volume 63, Issue 8, October 2023, Pages 1320–1329. Accessed online: https://doi.org/10.1093/geront/gnad022
Fancourt, D. (2017) Arts in Health—Designing and Researching Interventions. Oxford University Press. Accessed online:  https://doi.org/10.1093/oso/9780198792079.001.0001
Ludwin K, Capstick A (2017) Ethnography in Dementia Care Research: observations on ability and capacity. SAGE Research Methods Cases – Ethnography. London: SAGE. Accessed online https://bradscholars.brad.ac.uk/handle/10454/10921
Robson, C. (1993) Real World Research – A Resource for Social Scientists and Practitioner-Researchers. Blackwell Pubs. Oxford UK and Cambridge USA.
Social Value UK (SROI) (2024) Social Return on Investment, Standards and Guidance. Accessed online: https://socialvalueuk.org/standards-and-guidance/
Schneider, J. (2023). The arts in dementia: instrumental and experiential perspectives. Aging & Mental Health, 27(10), 1861–1863. Accessed online: https://doi.org/10.1080/13607863.2023.2205827
Schneider, J. (2018) The Arts as a medium for Care and Self-Care in Dementia: Arguments and Evidence. International Journal of Environmental Research and Public Health, 15, 1151 pps 1-9.