Framing your messages to change thinking

This is the third post of three about resources for research communication.

• First one was about Approaching and influencing policy makers and stakeholders
• Second one was about Identifying audiences and shaping the message

pine watt

I have been impressed with work done by Joseph Rowntree Foundation (JRF) to rethink their communication strategy in terms of framing messages based on how cultural models of poverty influence how people understand and interpret messages.

In Sep 2019, I read a fascinating Social Research Association blog post about communicating research by Helen Barnard of the Joseph Rowntree Foundation (JRF). In the post entitled ‘Communicating research: what we’ve been doing wrong, and how to get it right’, Helen writes about how presenting ‘research findings clearly, in accessible language and drawing out any implications for action’ needs to be coupled with an understanding of how people receive and interpret these in order to change thinking and ultimately policy. The prompt to explore communication strategies came from the frustration when presenting a clear cut case backed up by robust quantitative and qualitative evidence that nevertheless didn’t translate into the expected changes and the realisation that people did not get the ‘expected’ take away messages.

The rethinking of how JRF communicates about poverty came about as a result of extensive research and collaboration with the Frameworks Institute. The research identified different cultural models of poverty (or public attitudes) that come into play when messages and evidence about poverty were presented.

 “When I started to use the framing insights in speaking to the media, writing a blog or presenting to a stakeholder group, I could immediately see the difference.  Conversations which had previously been frustrating, suddenly became much more constructive. The ‘take-away’ messages that we saw played back by the media, politicians and others tended to be the parts that were ‘framed’.” (Helen Barnard).

I find this useful because it is not just talking about how to align messages so that it fits with the needs or interests of stakeholders but rather that messages need ‘framing’. When communicating you need to plan for telling a (sometimes alternative or contested) story in ways that convincingly engage with and challenge existing stories, attitudes and assumptions – and in order to do this successfully you need to know what these are.

For more on this have a look at a presentation about reframing poverty from Jan 2018 by Nat Kendall-Taylor, Chief Executive Officer at the FrameWorks Institute.

Reframing poverty: 8 ways to Open Minds from josephrowntreefoundation

Identifying audiences and shaping the message.

This is the second post of three about resources for research communication. And really it is all about asking yourself a lot of questions.

Emily Morter

Although, before sharing ideas for communicating research, it is worth just taking a moment to consider whether the research outputs are easy to discover and access.

• Is the research published in Open Access journals or is there a version available from an organisational repository? You may want to take a look at the Directory of Open Access Journals
• How are you making key research content available to people outside academia?
• Where will the content that you produce be available from and will this be a sustainable platform?

As discussed in my first post, a key part of developing a strategy for influencing and approaching policy makers and stake holders is to do the essential work of identifying who they are and what their needs and interests are. Once you have undertaken this work you can start developing your communication plans.

But don’t just think about or plan communication at the end of your research!

Establishing relationships with stakeholders and collaborating early on as part of your research  is key to achieving successful communication in later stages.

Who do you want to reach and what do you want them to do with the information?

(p.24 in Communicating your research, Module 5 in The Overseas Development Institute (ODI) Research Excellence Framework impact toolkit)

The Overseas Development Institute (ODI) Research Excellence Framework impact toolkit is an excellent starting point as it has some tasks for ‘identifying audiences’ and ‘communicating messages’. The toolkit was written by Helen Tilley, Louise Ball and Caroline Cassidy and published in March 2018 and I keep returning to it for ideas. 

The ODI toolkit shares an example from Heath and Heath (2007) about how to make your message stick, which outlines six dimensions to a good message:

1. Simple. The best messages are always simple. This does not mean dumbed down. Rather, you are pulling out the key message your audience needs to know. Keep it concise and don’t over-elaborate.
2. Unexpected. If you can make people sit up and listen you’re more likely to have impact. Counter-intuitive messages, a surprising fact or a different angle can be one way to do this.
3. Concrete. Don’t be too abstract. Make your message tangible, relating to real life examples where possible. For example, rather than saying 25%, say one in four people.
4. Credible. Your message can be clear, concise, well-constructed, but if it isn’t credible the audience is unlikely to listen. Is your message backed up by evidence? Is it believable?
5. Emotional. We know emotions provoke reaction. Is there a way to explore emotions with your audience, make them connect or care about the human story behind your research?
6. Stories. We use storytelling all the time for effective communications. Can your message tell a story rather than just hard facts? What’s the impact of the evidence?

More resources:

Communicating your research – a toolkit https://www.health.org.uk/publications/communicating-your-research-a-toolkit

Research to Action. Just starting out in research communications? https://www.researchtoaction.org/2018/11/just-starting-out-in-research-communications/

Heath, C. and Heath, D. (2007) Made to stick: why some ideas take hold and others come unstuck. London: Random House Books.

Approaching and influencing policy makers and stakeholders

This is the first post of three about resources for research communication.

My current role involves developing research impact so I have been thinking about research communication and [as usual] I have therefore been collecting resources. So I thought I would share ones I have come across and some that my colleagues Elizabeth Pente and Megan Taylor have shared with me that I think are really useful.

Your research may be relevant to a large number of different audiences and so it is tempting to fall back on saying it is for ‘everyone’, globally relevant or of interest to the ‘general public’. But to avoid shouting into the void, it is necessary to be specific about the stakeholders you want to engage with.

Jason Rosewell

Take some time to identify your research impact objectives, make use of ROMA – the RAPID Outcome Mapping Approach. Again the Overseas Development Institute (ODI) has a great resource available ROMA – a guide to policy engagement and policy influence

Apart from great advice on how to map out your objectives, the resources also offers guidance on developing a theory of change, which is necessary to show progress and impact as well as how to develop a monitoring and evaluation plan.

Once you have defined the outcomes you are looking for, the next step is to think about how these are likely to come about. Having an idea of how change is likely to happen will help you identify entry points and opportunities for the kind of actions you ought to take to set change processes in motion. https://www.odi.org/features/roma/develop-strategy/theory-of-change

I think it is important to be realistic about your spheres of influence and to realise that building contacts and a network is a long term process that needs to be mutually beneficial in order to be successful and sustainable.

• Consider working with established stakeholders with an existing platform that policy makers are already seeking input from.
• Work to identify overlaps and alignment with existing national policy, local policies and local action plans as well as other third sector initiatives. 
• Use Hansard search to identify specific MPs who have asked questions of Government in this area by looking at recent debates and reports on the topic.
• Check the Open calls for Committee inquiries in Parliament
• Discover and get in touch with the APPGs relevant to your area of expertise
• Follow the work of the Parliamentary Office of Science & Technology (POST) – the UK Parliament’s in-house source of scientific information and find out how you can contribute, eg to a POSTnote, summaries of public policy issues.
  

I recommend taking a look at the excellent Handbook to connect research with policy making (2019) by Chris Fox, Sam Gray and Jack O’Sullivan from MetroPolis/Manchester Metropolitan University. They cover a many of these points in a lot more detail with really useful lists of tasks that you can go through.

Engaging with the UK Parliament

All you need to know about the ways to engage with Parliament is available at the Research impact at the UK Parliament and I highly recommend following the work of the Knowledge Exchange Unit based there. They support the exchange of information and expertise between researchers and the UK Parliament. The team offer training around the UK and is on twitter where they post really useful opportunities for researchers to get involved.

More resources

For advice on policy briefs, see FastTrackImpact – https://www.fasttrackimpact.com/single-post/2015/12/19/How-to-make-a-policy-brief-that-has-real-impact

Successful Communication – A Toolkit for Researchers and Civil Society Organisations (2005) by Ingie Hovland. https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/192.pdf

Developing a communications and impact strategy by Economic and Social Research Council https://esrc.ukri.org/research/impact-toolkit/developing-a-communications-and-impact-strategy/

Wikipedia editing with the #femedtech network

I signed up to be curator for the (@femedtech) twitter account for the 11-22 Feb 2019 slot. I thought it might be useful to have an activity or theme to focus on and the idea of trying to organise an online asynchronous distributed Wikipedia- editathon came about. I have enjoyed taking part in previous wikipedia editing events and wanted to get more experience.

The idea was to offer an opportunity for individuals to work individually and together over the two weeks to create and add to Wikipedia articles. The aim was also to add information and sources on overlooked and underrepresented people with important and critical contributions.

The two weeks turned into a very collaborative effort, as people with a lot more experience that me, eg. Dr Sharon Flynn set up the event page and editors from the Wiki Project Women in Red offered links and resources to add. I added resources for ‘how to edit’ on the events page so that less experienced participants had a place to start.

https://en.wikipedia.org/wiki/Wikipedia:Meetup/femedtech_2019

Frances Bell, Julia Hengstler and Lou Mycroft were also very active in both supporting others, sharing their experiences and creating lots of discussion.

Later, following a suggestion by Julia, Sharon also set up a dashboard to see the amount of work done during the ‘editathon’. Something I had no knowledge or experience with so that was really useful to learn about.

I tried to do a few different things:

• Added some references and an open access publication to the article on Paulo Freire
• Added reference details to an entry on the books ‘Women’s way of Knowing’, prompted by Maha Bali
• Added reference details to a book stub entry on belle hook’s ‘Feminist Theory from Margin to Center’
• Picked someone from Wiki Project Women in Red to try my hand at writing a biography and increase the visibility of women in Wikipedia.

The activity prompted me to read more hooks [always a good thing], and adding references and short summaries was an easy way to contribute that felt like you were actually making a difference.

The challenges of sources when writing for Wikipedia

I just want to think-out-loud about my attempt to write a ‘biography’ entry. The person I, somewhat randomly picked, was Yukako Uchinaga. https://en.wikipedia.org/wiki/Draft:Yukako_Uchinaga

The draft submission was declined by a Wikipedia editor with comments around the style and the sources used:

“…this is a press release, not an encyclopedia article. First give her bio, including birth date and place, all degrees with date, title of her thesis. List her most cited publications, if any. Describe what she did at IBM, with a reliable source. Her speeches at forum are not encyclopedic content”

I don’t necessarily disagree with all the comments. It was definitely a challenge to find sources for the life and career of Yukako Uchinaga but what I found in the public domain seemed to suggest she is indeed a notable person with impressive achievements. For example, being awarded the ‘Prime Minister’s Commendation for Efforts Toward the Formation of a Gender-Equal Society’ and being the founder of Japan Women’s Innovative Network (J-WIN), a non-profit organization that promotes diversity in the workplace.

However, many of the sources are from non-academic sources, such as business news and forums or indeed Yukako Uchinaga’s long-time employer IBM. And I readily confess to not really having any ideas for how to get information on the title of her thesis from 1971 from University of Tokyo or even how to find out when she was born so that I can create a more traditional biographic encyclopedic entry. I suspect there may be Japanese sources that could help with this but I don’t read Japanese; I also wonder what would happen if I did find some, would they be accepted?

The email declining the article had a number of useful looking links to get support from and I have since then added an additional source but I don’t think it will ever be a traditional entry in the encyclopedic sense ‘required’. I am not sure what alternative formats might be a better fit for information like this?

There has, of course, recently been a number of instances [I am not saying this is one] where Wikipedia gatekeeping have been highlighted as reinforcing societal biases, see a recent article analysing the case of Clarice Phelps, whose page was deleted https://undark.org/2019/04/25/wikipedia-diversity-problem/

Anyway, the activity left me with lots to ponder about sources, what knowledge counts, limitations of English, the importance of Open Access, and page formats. I am certainly not done with editing Wikipedia and have more activities planned.

[This post has also been submitted as a story to the Open Space at femedtech.net ]

Evaluating and evidencing research impact

Building evidence for research impact

 

Photo by Susan Holt Simpson on Unsplash

Having written a post about Research Impact Resource, I thought I would add some more resources related more specifically to evaluating and evidencing research impact.

Designing an impact evaluation

Prof Mark Reed from Fast track impact (an excellent resource for all things impact) recorded a Vlog where he explains how to do an impact evaluation to prove whether or not your research had significant and far-reaching benefits. https://youtu.be/p9J_J-4sqH4 

Mark talks about how to go about designing an evaluation that proves your research has had an impact, that is significant and has reach, and you can prove is linked to your research (attribution). He suggests starting your evaluation design based on research approaches that you are comfortable with, eg. for a statistician this may be a multiple regression analysis, for a humanities scholar this might be about capturing stories. Mark recommends playing to your subject area strengths, the methods that you are already comfortable with.

How do I come up with an effective design? Mark outlines a model for ‘How to evidence impact’ from a forthcoming paper, which includes six evaluation designs:

1. Quantitative, experimental and statistical
2. Theory/logic driven
3. Systems/pathways analysis
4. Case-based and qualitative analysis
5. Creative and participatory
6. Evidence synthesis

How much evidence is enough evidence? Answer is: ‘when it is believable’. Get feedback from your colleagues to find out where there may be be holes in the impact evidence and then add to your design (eg. getting a larger samples or generating an in-depth case study.)

Social impact and social media

A recent paper explores the benefits of researchers sharing not just research findings but also the ‘social impact’ of their research on social media.

‘…the creation of a new method to evaluate the evidence of social impact shared in social media. The evaluation proposed is to measure the social impact coverage ratio (SICOR), focusing on the presence of evidence of social impact shared in social media.’ (Pulido, Redondo-Sama, Sorde´-Martı´and Flecha 2018, page 3)

Social impact in social media: A new method to evaluate the social impact of research (2018) by Pulido CM, Redondo-Sama G, Sorde´-Martı´ T, Flecha R. PLoS ONE 13(8): e0203117. https://doi.org/10.1371/journal.pone.0203117

Impact Evidence

Research Funding and Impact Consultant Stephen Kemp also has a useful website with resources and a blog post with some advice for what testimonials and statements about research impact could include: Guidance on testimonials and statements to corroborate impact from May 2018.

Another useful resource is the journal: Research For All – it is open access journal and has a range of articles including ones about evaluating impact from public engagement and ideas for involving users (often a key pathway to demonstrate impact). The journal is co-sponsored by the UCL Institute of Education and the National Co-ordinating Centre for Public Engagement.

See for example: Copley, J. (2018) ‘Providing evidence of impact from public engagement with research: A case study from the UK’s Research Excellence Framework (REF)’.
Research for All, 2 (2): 230–243.

Research impact resources

The impact unicorn in its natural habitat. (Image by rawpixel.com)

I have been looking at resources in the area of research impact, and as I am in the UK, this means a lot of this is related to the Research Excellence Framework (REF). I wanted to share what I have collated and I will probably be adding to this as I read more.

In the REF draft guidance on submission from July 2018 impact is broadly defined. Here is a snippet:

Impact includes, but is not limited to, an effect on, change or benefit to: the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding of an audience, beneficiary, community, constituency, organisation or individuals in any geographic location whether locally, regionally, nationally or internationally.

I am not sure whether Dr. Bayley coined the phrase ‘impact unicorn’ but it is a great phrase (and I borrowed it for the illustration of this post) and a good reminder that impact comes in all shapes and sizes. See the posts she has written about impact:

• Chasing the ‘impact unicorn’ – myths and methods in demonstrating research benefits by Dr Julie Bayley, 23rd Nov 2017.
• Sausages, unicorns and strip clubs. Or impact: the challenge of connection by Julie Bayley, Oct 4th 2018.

Research Excellence Framework (REF) Impact Toolkit (March 2018) by Helen Tilley, Louise Ball and Caroline Cassidy. Great toolkit for researchers who work in policy contexts.

ESRC Impact toolkit https://esrc.ukri.org/research/impact-toolkit/

Blog post summary sheet by David Evans

Make Your Research Known – 10 Tools to Increase Consumption of Your Research by David Evans, the World Bank blog, 26 Sep 2018. First and foremost the recommendation is to distil your message into two or three bullet points to communicate effectively with people outside your field. Also remember that the main points may need to be different for different audiences.

How was Social Media cited in 2014 REF impact case studies? by Katy Jordan and Mark Carrigan, from the LSE Impact blog, June 6th 2018.

‘Effective engagement is a matter of building relationships over time, as opposed to simply making material public via social media. An excessive fixation on metrics “bakes in” a dissemination model of social media, obscuring the relational value it can create (and the capacity for impact ensuing from this) in pursuit of ever more impressive engagement metrics in spite of the ambiguity about what, if anything, these entail for real world impact.’ (Jordan & Carrigan, 2018)

The “long tail” of research impact is engendered by innovative dissemination tools and meaningful community engagement by Kip Jones and Lee-Ann Fenge, LSE impact blog, 20 Feb 2018. Based on the authors’ article, “Meaningful dissemination produces the “long tail” that engenders community impact”, published in the Qualitative Research Journal (DOI: 10.1108/QRJ-D-17-00037).

‘The fact that the underpinning research began almost a decade and a half ago attests to the principle that research that is meaningful is never really finished, and that dissemination is more than simply a few academic journal articles’ (Kip-Jones & Fenge, 2018)

Developing approaches to research impact assessment and evaluation: lessons from a Canadian health research funder by Julia Langton, from the LSE impact blog, 19th OCt 2018.

McKiernan EC (2017) Imagining the “open” university: Sharing scholarship to improve research and education. PLoS Biol 15(10): e1002614.  https://doi.org/10.1371/journal.pbio.1002614

‘A university’s societal impact depends on the commitment of faculty to transforming their research into reusable information, sharing, and participating in community outreach. As said before, if we want such commitment, universities must develop ways of recognizing and rewarding these activities. Traditional scholarly metrics, like the number of articles published and journal impact factor, give an incomplete picture of true impact. In my opinion, we need a broader perspective’ (McKiernan, 2017)

Kate Williams, Jonathan Grant (2018). A comparative review of how the policy and procedures to assess research impact evolved in Australia and the UK, Research Evaluation, Volume 27, Issue 2, 1 April 2018, Pages 93–105.

ISRIA statement: ten-point guidelines for an effective process of research impact assessment. Health Research Policy and Systems 2018, 16:8. 

Research Impact on Legislatures, Sep 2018.  A briefing note providing an overview of research impact in legislatures. https://www.parliament.uk/research-impact.

See also Research impact at the UK Parliament for resources and training events https://www.parliament.uk/get-involved/research-impact-at-the-uk-parliament/

David Phipps, Joanne Cummings, Debra Pepler, Wendy Craig, and Shelley Cardinal (2016). The Co-produced Pathway to Impact Describes Knowledge Mobilization Processes. Journal of Community Engagement and Scholarship, vol 9, no 1.

For a critical review of the impact agenda see Professor Robert Macdonald’s paper: Robert MacDonald, (2017) ““Impact”, research and slaying Zombies: the pressures and possibilities of the REF”, International Journal of Sociology and Social Policy, Vol. 37 Issue: 11-12, pp.696-710, https://doi.org/10.1108/IJSSP-04-2016-0047

National Co-ordinating Centre for Public Engagement is a great place for resources and they also published a review of how public engagement featured in the case studies submitted for REF 2014.

Impact playbook for Museums, Libraries and Archives by the Europeana Foundation

Emerald Publishing Impact resources https://www.emeraldpublishing.com/resources

UK Research and Innovation on impact https://www.ukri.org/innovation/excellence-with-impact/

Presenting your narrative to demonstrate fit: reflections on applying for jobs and attending interviews

photo by Antony Pilling (all rights reserved)

Between July 2017 and Aug 2018 I have written about fifty applications and attended around ten interviews. I applied mainly to UK universities that were in commuting distance, charity/museum sector, national/international organisational jobs with a focus on research, communication, engagement, academic and research support and project coordination. Here is some of the stuff I have thought about based on a year of job applications and interviews.

Up to date CV

If, like me, you worked for one organisation for a long time [seven years] it can be really difficult to remember all the projects, tasks and changing responsibilities that you have been involved in. But it is important to reflect on these and consider the things you have learnt from some of the situations you have been in, eg. difficult negotiations, giving feedback, team communication skills, working with a variety of organisations/stakeholders/external partners etc.

Applications

All organisations are different and require different type of applications, and this is even more true if you are applying for jobs in different countries. Do the research to find out what the expectations are for a CV and application. If you are applying to a UK University, most of the time I would say that you need to make sure that you provide at least one example for each criteria. I also tended to include a short paragraph at the end about why I am interested in the role.

I found motivated cover letters very difficult to write and I am not sure I ever completely got the hang of them as they are in addition to a CV. But I did feel the best ones I wrote contained details of what I would bring to the role and about my personal motivation. But as I never got shortlisted for interview for any of the roles that required a cover letter, perhaps that is an indication that there is more work to do. It is difficult to say as those organisations requiring cover letters either sent very basic emails to say the job was filled or never got back to me about the outcome.

Writing more or less the same application for similar roles/criteria to different organisations can result in being shortlisted or not. All the things you cannot control, such as amount and quality of other candidates, play a huge role and you can only do your best application/interview for any given opportunity.

I asked my friends to read through numerous applications to give me feedback and this was always really helpful – also sometimes they suggested better examples as they knew work I had done. I can’t thank my friends enough for supporting me with the job search.

Interviews

Research your interview panel if you get the details of who they are. Look up their professional profile and social media presence.

This is really obvious but you need to prepare example answers for interview questions. A good rule is to prepare answers that have examples structured, for example following a STAR structure. Situation, Task, Action and Result. This allows you to develop structured and compelling answers where you outline the situation, detail the task required, the action YOU took and the outcome as result. If like me you find it difficult to constantly foreground yourself and your contribution then try and think about it as if you are talking about someone else BUT you do need to say ‘I’ a lot, as in I did this, I had this responsibility, I contributed this etc.

Prepare answers to the following questions:

• What motivated you to apply for the role and what skills and experience do you think you would bring to the position?
• Can you give us an example of when you have worked effectively as a member of a team?
• Can you give us an example of when you worked independently on a project/task and with a tight deadline?
• Can you give us an example of when you had to deliver some negative/critical feedback and how you approached this situation?
• Can you give us an example of when you have had to use your initiative [to make a decision, to accomplish a task etc]?
• How would you anticipate establishing and maintaining effective communication channels [with colleagues in other departments, external partners etc]?
• How would you handle conflicting deadlines for tasks? Or how would you prioritise a complex workload? It is likely there will be a question about how you organise/approach your work/projects.

Even though I am not sure I think it is quite fair on applicants, as they only really know what is in the job specification, I have realised that it is a very good idea to present a narrative about how you will approach working in the role. So if the role requires communication via social media, develop a communications strategy with details of channels and pros and cons. Presenting a written version at the interview of what you have prepared is a good idea. I know that a lot of interviews give you a task to prepare and this may include aspects of this. This is about presenting yourself and your skills/experience/knowledge in relation to what they want the role to achieve. A lot of this may well be hypothetical as you don’t know how the role will be. It is about creating a narrative with details that demonstrates the fit between you and them.

If you are asked to do a presentation and you are using power point slides, then bring a print out for each of the panel members so they have a copy to scribble on. Also bring any other relevant examples of your work and give the panel one copy.

Do try and think about one or two questions that you want to ask, you may weave them into the main bit of the interview, but it is your opportunity to steer a discussion. If it is a new role you could ask about the context for the role being created. If it is a permanent role you could ask about any professional development opportunities that are available.

Feedback

Always ask for feedback after an interview. Yes, I know it is no fun but most of the time it can be useful, maybe even in ways you hadn’t thought about. One organisation gave me the feedback that my presentation style was too informal and this just confirmed for me that this was not an environment I wanted to work in.