Skill Series: Data Analysis & Report Writing
This page will be particularly useful for those assisting with reports on mental health and education and any other project which involved audience research such as writing marketing or social media campaigns.
Remember!
Even if you are unable to be directly involved in some of the following areas in your action project, it is important to be aware of all the following stages in the process of writing an actionable report so that you know, when you start your action project, what has already been completed, what needs to be done and what could have been done better. This will facilitate the development of the project itself as well as your own professional development. Knowledge of evaluation methods and report writing is an extremely valuable skill in the job market.
- How to collate data
- How to analyse data
- How to present data
- How to write a report
A Brief Guide to Collating Data
Research projects are completed when we don’t know any of the following:
- Who are audiences are, or could be (e.g. when creating a marketing campaign)
- What our audiences want, think or respond to (e.g. when creating a social media campaign)
- Where the opportunity is for something new (e.g. when creating a business plan)
- How much to invest in changing a current approach (e.g. mental health or education reports
Make sure that your research question is/has:
- Specific
- A narrow focus
- Answerable – all research questions should be problem-based
Make sure you know the following:
- Who you are writing the report for?
- Who are you asking data from?
Again make sure these audiences are:
- Specific
- Skeptical about your research question (this will increase your chances of gathering actionable, rather than favourable, data to solve a problem)
It is important to take the following factors into consideration when writing research…
Method :
- What is your research question?
- What type of audience are you working with to collect data from?
- How much budget do you have?
- What timescale are you working towards?
Numerical data.
What type of questions does it answer?
Closed questions with yes/no or multiple choice answers.
What are the benefits of this type of data?
- It is easily scalable (e.g. on bar charts, graphs and other visuals)
- Fewer resources (factual data is easier to collect)
- Less subjective (the questions require only right or wrong answers)
- More truthful answers (factual answers themselves are less up to interpretation)
What are the disadvantages of this type of data?
- Predetermined, limited range of answers
- Slower process (you need more responses to make your data meaningful)
Word data.
What type of questions does it answer?
Open questions with descriptive answers.
What are the benefits of this type of data?
- It is a more dynamic process
- More detailed data which creates faster results (you do not need as many responses to make your data meaningful)
- Is able to address more sensitive issues (e.g. mental health)
- Stories are more memorable (e.g. enables you to include quotes and stories in your analysis to demonstrate a point)
What are the disadvantages of this type of data?
- Not necessarily representative
- Higher investment (e.g. in time and resources spent on evaluation process) needed
- Difficult to find trends (e.g. data up to multiple interpretations)
- Not as reliable (e.g. data based on emotive responses less objectively reliable)
Use the data you already have. Examples include:
- Social media analytics for demographic information (e.g. Google, facebook, twitter, instagram, youtube)
- Surveys and interviews from prior research
The most commonly used method.
Try it when you:
- Have lots of questions to ask
- Want to represent an entire group of people
Format: Online, iPad, interactive paper
Length: 10 minutes
Number: 1000 responses
Timeline: 2-4 weeks
Tools: survey monkey, google surveys, HotJar
Top Tips:
- Be aware of appropriate timing
- Have 3 colleagues take the survey first to test the wording of questions
- If you are using online surveys, leave them open for 1-2 weeks and promote them via email at approx. 3 times
- Make the first question easy, last question open-ended
- For sensitive questions, ask what a colleague would do
- Anticipate cross-tab analysis opportunities
Watch out for:
- Overlapping or missing answer choices
- Biased ranking answer choices
- 2+ consecutive open-ended questions
- 5+ answer choice options
- “Double-barreled” questions (asking 2 things in 1 question)
- Undefined time periods
Audience A tries version A; Audience B tries version B; which gets the best results?
Try it when you:
- Have stakeholders that disagree on a solution to a problem
- Anticipate a singular, “majority wins” outcome
Format: Email, social media ads, website, in-person
Number: 1,000+ (online), 50+ (in person)
Timeline: 1 day – 4 + weeks
Tools: Google Optimise,
Resources: “A/B Testing” by Dan Siroker, Optimizely.com
Top Tips:
- Have a single difference between variants
- Ensure similar audiences are trying version A and B
- Don’t run multiple A/B tests in parallel
- Before you begin: write down your hypothesis
- After you’ve finished: summarise test variations and results for reference later
- Not all tests will have a clear “winning” result
Watch audience behaviour while they’re in your space or another space.
Try it when you:
- Want to understand new audiences or behaviors
- Want to improve an existing process
- Expect easy to interpret behavior
Format: Dwell time, navigation path, public spaces
Length: 1-4 hours
Number: 50+ participants
People: 1+ observers
Timeline: 1 day – 1 week
Tools: Phone or paper
Top Tips:
- Consider using observer(s) who are demographically similar to the audience
- Maintain inter-observer consistency through training
- Create an “observational protocol”
- Track 3-10 elements of behavior or demographics
- Observer(s) should be unobtrusive
- Consider informed consent (signage)
Watch audiences while they use a simplified, early version of a potential solution.
Try it when you:
- Have a working prototype
- Have a simple task for a user to complete
- Are comfortable sharing in progress work
Format: In-person, screen-share
Length: 10-20 participants
Length: 5-30 minutes
People: Moderator (+ video)
Timeline: 1 day – 1 week
Top Tips:
- Give the participant a clearly defined task
- Define a list of user behaviors you expect to see
- Ask participants to narrate their thought process
- Watch for moments of confusion
- Resist the temptation to step in
People like to be asked their opinions, you’re (probably) not bothering them.
Try it when you:
- Need to gather stories, situations, characters
- Have potentially sensitive or complex topics to discuss
Format: Phone, video, in-person, recorded kiosk
Length: 5-30 minutes
Number: 3-5 interviewees per type (10-25% response)
Timeline: 1-4 weeks
People: 1 interviewer (+ record audio)
Top Tips:
- Have 5-10 questions planned, but stay flexible
- Be transparent with interview purpose, attribution
- Give interviewees an “out” for difficult questions
- Ask interviewees to articulate a problem or a situation, not to find a solution
- When conducting in-person, enable interviewees to read and report their demographic profile anonymously
- Plan for no-shows, re-schedules, and “bad” interviews
When audiences interact with each other, new conversations arise.
Try it when you:
- Have audiences in-person
- Want to hear diverse or conflicting opinions, attitudes
- Want to go deep on 1-3 topics
Format: 4-7 participants per group
Length: 60-90 minutes
Number: Moderator and notetaker (+ record audio)
Timeline: 1-2 weeks
Top Tips:
- Consider power dynamics of the group
- Mix individual questions, group conversation, and an activity
- Run at least one practice session
- Ask the group to reflect or react to each other’s answers
- Summarize what you hear from the group and see if they concur
A Brief Guide to Analysing Data
- What is the purpose of the report?
- Who is the report for?
- Why is the report being written?
- Complete some background reading about the research question to help you understand and analyse the data more effectively
- Assess the relevance of the data to report aims and objectives
- What data will you include?
- In what order will it be included?
Think about grouping together related data which may form individual chapters or sections.
Choose an order which is logical and easy to follow.
When analysing data to try and answer the following general questions:
- What conclusions can be drawn from the data?
- What are the limitations in the data?
- Do certain data conflict with each other? Why?
Tops Tips:
- Summarise data as it is gathered if possible to make the task less daunting
- Create hypotheses, then try to prove them using the data
- Look for overall trends and outlines
- Cross analyse between different types of data
- Find evocative quotes for any qualitative data
Remember!
It is not enough to simply present the data that has been gathered. You must relate it to the research question which the report is trying to answer.
A Brief Guide to Presenting Data - Infographics
If you would like technical help with inputting data in order to create visual graphs, charts and tables using Microsoft Excel, there are lots of links on youtube to tutorials such as this.
What are Infographics?
Infographics can help you to:
- Turn information points into a visual story
- Increase engagement and interest of the reader
- Increase the reader’s understanding of the report and remember the key information
Before you create an infographic, it’s important to think about the following 3 factors:
Message
- What are you trying to say? What is the core message?
Audience
- Who is the main audience?
- How are they going to read the infographic? (e.g on a printed report, phone, website)
Information
- What is the essential information?
- What data helps explain your point?
- What data delves deeper into the point?
- What data adds context to your point?
- What makes your point more interesting, engaging and interactive?
- What may confuse your audience?
- Charts (e.g. annotated bar chart)
- Maps
- Flow Charts
- Timelines
There are more advanced infographics such as: (google these to see what they can look like)
- Highlighted timeline
- Circular timeline
- Vertical timeline
- Thematic bubbles
- Visual timeline
- Thematic tube map
- Thematic tree map
- Less is more! Don’t put too much information into one graphic but make sure your there is enough to prove your point
- Colour, size and spacing can highlight the most important information
- The position of an infographic on a page can emphasise its importance.
- Lines between data on an infographic can help navigate people’s eyes across the graphic to help tell its story and prove your point
- Don’t be afraid to leave white space on a page – don’t feel you have to use the whole page
- Use simple shapes and lines and very few words
- Use as few colours as possible (maximum 2-3) – it is tempting to make your report more visually appealing by adding more colours, but it can have the opposite effect
- Use simple colours
- Keep information structured and symmetrical. Try not to scatter visuals all over the page for the sake of it (you could think about using columns similar to how they structure written information in a fashion magazine, for example)
- If you decide to use icons, make sure they are consistent throughout the whole report. A great source of free icons is www.thenounproject.com
You could think about using the following online software tools to help you create infographics:
- Illustrator
- Piktochart
- Powerpoint
- Infogr.am
- Easy.ly
- Canva
- Venngage
For more information, Visualising Data is a great source of advice, information and resources on infographics.
A Brief Guide to Writing a Report
- The purpose of the report
- Name, date and for whom the report is written
- Who the report is written for
- Why the report was written
- How the report was written
- Brief description of the content of the report, including aims, findings and recommendations for action
- Only include main points, do not include any detail or points of discussion
- Chapter and subchapter headings and sections with related page numbers
- Detailed description of aims and objectives of the report, research methods, research parameters and background context to the report
- Identification of problems encountered and limitations of the report
- Summary of results with supporting visuals (e.g. charts, graphs and tables) and stories
- Analysis of data gathered and collated
- Discussion with specific reference to problem being discussed
- Clear structure using subheadings laid out in a logical order
- Use bullet points where possible
- Acknowledge information sources
- The significance of the report
- Re-emphasise the most important points
- Do not introduce any new material
- Supporting information you’ve referred in the Discussion section (e.g. stories and visuals such as charts, graphs and tables)
- Alphabetical list of published sources (there are many types of referencing techniques, be consistent with the method preferred by who is reading the report)
- If you used any sources which you did not refer to directly in the Discussion section, include these in alphabetical order in a ‘Background reading’ sub-section
- Acknowledgement of organisations and individuals who provided information, advice or help
- Clear and brief alphabetical list of technical terms and definitions (e.g. acronyms, abbreviations and standard units)
You could use the following structure to help you formulate chapters, sections and even paragraphs:
- Introduce the idea
- Explain and expand the idea, defining any key terms
- Present relevant data and evidence to support your point
- Comment on each piece of data and how it relates to your point
- Conclude by showing the point’s significance to the report as a whole or making a link to the next section
- Organise your material into sections and headings
- Write the summary and contents page at the end of the process (when you know exactly what will be included)
- Use a clear and direct, precise writing style (don’t waffle)
- Answer the research question first, then demonstrate the proof
- Report by an audience question, not by research method
- Interpret data, never just share raw numbers and stories
- Present a mix of visuals and stories
- Separate what you deem to be “clear and obvious” results from results which need further research
- Suggest recommendations (including short-term and long-term), next steps and timelines
- Prominently post notable findings in analogue form
- Conduct a review of the research methods used, if possible
Remember!
The data doesn’t matter if no one can understand it! Aim to write as clearly and concisely as possible, relate every piece of data analysis to the research question and have a clear structure; say what you’re going to say, say it, then say you’ve said it.
Resources to help you…
Training & Events for cultural professionals working with audiences – this is a fantastic source of free online training in marketing techniques.