Event Planning & Management: Measuring Event Success (PDF)

11 == Measuring event success -- evaluation insights ---------------------------------------------- -- -- - explain the purpose of evaluation and what to evaluate; - develop an evaluation plan; - use different evaluation models and tools; - understand what to evaluate; - identif...

11 == Measuring event success -- evaluation insights ---------------------------------------------- -- -- - explain the purpose of evaluation and what to evaluate; - develop an evaluation plan; - use different evaluation models and tools; - understand what to evaluate; - identify what works and what doesn't work within your event and the planning process; - assess and use the findings of your evaluations; - identify impacts and legacies of events. #### The purpose of evaluation - your team, in the planning and delivery of the event - the impacts of the event - stakeholder satisfaction. ##### Rationale for evaluation - Technical team: responsible for building the set according to the design and taking care of the interior of the conference hall. Implementation team: responsible for connecting with the keynote speakers, releasing video and content in the hall and online, video mixing, coordination of the workshops and connections and, generally speaking, the streaming. This group also included two camera operators, a sound recordist and post-event production for the on-demand videos that had just been recorded at the conference. Video switcher hardware was used to cut or fade between multiple video sources and everything in and out of the workflow was the same full-screen resolution, with an on-site AV team. - Platform team: responsible for the platform and troubleshooting any problems that the participants had, as well as uploading new content from the client on a regular basis. - Finally, there was the script producer, who made sure that everything happened according to the script and that each speaker knew where to stand and what to do. - After the brief, we knew that there was a need for a full-service platform with the following requirements: enabling - one-to-one meetings - networking - meeting rooms - chat rooms - spaces to upload posts - graphics - creating extended profiles - an Expo space for partners - push messages from the platform - and the ability to embed external sites and media within the platform. - a programme delivering great content in a variety of formats, such panel discussions, videos, whitepapers; - networking sessions around specific topics engaging both remote and on-site audiences in a co-creative way; - workshops delivering bite-size education courses; - Powerpoint presentations with graphic animations and videos; - experienced moderators responsible for guiding both online and on-site attendees throughout the event; - simultaneous translations to overcome language and cultural barriers; - and finally, an experienced digital event strategist and live-streaming producer. ##### Event objectives ##### Evaluation and the events team's role ##### What to evaluate #### Developing an evaluation plan - Event purpose - Background to the client and the event - Description of the context of the event - The overall (strategic) aims of the client organization and of the specific part of the organization procuring the event - Identified and named roles and responsibilities - Objectives of the event and how they relate to the overall strategy of the client organization - Event branding, visual identity (including online) and fit with the client's brand and values - Key audiences for the event, segmented and described (e.g. in terms of attitude) - Key messages for the event, applied to audience segments - Communication channels, applied to specific audience segments - Internal (client organization) communications, including how information will be cascaded - Activities required and their relationship to organizational objectives - Communication dissemination plan - Key tasks and allocated responsibilities (on the client side and for event manage- ment) - Resources required and dependencies against a timeline - Event timing ##### Evaluation timeline ##### Evaluation models and tools #### Online event management software - a quick and easy release of template-driven evaluation questionnaires sent via an email direct to event attendees, using a set of core questions with the option to add supplementary questions, to evaluate speakers, workshops, sessions, options; - a chase email option that can be sent only to those participants who have not already completed the online evaluation questionnaire; - a manual completion option for users who return their evaluation separately by email or post; - the option to communicate directly with different types of event participants, from delegates to speakers and exhibitors, suppliers and clients; - immediate anonymized aggregated statistics and comments that save time and enable quick assessment of your event's success; - the ability to issue a follow-up evaluation so you can measure knowledge change and the longer-term effectiveness of your event on attendees; - the availability of reporting, from a brief summary to a detailed report. #### Evaluation issues - Anonymous or named responses: anonymity often inspires respondees to be more open and honest in their responses -- but they need to be able to trust that their -- -- -- -- -- -- - Online or paper? Surprisingly, some organizations continue to use paper forms handed out at the event, often 'encouraging' completion by responding with a free gift when the form is handed in. However, a paper format requires typing up the content, an onerous and time-consuming task. While online formats are immedi- ate, they may get lost in the attendee's email inbox or be discarded as spam mail, and response rates are never as high as those 'coerced' by free gifts. However, there are advantages, in that some people prefer to reflect on their experience and may make a more considered submission after the event. Online systems can accu- mulate responses automatically, providing initial analysis of anonymized data, relieving the event team of the task of manually inputting data. Some online response systems can also identify who has not responded and chase them up at a later date, as well as sending subsequent questionnaires on different topics. - The importance of developing an ongoing picture of event delivery through consistent evaluation cannot be overestimated. Online systems provide the ability to compare and contrast evaluation and other reporting measures -- such as attendance rates, no-shows, booking lead times -- that can be used to improve and enhance delivery of any event as well as specific events in a series through the analysis of trends. Feedback should not only remain with the client or events team, but event participants should be informed of changes that have been made as a result of their input. ##### Online and social media evaluation ##### Identifying what works and how to make improvements in your event planning process - What has worked well? (and how can we improve it?) - What hasn't worked well? (and how can we learn from it?) - What issues, ideas or concerns does this raise? (and how can they be progressed?) - the planning stages of the event; - immediately pre-event; - during the event -- on-site; - at the end of the event; - post-event after a gap of some months. ##### Examples of questionnaires and questions - Event participants' experience - What they have learnt or how much - How they will change what they do as a result; what specific action will they take? - Relating to the specific event objectives, whether directly or indirectly - Whether the event was memorable or worthwhile (you would have to word this question carefully to get a useful result). Question type: - Yes/no - Yes/no with don't know - Scale 1--5 - Scale 1--10 - Rank options according to preference - Multiple choice -- single answer - Multiple choice -- multiple answers - Free text (comment box) Is the question mandatory? - If a question is mandatory, this could lead to non-completion of the questionnaire. Consider the wording of the question itself: - Open/closed questions - Single question not multiple - Specific - Measurable - Does it relate to the event objectives (explicitly or implicitly)? Demographics - Consider how you want to be able to analyse your event participants -- -- -- -- -- -- ##### Analysing and using the findings of your evaluations  Usefulness Presentation  Content **Knowledge and skills: averages** ------------------------------------ -- -- -- -- -- -- -- -- -- Access Venue 4% Neutral -- --- --------- -- -- 1 1 #### Impacts and legacies of events ##### The triple bottom line: environmental, social and economic sustainability - Team meeting - Product launch - Internal sales kick-off - Roundtable - B2C fan event - Virtual exhibition - Virtual conference - Platform -- get it sorted from the beginning - Virtual vs hybrid? - Venue - Staging - Film crew - AV -- live-streaming content into the platform - Pre-recorded content or live on the day - Guest registration - Event moderators -- where do guests go if they have a problem? - Engagement -- polls, questionnaires, breakout rooms, networking zones etc - Bringing remote guest speakers onto 'stage' - Exiting the event - Collecting data and reports after completion - Consider how you can improve your event planning and delivery processes, how you engage with suppliers as well as their performance -- and yours -- by identifying what works and what improvements you can make. - If you build in improvements and reviews as part of the way your events team works, you will find that their creativity focuses on getting better at what you do -- and - Large events focus on evaluating the long-term benefits to the local community, the environment and the economy. 1. In events you have attended or worked at in the past, what evidence of evaluation have you seen? 2. What three things would you choose to evaluate and how would you evaluate them? 3. How would you build in evaluation to your next event project?