Exploring the Art of Designing Feedback Loops for B2B Events with Alex Temple of Explori

Ketan: What should one consider when they're designing feedback?

Right.

What are the things to keep in mind?

Alex: I think if you strip it all the way back the most one, the first thing that I would always kind of advise people to do when, when they're thinking about their their event, feedback would be think about your objectives first.

Why are you putting the event on?

Right.

What's the purpose of the, what are you as the planner or the organizer trying to help your attendees achieve?

Right, Because that should be the goal.

Is that there?

Then you can start to look at right, what are the metrics that are important to you?

For instance, if you take a corporate event, if it's an internal training event, then asking how likely you are to recommend that, Sorry, how likely you are to return to that event is doesn't mean anything because it's a mandatory company event, Right?

That metric doesn't mean anything.

But really what you want to understand is what, you know, whether people as a result of attending that event and that experience may feel more positively towards the company or the brand as a whole.

Right?

That would be a great metric or or something to put in your putting the survey.

So, I think the first thing would be to think about your objectives, design your questions around how can we understand how people have met the objectives that we, that we want them to meet and then designing the survey from there and cutting out things that don't actually matter.

You know, surveys are kind of or feedback forms historically have got a bit of a bad press and you know, and generally treated in Yeah, exactly.

You know, you laugh, but it's true.

And so they're generally treated as like tactical exercises for marketers to go, oh, well 20, you know, 90% of people were satisfied or you know, so and so many people said that it was a great event or whatever it might be, right.

Whereas if you ask the right questions as I was talking a minute ago about forward looking metrics, you can, you know, it's such a rich data source and it can be so strategic for your business.

So asking the right questions is is really, really crucial and thinking about, you know, what you're actually going to do with the answers.

You know, don't ask the question that you don't either want the answer to or you don't, you're not going to use the answer to.

So a lot of people will talk, will ask questions about, you know, you know, whether the wifi was good or whether the food was good in some instances, yes, you know, it's it's important to ask those questions, but sometimes it's just not, you know, attendees don't, you know, there are things that people that you can't change, right?

Ask questions about the parts of the experience that you as the planner can influence and make changes towards that, that would be, you know, kind of a must have for me.

Yeah, so what I'm hearing is less, is more one, you've got to try everything back to the goals that you have, putting that event in the first place and it's all right not to ask about every little detail, you've got to pick the top five, top six, whatever you number that you have, and you have to be like, you know, I'm gonna let my myself to 55 questions around this, or 10 questions around that or even limiting the survey to, you know, no more than 20 questions because you'd be surprised how many questions people are willing to answer if they're engaged with the experience and they believe that their feedback actually matters.

So, you know, in terms of actual delivery, deliver the survey as soon as soon as you have delivered the event after the event and then also communicate your findings back to your audience because for me, you know, how many times have you been to, like you said, you went to an event, the survey was on paper, you never heard from them before.

Even if you went back to that event, you have no belief or trust that your feedback actually mattered and, you know, experiences and events are designed or supposed to be designed for the people who are attending them, right?

So they're part, you know, if you can help people engage with the feedback process, they feel like they're part of building that experience and building that event and they're more likely to give you not just more feedback, but more honest feedback, which is, you know, the organizer and the planner is going to be immensely valuable for you when you're trying to, you know, curate that experience year after year.

So yeah, there's a couple of different points there.

Yeah, just just double clicking on what you said about figuring out what's important.

Right?

Is there such a thing as an absolute must have on a survey question or does that completely depend on the goals of your event and the format?

, is there an absolute must have, I think for us, I would say that always have a question around satisfaction.

So overall, how satisfied were you with the event?

net promoter score is a great metric.

It can be a little bit of a red herring sometimes.

So I would, I would always use it as part of a variety of metrics to use.

Don't just use that one by itself because NPS is a measure of advocacy, right?

How likely you are to recommend the event and how likely you are to recommend the event is not always indicative of how successful that event has been at meeting and attendees objectives or the planners objectives.

Right.

So I always use it as part of a suite.

So probably things like overall satisfaction.

Net promoter score.

If you're a trade show, using likelihood to return to the loyalty question.

If you're a corporate event where it's more mandatory if it's internal than using a metric value for time can be really, really valuable for you to use.

So yeah, those would be the ones that I would probably look at the most.

Yeah.

And so I used to work at a startup where we had built a tool for measuring nps.

Right?

And the biggest problem like you said with measuring nPS is that Justice Court doesn't tell you anything, right.

, so we started building an analytical model around the text that comes the comments that come along with that accompany the score, right, Which is actually where the meat of the matter is, Right.

so we did find a lot of value from that.

But I imagine for a larger event where you've got tens of thousands of people like an MX, for example, running and I am running an NPS plus with the comment section, that's a huge undertaking.

How does somebody go about analyzing feedback like that?

, that's yes, it's an interesting question.

You know, I met fully enough, you know, that, you know, one of the, one of the events that we work on and they, you know, they do a phenomenal job is a, is a brilliant brilliant events and they take their feedback really, really seriously.

And I think the way that, that they go about it is, you know, making sure that there just taking the time to dig into the, you know, the scores, if you just take mps, for instance, right?

You've got your detractors scoring 0 to 60 passes in seven and eight.

And then you've got your promoters in nine and 10.

Now, just because somebody scores for us, you know, perceptually scoring a seven or an eight out of 10 is actually pretty high score, but you kind of get cut out of the, you know, the analysis, I suppose, or at least the overall score, because the net promoter score is detractors minus, sorry, promoters minus detractors of the difference between the two.

So, passives are kind of like, you know, supposedly that sat on the fence, so they don't make much of it, But but not always, that's not always the case.

There are, you know, there's Softwares out there that allow you to you to, you know, put in text and then they draw kind of conclusions based on, you know, language that ai software that does that machine learning software that kind of draws out themes especially.

So, you know, that's that's something that you could do if you've got a huge amount of a huge amount of comments.

The other thing would be to actually dig into the score and see, right, Do I have like, a really high percentage of people who scored six, right?

Still class in the detractors, but, you know, that's their five points ahead of somebody who scored one.

That's a that's a big difference, figuring out, Right?

well, maybe I just have a large group group of people who actually would consider themselves passives.

So let's have a look at people who scored six, let's have a look at their comments, and if we can bump them up next year into your seven and eight, you know, and your sevens and eights and nines and tens, then all of a sudden you've got this massive switch swing upwards as a positive mps.

but yeah, I think that would be that's that's one of the things to to dig into, and you do just have to take the time.

But like, with all of these metrics, it's important to have context.

This is why mps is can be quite a red herring, if you don't know what good and bad looks like, because, you know, you might have your mps for an event might be minus 10, right minus.

Typically you go, oh, that's not good, you know, I'm in the negatives, but benchmark is minus 20 Right?

And you're scoring minus 10, and that's that's amazing, you're 10 points above the benchmark, which is incredible.

But if the benchmark is plus 30 and your minus 10, well, okay, now there's a big difference, but, you know, that's why.

And the other thing is because of the mathematics of of how mps is structured, it can swing massively.

So you do see these events who will go from, you know, like a plus 22 a plus five, you know, in a year just because a proportion of their audiences shifted from like seven and 8 to 6 to six for instance, so yeah, it can be challenging, but that's why I said use it as part of a suite of metrics that measure every aspect or different aspects of the event experience and then kind of bring those together so that you can actually look at it from a holistic viewpoint and and should we look at incorporating feedback at every stage of the event and by that, I mean, do we need to collect feedback at the pre event during event?

Post event phase.

I mean it depends on what probably I would say, not always because if you're gonna use feedback strategically, you need to get feedback from people once they have absorbed and part of the entire event experience, pre event, nothing has happened.

The data that you can collect pre event would be around pre registrations or you know, things like that, but again, that only is indicative of like how many people are kind of visiting or due to visit, come to the event then you don't you in event feedback.

I mean everybody for traitors definitely, you know sales people will walk around the floor and chat to their clients, their exhibitors there and you know, kind of get feedback on the floor, which is always a good way, you know that's also a relationship building exercise, so people, you know, you get a little bit of facetime, but again, if you're gathering feedback at an event, whether it be a corporate event or whether it be a trade show and, you know, if it's the it's the morning or the afternoon of the first day, for instance, then somebody might have had a fantastic time at the end of day one, but by the time they finished day three, well, it wasn't that great.

So it skews your view a little bit, that's the challenge.

We're doing any kind of on site research is that it's fantastic to get like an in the moment and viewpoints of it, but people haven't had a chance to experience, to digest and absorb the entire experience.

So, their viewpoint on day one would be different from day three and maybe different from day four, which is when you would send out a survey, if it's a three day event.

So yeah, there are some complexities around doing, you know, gathering feed, specifically feedback at, on site and event, gathering data at an event in terms of tracking data.

That's a different story because, you know, we have the technology now to track, you know, where people go within the event and you can use that to enhance your your viewpoint and enhance your analysis because you can link it to your feedback greater afterwards to see if let's say if certain people attended a certain session, whether they were more than more or less satisfied than somebody who had attended a different session for instance, Now that's really, that's really key inside.

But even still it's still dependent on you delivering or gathering feedback, post event.

So I think feedback, you know, pre and in event not the most strategic, you know, asset to you.

Post event feedback is by far and above the most strategic asset you have for an event organizer or an event planner, awesome.

So I imagine this is where feedback, I mean over feedback starts becoming annoying as opposed to useful.

Right?

So there is such a thing as too many questions, too many, too much feedback or asking too much for too much feedback.

Yeah, it goes back to what I said before, you don't ask questions that you don't want to know the answer to like everything, you know, event organizers were in such a privileged position.

We have so much data at our fingertips and we've been, it's a problem that we've been known in the industry.

I imagine you, you know, it, you know, we deal with data all the time if you have too much data and you don't have a plan of how to use it or it's just it's not joined up, it ends up scrambled, somebody looks at that and goes, there's no way I'm spending my time trying to touch that because it's just not worth it.

And so yes back can feed back data if collected inconsistently, especially across.

I mean if you take a corporate event, you know, team, that program, let's say you've got 500 events a year.

If you're not collecting feedback consistently, you're not asking similar questions on the same scale.

You know, if you're asking five point scales here or four point scales over there, you can't compare anything and therefore the data just gets style owed, you know, in it's it's useless, right?

So in that vein, yes, too much can be a bad thing.

but it's only a bad thing if it's if you're collecting it inconsistently and you're not you don't have a plan to use it.

so, but if you're using if you have like a proper framework for collecting feedback, you're doing it consistently and you can control that consistency across all of the events that you're that you're planning or organizing, then it doesn't matter how much you get, in fact, the more the better from from a from a data perspective, the more feedback you get, the more robust data set is, the deeper you can go on your analysis, the better the insight and that insight ultimately is what's going to drive your strategy moving forward because you've got a robust data set to win those arguments to make those decisions defend those, but and so on and so forth.

So that's a really, that's a really good thing when it comes to asking questions.

Well, yes, if you ask 1000 questions on the survey for a half day event, you're not going to get anybody to respond.

So that that's not good.

Right?

But be smart about how you ask your questions again, only ask questions that you want to know the answers to that.

We're actually gonna be useful for you.

So yeah, so yes and no, there is a point where it gets annoying but over overall, I'd say the more data that you have, the more feedback data you have assuming it is collected correctly and consistently will be, you know, it's one of the best assets, you know, in your arsenal for an event planner.

For sure.

You couldn't incentivize me enough to answer more than five questions incentives work.

I mean this is this is the thing, right?

So there's an argument to say that incentivizing people on a survey can skew the results because people might just go click click, click click click gift card.

Right?

Well, yes, there is part of it that there is some science to suggest that.

But in general, if you incentivize people, it does mean you get a better response rate.

It means that people are gonna complete survey.

So therefore gives you more data and more data as we know, it means a better, better level of insight for you to to using your strategy.

So I think it's just about choosing what is going to incentivize you getting, I mean, maybe a £10 amazon gift card isn't gonna do it.

So I don't know what would you, what would you want to be incentivized by?

What would what would get you to answer a survey?

More than five questions.

That's interesting.

Maybe a higher denomination coupon or maybe a bottle of wine.

That's always a good incentive.

Yeah, there you go.

I mean, and so anybody who's running an event and you're their target market, you know, this is this is you we understand you very well, we know you don't want a £10 gift card, we know you want, you know, a chance to win this fantastic bottle of wine.

Right, okay.

Yes, that's gonna get me that's a good incentive.

I think on the, on the topic of incentives planners and organizers know their audience way better there than any other kind of research firm or any other kind of survey provider could ever do.

We have, you know, for us, we have data to back up, you know, some suggestions we have, but, you know, your audience.

So to something, if you're going to incentivize and choose something that you believe they're actually going to be incentivized by.

So how soon should people act on that feedback, how soon does that feedback start losing value or does it ever lose value?

that's a that's a really good question.

I think that Yes, the value will ultimately decrease the further away from the event.

It is, right?

So feedback from an event in 2019 is not gonna be as valuable as feedback tomorrow from this webinar.

For instance, if you're looking at it, unfortunately feedback wants, it has been implemented into your strategy.

So you've had your post show wash up meeting, you've gone, this is how the event went.

This is our feedback.

You know, we'll mention the feedback loop in a minute.

But once we've actually implemented changes based off what our attendees said, so off the feedback, then you've kind of, you've squeezed as much as you can out of that out of that data set.

Now the other, you know, the other side of the coin is that when tracking performance over time.

So if you're looking at the metrics, not necessarily the open ended part of the feedback questions, but if you're looking at your core metrics.

So things like, you know, overall satisfaction, net promoter score about you for time, loyalty.

If you're looking at behavior and sentiment impact scores as well, things that can be directly measured over time, then arguably that those, that data doesn't lose value.

It actually grows in value because if you're tracking it over time, you might be able to generate your own benchmarks, internal benchmarks because you've got everything consistent, but the very least you'll be able to track your performance over time.

So if you've got historic data, let's say if you just pick one metric, if you just pick net promoter score, you can track that over time for let's say the last three years or last four years, let's say two years pre pandemic in pandemic and then now you've got a historic analysis so you can see where you came from before the pandemic kind of hit and changed events for forever.

I think arguably to where you are now that data from 2018 2019, Beginning of 2020, that's gonna be massively valuable for anybody who's running an event now because they can see where they were, they can see what happened in the middle and then they can see where they are now.

So, and that helps we're telling the story of how of your events, of the success and how people's expectations have changed.

You know, we've seen it With net promoter score specifically for the expo function, you know, for I think it was like plus 10 or plus 12 or something like that for attendee net promoter score pre pandemic.

It then obviously trade shows didn't happen.

So plummeted the response to the virtual events was horrendous for some and phenomenal for other massively volatile.

But then coming out at the other end of it now it's coming back onto an even keel.

So you know, we're able to take that data and look at it over time.

So when you're looking at that perspective, it probably increases in value when you're, when you're looking at it, when you're looking at your scores and your performance over time.

But then there is, you know, once you've used it, I suppose in your in your strategy then most of it is then not redundant, but it becomes, you know, dead data really.

So I suppose the answer to that is yes and no.

Yes.

Some questions like improvements will be redundant after you've used it in your strategy moving forward and once the next event is run, well you don't need to use that anymore.

But your key metrics that you're tracking over time, you know, they can increase in value over time because it helps you tell the story, helps you know where you come from and and also predict your future success.

I can actually see a research report come out from just that answer.

It it's going to be a guide and you know what I'm going to bring out.

Thank you for that.

That's deep.

so for somebody starting off building a feedback and doing webinars, Right, Simple event.

do you have any any rule of thumbs or any advice of how they should think of building that feedback?

I think the simple answer is just to start, you know, when you, you know the there's a formula, a methodology called the feedback loop.

Right.

And so that's five stages and most people do the first two stages which are to collect your data, which is simply running a survey and analyzing that data.

So looking at it and seeing if you can draw any trends.

Right.

Step three is to share your results internally with the business so that you can communicate how well that event is done or how well they haven't hasn't done, then you need to make decisions.

So step forward to decide on what you're going to do with that data.

What are you gonna do to change the event to improve it based on the feedback that you have received?

And then step five is to actually implement that actually in the next edition of the event, make the changes that have come out of the research and come out of the analysis you've done of your feedback.

And so what we talk about in in research is closing that feedback loop, closing it from, you know, one stage 12345 and then repeating that process all over again.

So you end up with almost like a virtuous circle of feedback, right?

You collect, you analyze, you share, you decide you implement and then you go around that's that loop again, because once you do that And you know, you start off small, you don't have to do it with every single event.

If you've got 500 events, it's going to be hard to implement this process in one go to start off with one edition of an event or 11 business line or one portfolio, whatever it might be.

And yeah, that that would be my advice is just to start and then focus on closing that loop that loop, because the other thing that we get asked quite a lot is, you know, what's the value of completing surveys?

Do people complete surveys?

And as I mentioned at the beginning, when people feel like they are part of building that experience, they're gonna give you better feedback, they're going to give you more honest feedback and you're going to get more people that feedback and the way to drive response rate and drive engagement with surveys and feedback is to complete that feedback loop once you have decided that you're going to make a change based on what your audience said, tell your audience that that's what you're going to do.

Don't hide from it.

So you said this.

So let's so we're gonna go and do that as a result of this and then shout about the changes that you make because when people turn up to an event and they realize, oh, hang on a minute I have actually been listened to then if something goes wrong or something or something goes right at the event and they're gonna want to share that with the person who put that event on and it just, it builds trust.

So yeah, that would be the starting place, you know, that would be for me, where to start is first of all just to get going, start, start measuring and as best you can try and be consistent with at least your core questions so that you can track over time regardless of kind of what you know what type of event it is.

Always try and ask your key metrics, you know, things like your satisfaction, your net promoter score your, you know, your value for time questions.

If you try and ask, you know, three or four of those key metrics on every single survey in the same way and make sure that they're always on the survey, then that's going to give you a huge, hugely robust data set to start drawing kind of long term strategic conclusions and actually start to evaluate the future health of, you know, your event assets in your event programs.

So that was kind of long term.

But yeah, if you can, if you can just start with, you know, trying to close that feedback loop and being consistent in the way that you ask questions, then you're doing more more than a lot of the, you know, a lot of the industry at the moment.

‍