How to Use 'How Might We Sessions' to Rapidly Generate A/B Test Ideas

By Simon Witton

11.05.2020

One of the regular activities of an optimisation team—whether in-house or an agency—is to generate ideas and hypotheses to conduct A/B tests. And one of the most valuable tools in generating these ideas is a How Might We session.

At Endless Gain, we use How Might We sessions in our process to generate lots of experimentation ideas in one concentrated time slot.

What Is a How Might We Session?

A How Might We session (HMW) is an idea generation tool, often used within UX design and product teams. The session enables a small group of stakeholders to explore a potential problem in depth and rapidly generate ideas on how to solve it.

HMW sessions typically involve between 3 and 10 stakeholders and centre around an HMW statement.

The idea is simple: take a problem, turn it into an actionable HMW statement, and allow the participants to ideate ways to solve it. When done right, HMW sessions can generate dozens of workable ideas within 1-2 hours.

When to Conduct How Might We Sessions

HMW sessions require research to give them a focus, so they must always be conducted after your primary and secondary research is completed and you have enough data and insights to tackle a problem realistically.

When to conduct how might we sessions

Image credit: Dan Nessler

At Endless Gain, HMW sessions follow our ‘Discover’ and ‘Define’ phases. The output of the sessions are experimentation ideas, which should pre-date the majority of the ‘Develop’ phase.

HMW sessions are collaborative.  Our UX Researchers, Optimisation Strategists, Designers, and Developers all get together with our clients to ideate around the defined problem.

By having a broad range of stakeholders in the room, we get a wide range of viewpoints. By having multiple viewpoints—of people who are very close to the problem and of those who aren’t as close—it creates a great balance with which we can generate a wide range of strong experimentation ideas.

How to Conduct a Productive How Might We Session

when to conduct how might we sessions

The research phase of the
optimisation journey we create for clients

The first step, as always, should be research.

At Endless Gain, we gather tonnes of data from multiple research points: Analytics, Session Recordings, User Lab Videos, Customer Surveys, Accessibility Audits, and Biometrics (see diagram above for full list). We then cross-reference this data to identify key pain points for the customer within their journey on your websites.

Once the pain points have been identified, take one of the key friction areas and turn it into an HMW statement. HMW statements are called so because they follow the format of ‘How might we solve x issue’.

For example, if our research told us that only 8% of customers who reached the PDP progressed to the ‘add to basket’ page, the HMW statement would be:

‘How might we encourage more users to add to basket?’

This HMW statement is then taken into an HMW session where we explore the problem together and use Post-it notes to capture each idea. For the final part of the session, we discuss all ideas generated as a group and categorise the ideas for easier implementation later.

To summarise, this is how an ideal HMW session would look:
how to conduct how might we sessions

 

6 Tips for a Successful HMW Session

1.      Get Your HMW Statements Right

The key to a successful HMW session is to create clear and specific How Might We statements. Vague and broad questions will not give you the specific ideas you need to solve the problem at hand, while hyper-focused questions will narrow the scope of your ideation.

Avoid negative statements; make them positive, action-oriented, and customer-focused. You’re solving a problem, not trying to pin the blame on anyone or anything or forcing customers to do things they wouldn’t want to do normally.

Another way is to concentrate on removing the negative emotional experience of the customer. This might sometimes work better than trying to enhance the positive emotional experience.

And importantly, do not make the statements long and convoluted, or the first 15 minutes of your session will be wasted in simply explaining the question!

This set of suggestions on HMW statements from Stanford University’s d.school is a great way to ensure you get your questions right.

Here are some examples to put you on the right track.

✖ How might we make users know that our clients’ site is authentic? (too broad)

✔ How might we increase users’ trust towards this website? (specific)

✖ How might we increase sales? (not research-driven)

✔ How might we decrease drop-offs during checkout? (action-led and research-driven)

✖ How might we redesign our PDP? (Not action-oriented)

✔ How might we increase the number of users interacting with the action area on the PDP? (specific and action-oriented)

✖ How might we get more credit accounts? (too broad)

✔ How might we encourage customers to pay by credit? (specific but open)

2.      Create an Agenda and Share the HMW Statement to the Group at Least Two Days Before the Session

Not every participant may be familiar with the format of an HMW session or even been part of any idea-generation session before. Therefore, sending out an agenda beforehand will allow people to be aware of the goal of the session and prepare for this in their own time.

It also allows everyone to research around the problem and search for solutions to the problem before the session. This will allow a maximum number of ideas to be generated within the short space of time you have.

3.      Keep the Session Short and Productive

Don’t go into an HMW session with the mentality of ‘getting it over with asap’. At the same time, don’t drag the session out so much that everyone is bored out of their minds. We find that a one-hour session focusing on one HMW statement works the best.

The onus of keeping the stakeholders participative and interested lies with the person who heads or moderates the HMW session.

This person should be able to understand if a discussion is dragging on or if sufficient attention is not given to certain aspects of the problem at hand. They should also instil the right expectations about the session among the participants before the session is conducted.

4.      Start Your Session with a Short Recap of the Research

It is entirely possible that at least one of the members of the HMW session isn’t aware of any of the research—maybe not even the key research that led you to the session!

So, starting with a quick 5-minute recap of the problem at hand and the research that identified this problem, keeps everyone focused on the issue and ensures everyone in the room has all the information they need to participate productively in the session.

Also, who doesn’t need a refresher every now and then?

5.      Emphasise on Quality Rather Than Quantity

While it’s okay to let everyone think up creative solutions to a problem, focus on getting quality ideas from each participant, rather than a large number of ideas.

It’s fine to discuss a lot of things, but when you compile the final list of experimentation ideas for circulation among the team, the ideas in there should be actionable, backed by research or psychological principles instead of guesswork, and testable.

6.      Don’t Restrict Yourself to the Specificity of a Problem

Just because a problem was discovered on a certain page doesn’t mean that the HMW session should be solely focused there.

It is well known that other areas of the site affect customers’ behaviour, so allowing participants the freedom to explore the problem without constraint could unearth some larger and more unique ideas and still solve the problem. It may even unearth the larger cause of the problem.

The Outcome of Our HMW Sessions

Results of how might we sessions

Share the ideas generated
in HMW sessions

The output of our HMW sessions are experimentation ideas and sometimes JDIs (just do it). Whether initially creating the Post-it notes virtually or the old-fashioned way of taking notes, we always transfer them to an online whiteboard tool such as Miro.

This allows for better storage of the ideas and gives you the flexibility to easily move, group, and colour code the ideas into categories (see the image above).

Once the ideas have been sorted, our team of Optimisation Strategists take them, flesh them out, validate them with our supporting research, prioritise them, and insert them into the design sprint.

Our designers then take these test ideas and create experiment designs from them.

Here is a test we did for one of our clients, where, based on ideas generated from an HMW session surrounding the PDP, we developed a full-page redesign addressing the issues found during our research.

The session was called “HMW encourage more users to add to basket?”

We made the following changes in the Variation:

  1. Increased the visual prominence of the images: Heatmaps showed that images were the most interacted element on the page, and in lab sessions and on-screen recordings users spent most of their time on the PDP interacting with images. So, we gave the image area more visual prominence and solved usability issues around image handling, particularly zooming, by adding a clear zoom button on the image.
  2. Created a much clearer goal above the fold: By changing a three-column page to a two-column page and moving additional product carousels down the hierarchy, the primary objective—to add to bag—was made much clearer for the user when they first land on the PDP.
  3. Introduced triggers to the action area: We took the three most important issues to the customer—stock level, delivery, and returns policy—found through website surveys and added that messaging into triggers right below the primary CTA to encourage the user to add to bag.
  4. Completely redesigned the stock messaging: Our research highlighted this was a particularly confusing stock system for the user. So instead of giving this information upfront, we served up the stock level of the product as the user was selecting their size and colour, making the information relevant and clearer.
how might we test wireframe

Control (above) and
Variation of redesign

The experiment was successful across all devices.

Of course, the test ideas generated in our HMW sessions don’t always lead to full-page redesigns; some are smaller, more iterative test ideas. But the value HMW sessions have is that they allow you and your teams to think without constraint. It is definitely one of the most valuable tools you can use for optimisation, whether you are an in-house optimisation team or a CRO agency.

Simon Witton
Hi I'm Simon, Senior UX Designer at Endless Gain. I design experiments, user flows and prototypes that focus on improving the experience for the user and conversion rates for our clients. I am passionate about all things creative, from design to video games and music.

Our Clients