Conducting surveys to test layouts

Design is one of the tools that helps Dodo solve business tasks. We use it to convey messages to the audience: we tell customers about new products, promotions, and features in the app. To check whether our message is clear and if a particular layout solves the task at hand, we collect feedback from the audience by carrying out surveys.

The survey enables the perspective through the eyes of people who encounter our ads in the street or inside pizzerias. It gives us an impartial point of view, which is hard to obtain from the team itself.

Designers are familiar with the brand and the product, and they also perceive communication differently: they notice the technical aspects of the design, but cannot always assess whether the idea is clear to a person who sees a new pizza for the first time and knows nothing about the brand values.

When we use the survey method

We don't put every layout to the test. We have templates and a clear algorithm for solving regular tasks. There are 3 situations which require a survey:

1. There is a task that we have never faced before, and we need to come up with a solution from scratch. For example: we announce a new unique product that has not been on the menu previously.
2. We do a significant special project. For example, we’re launching a collaboration with partners.
3. The team can’t reach a compromise. For example, the team members have disagreements about visual solutions or the idea itself; we can't figure out which layout excels.

When to conduct the survey

Conceptualization. At this stage we choose the idea that will form the key message of the layout. For example, we plan to start selling croissants, and the team has an idea to draw a cat in a red beret on the layout, because it will bring associations with French pastries. To test the hypothesis, we show people the sketch and use a survey to find out what feelings or associations it evokes.

Assembling the master layout. We already have an understanding that the idea is clear enough, so we move on to working out the details: we try out different graphic techniques and refine the wording. We can show people two mockups with different accents if we want to see which one conveys the message more accurately.

Survey algorithm

In order to conduct an effective survey, we need to understand why we are conducting it in the first place and formulate the questions in a way which will not lead participants to giving conventional and socially approved answers. So let's pay special attention to the first two items of the algorithm.

1. Clarify the task

   At the stage of discussing the brief, we clarify whether we are going to conduct the survey on a layout. If so, we check whether the objectives are clearly defined and if they are suitable for evaluation by the survey method.

   It takes an average of a week to conduct and process the results of the survey, so we start all the preparation beforehand. We check if the plans have not changed with the owner: what task the layout should solve, what result we want to achieve. Based on the goal, we select the audience and formulate questions.

2. Compose a questionnaire

   There are three parts to a standard survey: demographics, filtering questions and main questions.

   • Demographics is needed to collect socio-demographic data about the audience: gender, age, place of residence, income level, and so on.
   • Filtering questions help to narrow the funnel and to select respondents whose answers we are most interested in. It makes no sense to ask a person who has never heard of Dodo Pizza whether he or she recognizes the company in a layout without a logo. That's why we first ask multiple-choice questions and identify the target audience: people who eat fast food, use delivery service or know the brand.
   • Main part is the questions about the layout itself. For example, if we want to check whether we managed to highlight a unique ingredient or promotion, we show the layout for a few seconds and then hide it and ask what the person remembers.
3. Approve questionnaire with the team

   Show the questions to the owner so that they could add something or adjust the survey. If we aren’t short on time, ask for help from copywriters and editors: examine the text for errors, correct the wording.

4. Create online survey

   We use the Anketolog service. The service has access to a large database of respondents, and they can be sorted out by customizable parameters. For example, we can conduct a survey only among women with children or among residents of small towns. It takes 1-2 days to collect responses.

5. Analyze results

   We get the results of the survey in the form of an excel spreadsheet with respondents' answers. Typically, we carry out rather basic surveys, so the designer only needs to look at the spreadsheet to process the results. We look at the answers percentage and conclude whether our hypothesis is confirmed.

6. Share results

   Sometimes we don't just confirm our hypothesis, but make some discoveries: for example, with the help of a survey, we find out that if there are several brightly coloured plates on the layout, they interfere with the information we put on it. This is an important insight.

   We save the results of each survey on Notion. We also share interesting findings in the @design_team channel on Loop, so that the team can take note of important conclusions and use them in other similar tasks.

Read next: