Using a wizard to help people get answers during COVID-19: Research summary

Date: March 19, 2024

For Canadians and those hoping to visit or immigrate to Canada during the height of the COVID-19 pandemic, travel restrictions were complex and constantly changing. Finding answers about travel was one of the top COVID-19 tasks visitors were attempting to complete online. Although lots of information was available on Canada.ca, finding the right information and synthesizing it into clear answers was difficult for individual travelers. Travel, transport, and immigration are handled by multiple departments, which complicates the task of finding all the information relevant to one's own particular situation and travel plans.

We conducted usability testing with potential travelers and found that synthesizing complex travel information into an interactive wizard helped participants complete key tasks. A wizard is a way of guiding people to the answer they need through a series of interactive questions. Participants using the wizard arrived at the correct answer to questions about travel restrictions more often than participants who were constructing answers from information that is distributed across pages managed by different institutions.

Try out an early prototype of the wizard to see how it works.

Wizard prototype: Find out if you can travel to Canada

Content design in a crisis

Starting in March 2020, Digital Transformation Office researchers and designers worked with the Treasury Board Secretariat, Canada Border Services Agency, Global Affairs Canada, Privy Council Office, Public Health Agency of Canada, Health Canada, and Immigration Refugees and Citizenship Canada to improve and consolidate all COVID-19 content on Canada.ca. The goal was to make answers easier to find, easier to understand, accessible, and mobile friendly.

By September 2020, vaccines were not yet available, and ongoing border restrictions had made the travel situation more complicated and top of mind for Canadians who needed to re-enter Canada. Call centers and Public Health Agency email inboxes were receiving a high volume of inquiries with answers that were available on various pages on Canada.ca. Initial usability testing and website feedback highlighted struggles experienced by people navigating web pages, trying to figure out which rules apply to their circumstances. Responses from the COVID-19 task success survey showed that the average reported success rate for travel-related tasks was 59.7%.

The studies and survey comments confirmed key challenges with the live content: participants struggled with duplicate and scattered content, long pages, and missing or misleading information. This combined evidence indicated that many people were failing to arrive at accurate answers to their questions, which could result in being turned away at the border or potentially exposing themselves and others to heightened risk of infection.

Our ultimate goal was to reduce errors for travelers arriving in Canada. We could do this by making it easier to get answers, thereby lessening the burden on department staff by reducing calls and emails about questions that could be otherwise answered on the website.

Design solution

Initial usability testing gave us some insights about what travelers struggle with when understanding travel restrictions on the current website.

When looking for an answer to their question during testing, participants would get overwhelmed by all the possible links that could lead them to what they were looking for. For example, from a participant's perspective, information about quarantine requirements could be found behind the link labeled "travel advice" or it might be behind "prevention and risks." Some pages would contain a partial answer, but still require piecing together information from multiple pages to form a complete answer.

I couldn't really find where I could find a specific section of information on the site.

We also observed that participants would skip past the legal content on a page even when that content was relevant to their situation. Government policies often have legal considerations attached to them, such as the consequences of not adhering to certain laws. Depending on an individual's particular situation, missing a key piece of legal information because they are overwhelmed and confused by other legal content that doesn't apply to them could lead to a negative outcome.

We believed we could alleviate these difficulties by distilling information into an interactive wizard. Through interdepartmental workshop sessions with our partners and subject matter experts, the team designed the "Find out if you can enter Canada" wizard and a series of supporting pages. The wizard presents the user with questions about their situation to get a customized answer. Or, if the person prefers to look up details for a specific situation themselves, the supporting pages allow them to find answers within a consolidated group of webpages organized into topics, rather than multiple overlapping pages authored by multiple departments and agencies.

Reformulating existing information into a wizard takes some work. In this case, because multiple departments and agencies were involved, it meant working with partners to consolidate information, and creating a plan for keeping it up to date. It also meant introducing additional web content for users to choose from – although we believed a wizard would be more helpful for users than the status quo, we needed to be sure it was effective. The team needed to establish that presenting travel information via the wizard would improve the clarity of travel restrictions, and thus reduce calls and emailed questions.

Some departments didn't have prior experience with online wizards for entry requirements. They were deeply concerned that people could generate the wrong answer and incorrectly arrive at the border, only to be turned away. They looked to the usability test to ensure that the answers truly were correct for the given scenarios, and that the answers still made it clear that the final entry decision would be made at the border.

Research methodology

We needed to gauge the success rate of those using the wizard prototype compared to those using the existing information on the live website. Could research participants successfully complete task scenarios by finding the correct answer to commonly asked questions about entry requirements?

Task scenarios

The initial success rate for travel-related tasks on the existing live website was assessed via remote usability testing with 16 English- and French-speaking participants on June 23 and July 9, with participants using their mobile devices. During this round of testing participants were also asked to complete other non-travel related tasks that were not included in the travel success rate.

Later, a prototype of the travel wizard was tested via 60-minute usability testing sessions with 21 participants using their preferred desktop or mobile devices in September 2020.

We recruited 10 English-speakers and 11 French-speakers who were a mix of Canadian and non-Canadian participants. Half of the 6 non-Canadian participants were not native English or French speakers.

Participants were asked to answer questions about entry and travel requirements using either the live website or the prototype, within a time limit of 5 minutes. The specific questions were drawn from the working group input about the most common call drivers and border rejections.

The results: what we learned

We saw an improvement in task success: 74% of tasks were completed successfully using the travel wizard prototype, compared with a task completion rate of 31% for participants using the initial live site. Earlier testing on the initial live site included 4 task scenarios, while later prototype testing included 9 task scenarios.

The most notable improvement in task success was on the task relating to immediate family:

Farouk is an Indonesian who lives in Indonesia. His brother is getting married in British Columbia in August. He plans on being in Canada for 20 days. Will he be allowed into Canada?

17 of 20 (85%) of participants using the wizard prototype were able to correctly answer the question for this scenario. Whereas the live site required users to seek out and understand a definition of "immediate family", the wizard presents a definition and examples right at the moment when a user is asked about their relationship to the traveler seeking entry.

Previously, 0 of 10 participants were able to answer this question correctly using the live site. The wizard asked specific questions about the user's context that enabled people to find the answer for their situation.

These results reinforced the insights from early usability testing that led us to believe that content presented in a wizard format would be more effective; when trying to understand rules and requirements, users need answers, not information. People get confused by content that is irrelevant to their task, and wizards reduce cognitive load while helping people find and focus on customized answers.

Interactive wizards: a tool for communicating effectively in a crisis

In a crisis scenario requiring timely and clear communication of complex information, like the COVID-19 pandemic, we found the wizard model to be an effective design approach. Wizards can help government organizations:

Increase findability by connecting users to relevant answers

Wizards allow us to segment complex details that don't apply to everyone, so users only need to read as much as is absolutely necessary. By asking questions about the user's situation, we can match them to the specific details they need to understand, without forcing them to wade through information that's not relevant to them.

Present the right information at the right moment

Wizards help web and policy teams present the right information at the right moment. In a crisis, people are looking for answers and hurry through pages, skipping past blocks of legal content that don't answer their immediate question. Wizards enable presenting relevant legal information on the answer page, once the user is sure they have found what they're looking for, which is more effective than simply making legal information available earlier in their search.

Clarify ambiguous terms

It's important to clarify ambiguous policy terms that might lead users to a mistaken conclusion, while being careful not to overload them with too much information. For example, there were reports that many people intuitively included grandparents in their definition of "immediate family members" when interpreting requirements on the original site design, even though this was incorrect according to the policy. Using the wizard, once someone indicated that they were visiting an immediate family member, further questions were able to clarify the exact relationship requirements, providing a clear definition at the right moment. Communicating policy concisely and in plain language necessitates using mental models like "immediate family" which don't have a single universal meaning for everyone. When a policy conflicts with a common mental model, turning scenarios into questions helps people understand exactly what is meant.

Reduce the complexity of frequent updates

Implementing the wizard also made updating information less complex. During 2021, travel restrictions changed frequently in response to vaccine availability, testing options in different countries, quarantine requirements, and the Omicron variant outbreaks. Out-of-date content on the supporting pages became a challenge and caused confusion for visitors who arrived there without using the wizard. The team eventually shifted focus to keeping the wizard up to date, making changes as soon as rules were announced, and removed the duplicative supporting pages that had been created in 2020.

Impact

Once implemented, the Travel wizard was used by over 19 million people between July 2021 and July 2022. At a time when uncertainty was a daily reality, taking requirements information that was scattered across many pages and synthesizing it into clear, tailored answers was critical to empowering people in making already difficult decisions.

If you or your team are interested in learning more about wizards, read our blog post:

Blog post: The magic of using interactive questions