Find a job: Research summary

May 23, 2019

The Find a job optimization project was completed over the summer of 2018. The goal was to make it easier for Canadians to find government jobs on Canada.ca. The project team included the TBS Digital Transformation Office (DTO), Employment and Social Development Canada (ESDC) and the Public Service Commission of Canada (PSC).

What made this project stand out

In contrast to Google and Indeed’s single platform model, the Government of Canada has several online job posting platforms. While there is occasional overlap, not one of the platforms includes all available jobs in the Government of Canada. The project streamlined and improved the paths to these platforms. We also experimented with moving towards a single platform approach.

The 3 largest job posting platforms linked from Canada.ca’s Find a job page are:

• GC Jobs - posts most but not all Government of Canada openings, including recruiting programs and inventories
• Canadian Armed Forces - current military openings
• Job Bank - private and some public positions, including provincial and municipal

Job Bank has thousands of searchable jobs posted by private and public employers across Canada. Canadians applying for Employment Insurance are auto-enrolled into Job Bank’s job search service. However, GC Jobs feeds in only a subset of the jobs posted there. As well, many of the jobs posted by the Government of Canada in GC Jobs do not use job titles from the National Occupation Classification. Because of this, when users start by searching, the GC jobs don’t show up in the NOC-based results.

Establishing a baseline

We started the project trying to understand which jobs were posted on each platform. Once we understood the web traffic patterns and needs of Canadian job seekers, we chose specific postings as targets for baseline tasks. These targets could only be found by using the separate platforms. Some example targets included:

• an infantry soldier on the Canadians Forces platform
• a Border Services Officer on CBSA’s site or GC Jobs
• a student Federal Work Experience summer job on Canada.ca or GC Jobs
• legal assistant positions in Vancouver - people had to use both Job Bank and GC Jobs to see that the Department of Justice Canada had a position available

The target job posts were crafted into 7 task scenarios. Participants used these scenarios to find the specific Government of Canada job targets to which they could apply. At the end of each test session, we had the participants repeat the Legal assistant in Vancouver task on Google or Indeed (whichever they used more). This let us check our hypothesis that people don’t expect to have to check multiple platforms to find all the available jobs.

In the Baseline study, 16 Canadian student participants performed 111 trials on the live Canada.ca pages. They did an extra 15 trials on Google or Indeed. Two of the sessions were conducted in French. The Canada.ca overall success rate was 59%. This was primarily because many students seemed to believe that Job Bank was a single platform for all jobs, including public federal government jobs.

Participant 3: “I would assume these are Government of Canada jobs because it is on their website.”

Some student participants eventually found the GC Jobs platform or the Forces site. Others never did. Not one student was able to successfully see both the Justice Canada legal assistant position and the other private positions for that job via Canada.ca. However, all succeeded when they used Google or Indeed.

Participant 1: “It's easier on Indeed - it shows all the jobs”

Designing for user success

To address the issues we identified, we held workshops to analyze video evidence of the baseline participants performing the tasks. This let us see what paths they were taking and what might be leading them astray. For example, the student summer job task showed us how difficult it was to navigate through Canada.ca to the Federal Student Work Experience (FSWEP) page. We started building prototypes of alternatives designs and paths.

We also experimented with ideas on quick-fixes towards the single platform idea. Three different teams investigated different avenues to add NOC job titles to the postings sent to Job Bank by the GC Jobs feed. The Public Service team tried having a person assign NOCs. The Digital Transformation Office tried adding NOCs to the feed. The Job Bank team applied their autocoder software to the feed. While none of these experiments assigned NOCs to 100% of the postings, the autocoder came the closest.

In the end, all the experiments reinforced the idea that the hiring managers need to be the ones to apply NOC titles, rather than imposing them later via technology. After the project ended, we discovered that all GC positions do have NOC codes, but they aren’t passed on or used within the hiring and job posting system.

Simplify and reroute

The team designed new streamlined navigation pages. These reduced the number of steps to get to the job platforms and recruiting program pages. For example, the Find a job page had 14 options on the Canada.ca live site. The streamlined prototype had only 6.

We also had extensive discussions about language. For example, students hadn’t understood the phrase ‘graduate recruiting programs’. “Recruiting” is the Government’s task. Graduates are just looking for a job. We also reworked link and doormat text to make it clearer, and balance the weighting of competing options.

For top task content, we frontloaded topic links and doormats with verbs so users could see what tasks they could do on each page. We tuned headings to job seekers, not to the government. For example, we changed “Federal Student Work Experience Program” to “Work for the government while you study.” Making the doormats more descriptive and action-based eliminated clutter and led users to their task faster.

Measuring success rates on the redesigned prototype

The team tested the redesigned prototype with a new set of 18 Canadian student participants. They performed 123 trials of the same task scenarios. Our target was either 80% success or an improvement of at least 20 points over the baseline score. The revised content and design improved findability rates from 64% to 90%, and overall task completion success rose from 59% to 85%.

This chart shows the task completion success rates across the baseline and redesigned validation test on the prototype for all 34 participants:

Overall

• adding NOC titles to the three job targets in Job Bank improved task success by 19.5% percentage points
• adding the NOC for the Legal Assistant position at Justice Canada to Job Bank meant that participants could see all of the positions available on a single platform, driving success for that single task up from 0% to 94%
• using verbs and a primary action button to direct the user to the application page, produced a 30% percentage point increase in findability for the FSWEP task

The future is searchable

For the validation study, the Job Bank manually added NOCs to 3 of the target job postings on the live site to make them searchable. This ‘single platform’ experiment contributed 19 points to the increase in overall task success from 59% to 85%.

Finally, we found that concise, readable job titles are critical. People can’t search for a job if they don’t know what it’s called. People failed the “government border officer” task on GC Jobs because the non-standard job title didn’t use the word “border.” The GC Jobs posting title was “CBSA Officer Trainee - Developmental Program”. The NOC title in Job Bank was “Border services officer”. Like Job Bank, Google recommends and supports a structured data approach based on the U.S. Standard Occupational Classification for job titles. The results of this study suggest hiring managers should title jobs with NOCs when possible to maximize findability online.

Key drivers of success

The team identified 4 factors that had the greatest impact on success rates:

• eliminate redundant pages and orient content for users and their top tasks
• design job posting pages for answers, not information, with prominent Apply buttons
• use searchable NOC-coded job titles for findability in all job search platforms
• meet user expectations for a single job search platform by adding coded GoC job postings to Job Bank

Request the research

Email us at dto.btn@tbs-sct.gc.ca if you have questions, or would like to know more about this research.

What do you think?

Let us know what you think about this project. Tweet using the hashtag #Canadadotca.

Explore further

• Finding a government job…it should be so easy… , a blog post by our optimization partner PSC
• Read overviews of other projects with our partners