A Tool for Assessment Planning
Empower Service Delivery
Data Stewardship
Developing assessment materials was routinely mentioned by managers as one of the most challenging steps in the staffing process during our user research. Many reported procrastinating on this step as they felt overwhelmed with scheduling challenges and unclear about how to develop materials. This significantly added to the overall time to staff.
Talent Cloud’s assessment planning tool is designed to facilitate discussion between managers and HR advisors early in the staffing process (before a job advertisement is finalized). It consists of two components:
Notably, neither component of the assessment planning tool is designed to promote a particular assessment method for a particular skill or provides tests associated with the assessment method. It’s purely for planning purposes. (The team looked at ways to connect the tool to academic references, making suggestions to managers on literature-supported assessment methods for each skill, combined with data on how previous GC managers had tested each skill on Talent Cloud, but we didn’t have the resources to complete the work.)
The assessment planning tool is currently in beta for managers and HR advisors to use.
The first component (basic plan) received positive feedback and was well-used, and is now ready to be upgraded with a planning calendar, email templates for key stages, and additional support for managers.
The second component (ratings guide builder) had mixed results in live testing. It surfaced several underlying challenges managers face when developing assessments, as well as the high degree of variance in advice provided by HR advisors in different departments. Some managers found the model very useful, others found it confusing. In many departments, this component of the tool was used almost exclusively by HR advisors (who had an easier job using it than managers). To address these variances, improve usability, and to connect the ratings guide methodology to the final assessment justifications required for each applicant (Record of Decision), our team is working on a new design for the ratings guide component. It has been through several rounds of workshop redesign, and is ready for a round of live testing using off-platform tools. (See also A Tool for Record of Decision in this section of the report.) While complicated to develop, we believe that creating a simple logic frame that all managers can use will dramatically help speed time to staff in the assessment phase of staffing and early planning stages (pre-job advertisement).
As an HR best practice and to avoid delays while candidates are waiting to be contacted, some departments require that all assessment materials be completed before the job advertisement can be published. This shortens how long applicants wait for testing and interview stages, but it requires more upfront work from managers. After observing manager and HR advisor interactions around the development of assessment materials, Talent Cloud has found that staffing processes run best if there is at least an assessment plan in place when the job advertisement is finalized, even if the assessment materials themselves (tests and ratings guides) are still in development at the moment a job advertisement goes live. (Although we agree with HR advisors on this one - it’s best if everything is set to go before a job advertisement is posted.)
The assessment planning tool was used by some managers and HR advisors to facilitate discussions while the job advertisement was still being finalized. As a result of these discussions, some managers added screening questions to the application process. In other cases, managers were prompted to have more in-depth discussions with their HR advisors on what skills were absolutely required and whether there were any overlaps. They then revised the skills on the job advertisement to ensure they could assess applicants in a manageable way.
As for the ratings guide builder, some used the tool to finalize their assessment materials while the job advertisement was open for application. In those cases, the managers were able to start assessing their applicants as soon as the applications became available.
Overall however, the ratings guide, in its initial implementation, has not been broadly adopted and most managers have instead opted to use the standard departmental tools for putting together ratings guides for their assessments.
On the whole, while this tool was one of our more heavily debated and revised products, we ultimately found that its inclusion was definitely helpful for managers in reducing time to staff, especially when it came to moving swiftly enough through assessment stages to keep top applicants interested and feeling engaged.
The assessment plan builder was useful as a tool to facilitate discussions between managers and HR advisors. It also helped managers to take a more holistic view of the staffing process while they were still finalizing the job advertisement. This improved quality and cohesion across staffing stages, as well as overall time to staff.
One of the challenges managers faced when building the ratings guide was to articulate the passing criteria for each skill and distinguish between various levels. While Talent Cloud doesn’t provide any advice on the assessment, our team is working on a logic framework that aims to help managers consider this in a consistent way. Using this framework, our team has developed generic behavioural indicators for some of the most commonly requested skills as proof of concept. With more time and capacity, we believe behavioural indicators can be developed for a wider variety of skills and be pre-populated in the ratings guide builder, which in turn can help accelerate the discussion between managers and HR advisors. These behavioural indicators should be developed with the support and validation of the communities of practice for the various classification groups.
During assessment planning discussions, managers routinely requested sample questions and existing assessment materials, to which HR advisors responded differently across various departments. Some managers also suggested building a GC-wide inventory of assessment materials developed by managers in the Government of Canada that teams from different departments can reference. Assembling a library of assessment materials gathered from GC partners was, unfortunately, beyond the capacity of Talent Cloud at its current resource levels. But the concept of an assessment learning library for managers represents an opportunity for a broader discussion across the public service.
Notably, both managers and HR advisors (who have claimed that specific job) can create content and edit all parts of the assessment plan from their respective portals. Activity is tracked, and managers and HR advisors can add questions and comments for each other.
