We are excited to announce the launch of our new Proctoring feature, designed to enhance assessment integrity and provide valuable insights into candidate behavior during Screen tests. With the introduction of Behavioral Events Detection, hiring teams can now have an additional layer of monitoring to help make informed decisions based on candidate actions. This article will guide you through the key features and functionalities of Proctoring and provide an overview of the behavioral events tracked.
Enabling Proctoring on your Screen tests:
Proctoring is configured on a per-test basis, allowing recruiters to tailor the monitoring settings to suit their specific requirements. Additionally, you can also utilize global Screen test settings to enable the enforcement of Proctoring features at the account level. If teams are enabled on your account, the settings will be enforced for the specific team(s) you enable this on.
Proctoring is disabled by default, giving recruiters the flexibility to choose when to enable it on new tests. Recruiters can enable the entire set of behavioral signals or selectively choose the signals they wish to monitor for a given test. The number of behavioral signals displayed in the candidate report depends on the signals selected at the time the invitation was sent to the candidate.
Important Note: Proctoring can only be enabled for tests that have not had candidates invited yet. Existing tests with candidates will need to be duplicated and have the proctoring settings enabled before inviting your first candidate.
Assessment and Hiring Policies Section:
To effectively utilize the Proctoring feature, hiring teams should utilize the Assessment and Hiring Policies section in the Screen test settings. This field allows teams to clearly define their expectations for candidates in conjunction with the enabled Proctoring features. These will be added to the Before You Begin section of the intro page for candidates to review.
Behavioral Events Detection Signals:
Behavioral Events Categories: Proctoring tracks five main categories of behavioral events, providing valuable insights into candidate behavior:
a. Copy-Paste Tracking: This feature monitors any attempts made by candidates to paste code into the solution area of the IDE. Events related to copy-pasting will also be listed in the Timeline tab of the candidate report, allowing recruiters to inspect the pasted code.
b. Tab Switching: Proctoring monitors whether candidates navigate away from the Codility browser tab to engage in other activities.
c. Time Spent on a Task: Abnormally short durations of time spent on a task, compared to the expected timeframe, can be an indication of suspicious behavior. Proctoring highlights instances where candidates complete tasks significantly faster than average, allowing your teams to review these cases more closely.
d. Copying a Task Description: Candidates attempting to copy the task description, despite the inability to copy it within the IDE, are detected by this feature. Such behavior could indicate a candidate's attempt to look up the task or employ AI platforms like ChatGPT.
e. Snapshot capturing: Takes webcam snapshots of the candidate at regular intervals and when proctoring events occur. Screenshots are kept for 30 days and can be accessed in the candidate report.
Recorded Proctoring
Video and sound can be recorded during the candidate's assessment, which can be played back when viewing the Candidate report.
To learn more, this article will provide more insight.
Screen Proctoring
When Screen Proctoring is enabled, it will record the candidate's screen. The candidate should only have one monitor open if they use multiple screens, as well as the most updated versions of their browsers.
Proctoring and the Candidate Report:
Under the Suspicious Activity checks, a new field called Behavioral Events Detection has been added to the candidate report. This field will only be visible in the candidate reports of tests that have proctoring enabled. The number of behavioral signals displayed in the candidate report depends on the signals selected at the time the invitation was sent to the candidate.
The timeline tab of the candidate report will also provide a detailed breakdown of the events detected as shown below:
You can expand captured screen images for review:
Additionally, your teams can easily view the Behavioral Signals column on the Test Mission Control page, providing a clear overview of candidate behavior during the assessment.
It's important to remember that not all behavioral signals, on their own, are indicative of cheating. The responsibility lies with the recruitment teams to review the reports and make informed determinations about candidate behavior. If uncertainty arises, we recommend following up the assessment with a CodeLive interview to further discuss any concerns with the candidate and review their solutions.
Candidate Disclosure on Intro Page:
Candidates participating in Screen tests will see a disclosure on the session's intro page, indicating that the hiring team has enabled event tracking and informing them of the specific behaviors that will be visible to recruiters. We highly encourage you to utilize the Assessment and Hiring Policies section of the test settings in conjunction with the Proctoring Settings to ensure transparency and set clear expectations for candidates.
Candidate Experience:
When a candidate begins their test, depending on which proctoring features have been enabled, they will be notified before progressing.
- Screen Recording: It will allow for a continuous recording of the candidate's screen; they can choose between sharing a tab, a window, or entire screen.
- Video Recording: It will allow for the candidate's camera and audio to be recorded; their browser settings must have proper access permissions enabled.
Please note that candidates must select "Allow" in order to proceed with the assessment if these features are enabled.
Please reach out to your Customer Success Manager or support@codility.com if you have any additional questions.