Codility Screen's new Typing Pattern Detection is a passive behavioural proctoring signal designed to identify candidates who may be retyping solutions generated by external AI coding assistants like ChatGPT or Claude. This feature helps you maintain assessment integrity by flagging a growing and previously difficult-to-detect method of assessment evasion.
Typing Pattern Detection provides a reliable indicator to ensure the quality of candidates progressing to the interview stage.
- Detects Evasion: It addresses the increasingly common practice of using AI tools on secondary devices to bypass assessments.
- Reduces Hiring Risk: It lowers the risk of hiring an individual whose assessment score doesn't accurately reflect their true coding ability.
- Supports Human Review: The flag is a risk indicator to support your team’s decision-making process, ensuring you maintain control over how proctoring data is acted upon, rather than relying on automated disqualification.
- Zero Friction: The signal collection is passive, adding no interruption or distraction to the candidate's assessment experience.
How it Works and the Candidate Experience
Typing Pattern Detection is part of the Behavioral proctoring signals. It is on by default but can be turned off per assessment.
The system works by analyzing the candidate's typing behaviour after they submit their assessment. It tracks the sequence and structure of keystrokes to determine whether the solution was written organically or in an unnatural coding pattern consistent with copying from an external source. Authentic coding is typically iterative, involving edits and rewrites, which differs from the sequential input seen when retyping.
- Passive Collection: The analysis takes place post-submission. The candidate is entirely unaware that this signal is being collected or evaluated, ensuring the mechanism does not interfere with the performance of legitimate candidates.
- Risk Flagging: When an unusual pattern is detected, a risk flag appears in the candidate's Screen report alongside existing Level 1 behavioural proctoring signals.
Current Limitations
While broadly available, the signal has some limitations to be aware of:
-
Task Type Coverage: The signal is currently not available for task types where keystroke analysis is not applicable, including:
- Multiple Choice Questions (MCQs)
- Debugging tasks
- Essay tasks
- File upload tasks
- VS Code tasks
- Code Length: Solutions that require less than 10 lines of code are automatically skipped to reduce the potential for false positives.
- Interpretation: The flag suggests a high likelihood, not certainty, of retyping. Codility recommends that this signal always be reviewed in context alongside other proctoring data, solution quality, and human judgment; it should not be used alone as grounds for automatic candidate rejection.
If you have additional questions, feel free to reach out to support@codility.com or your Customer Success Manager.