Similarity Check automatically cross-checks your candidates' solutions against every other submission we've received—over 12 million assessments.
We also scrape the web for leaked solutions, and if we find one that looks similar to others (either online or submitted by other candidates), we'll let you know. Our system can recognize similarities even if solutions have been modified (e.g., changing identifiers, reformatting code, or making small changes in the code structure).
IMPORTANT: The Similarity Check applies to all tasks with the exception of project tasks (multi-file tasks), technology-agnostic exclusive tasks, technology-specific exclusive tasks, bugfixing tasks, multiple-choice questions (MCQ), free-form (essay), upload, and SQL tasks.
To promote the efficiency of your hiring process and help you focus on the best candidates, we complete Similarity Checks for all Screen candidates who reach your Passing Score. If no Passing Score is defined in your test, we verify submissions of all candidates who scored at least 40% on the task. We don't complete Similarity Checks for submissions that scored below your Passing Score or below 40% if no Passing Score is set.
Check out the results of the Similarity Check in Suspicious activity detection in the upper part of the test report, and then click Verify to view more details:
To see a detailed report, click Resolve within the relevant task:
If the other solution belongs to one of your candidates, their session will be linked. If it's similar to any solution in our database that doesn't belong to your account, it will be shown simply as "another candidate".
Please note that solutions for some tasks are too simple to discern whether candidates cheated or that the similar submissions are merely coincidental. Also, in rare cases, a single candidate might submit a similar code if they're taking Codility coding tests for multiple companies. We detect such cases whenever possible, but this isn't always possible since test sessions may be anonymous.
What is being checked in AI misuse?
In short, the submitted solutions are cross-referenced against AI-generated solutions. If they meet a certain set of parameters and indicators, they get flagged for review, as each AI model tends to converge on a similar answer for the same question. Similarity check operates with dozens of such solutions for each task and technology to account for possible variations.
Does Codility use AI to make decisions on which solutions to flag?
We do not use AI to decide whether a solution should be flagged as potentially plagiarized.
What should I do if I receive a similarity warning?
The warning doesn't necessarily suggest cheating. If you have any suspicions, we recommend contacting the candidate to ask for clarification. You can also conduct an interview with the candidate in report mode and see if they can explain how they came up with this solution, their train of thought, etc. If you need more details from us before approaching the candidate, please don't hesitate to contact us at support@codility.com.
Similarity statuses and what they mean
- Not started: Your candidate hasn't started solving their test yet, so a similarity check hasn't been initiated.
- Not found: No similar solutions have been found.
- In progress: shown after the automatic check started on the candidate's submission. This status remains until either no similarity is found or until manual review is completed. If a similar solution is found, our tech team reviews the Similarity Check report (shouldn't last more than 3 business days) and presents the findings to you.
- Found: Resolve: A similarity has been detected and is awaiting your review.
- Acknowledged: You've reviewed the similarity report and believe that the similarity in solutions suggests improper conduct.
- Dismissed: You've reviewed the similarity report and do not believe that the similarity in solution suggests improper conduct.
- Skipped—Did Not Meet Score Requirements: Solutions for tasks that fall under the test Passing Score (40% default if no passing score is set up on a test) are skipped for similarity review.
- Skipped—Setup: Skipped due to the absence of an eligible task for AI misuse and Plagiarism check. Examples include multiple-choice questions (MCQs), free-form essays, uploads, and SQL tasks, among others mentioned in this article.
If you have any additional questions about Similarity Check or would like to leave your feedback, please reach out to your Customer Success Manager or contact us at support@codility.com.