Similarity Check is an automatic cross-checking of your candidates' solutions against every other submission we've ever received - over 12 million assessments.
We also scrape the web for any leaked solutions, and if we find a solution that looks similar to other ones (either online or submitted by other candidates), we'll let you know. Our system can recognize similarities even if solutions have been modified (e.g. identifiers' change, code reformatting, small changes in the code structure).
IMPORTANT: Similarity Check is applicable to all tasks with the exception of project tasks (multi-file tasks), technology-agnostic exclusive tasks, technology-specific exclusive tasks, multiple-choice questions (MCQ), free-form (essay), upload and SQL tasks.
In order to promote efficiency of your hiring process and help you focus on the best candidates, we complete Similarity Checks for all Screen candidates who reached your Passing Score. If there is no Passing Score defined in your test, we verify submissions of all candidates who scored at least 40% on the task. We don't complete Similarity Checks for submissions that scored below your Passing Score or below 40% if no Passing Score is set.
Check out the results of the Similarity Check in Suspicious activity detection in the upper part of the test report, and then click Verify to view more details:
To see a detailed report, click Resolve within the relevant task:
If the other solution belongs to one of your candidates, their session will be linked. If it's similar to any solution in our database that doesn't belong to your account, it will be shown simply as "another candidate".
Please note that solutions for some tasks are too simple to discern whether candidates cheated or that the similar submissions are merely coincidental. Also, in some rare cases, a single candidate might submit a similar code if they're taking Codility coding tests for more than one company. We detect such cases whenever we can, but since test sessions may be anonymous, this isn't always possible.
What is being checked in AI misuse?
The same mechanism used in defining AI Resistance of our tasks is applied in Similarity check. In short, the submitted solutions are cross-referenced against AI generated solutions. If they meet a certain set of parameters and indicators, they get flagged for review as each AI model tends to converge on a similar answer for the same question. Similarity check operates with dozens of such solutions for each task and technology to account for possible variations.
Does Codility use AI to make decisions on which solutions to flag?
We do not use AI to make decisions on whether a solution should be flagged as potentially plagiarized.
What should I do if I receive a similarity warning?
The warning on its own doesn't necessarily suggest cheating. If you have any suspicions, our recommendation is to connect with the candidate to ask for clarification. You can also conduct a Interview interview with the candidate in report mode and see if they can explain how they came up with this solution, their train of thought etc. If you need more details from us before approaching the candidate, please don't hesitate to contact us at support@codility.com.
Similarity statuses and what they mean
- Not started: Your candidate hasn't started solving their test yet, so similarity check hasn't been initiated.
- Not found: No similar solutions have been found.
- In progress: shown after the automatic check started on candidate's submission. This status remains until either no similarity is found or until manual review is completed. If a similar solution is found, our tech team reviews the Similarity Check report (shouldn't last more than 3 business days) and presents the findings to you.
- Found: Resolve: A similarity has been detected and is awaiting your review.
- Acknowledged: You've reviewed the similarity report and believe that the similarity in solutions suggests improper conduct.
- Dismissed: You've reviewed the similarity report and do not believe that the similarity in solution suggests improper conduct.
- Skipped: Solutions for tasks that fall under the test Passing Score (40% default if no passing score set up on a test) are skipped for similarity review.
If you have any additional questions about Similarity Check or would like to leave your feedback, please reach out to your Customer Success Manager or contact us at support@codility.com.