Once a candidate has completed your CodeCheck session, you'll use their Candidate Report to review their score. Reading these reports correctly is really important to ensure you're identifying which candidates you want to move forward with first.
You have several tabs you can switch between: Summary (overall summary of the session), Details (candidate and session details), Timeline (time used and user notes).
In the header you also have buttons that will allow you to request a style assessment, reopen a session in CodeLive, export the report in pdf format, as well as share, cancel, or reopen the test session.
OVERALL TEST SCORE:
The first thing you'll see is the candidate's overall test score, made up of the combined total points achieved on each task. It factors in both the code's correctness and performance, and also includes the similarity check indicator so you can easily tell if the candidate's work is their own. You can stack rank candidates by this overall score, and then dig deeper to see how they reached this total, task by task.
INDIVIDUAL TASK SCORE:
Every task is scored for correctness, and where possible, also performance.
"Performance" corresponds to how their solution performs with large data-sets in terms of running time complexity. To produce scalable code, a candidate requires good algorithmic knowledge and developed programming skills. In other words, for a candidate’s solution to be scalable, it must be suitably efficient and practical when applied to large data-sets.
Symbols of the form "O(…):", presented in the evaluation report under the label “Detected time complexity”, describe the scalability of the solution. This notation shows the precise relationship between the input data size and how quickly the solution is able to compute the answer. For example, if the data consists of N elements, then the expression "O(N2)" (read: "N squared") means that there is a quadratic relationship between the number of elements and the running time of the solution. In other words, increasing the input size by 3× will roughly increase the execution time by 9×, increasing the input size by 5× will roughly increase the execution time by 25×, and so on. O-notation is a popular way of describing the behavior of algorithms (see Wikipedia), and Codility employs it to describe the scalability of the solution program.
"Correctness" is the program’s ability to produce correct results for inputs of moderate sizes, including various corner cases. For a programmer to produce code that runs correctly requires care, attention to detail and a good understanding of both the problem and the programming language. Where performance cannot be assessed, correctness is the only contributor towards the total score. If a candidate passes all the test cases, he or she gets the maximum score. If the answer is wrong the details of what was expected and what they actual output was is provided.
- Pink Arrow: Text pasted into editor
- White Arrow: Verification attempt
- Blue Arrow: Final solution of your candidate
You can use the timeline to get a play-by-play of what the candidate was doing over the course of their given task. The arrows help highlight key points in their coding session so you can look at what they were attempting to verify and what they imported from an external source.
We like to emphasize that just because something was pasted into the editor, it doesn't mean that the candidate cheated. A lot of candidates like to use their own IDE for writing the bulk of the code, and then fine-tune it inside our editor as they run verifications.