Codility task library includes tasks of various difficulty levels ranging from very simple, to very hard. Before any task gets published and becomes available for you, we need to decide which difficulty level is the most appropriate for it.
IN A NUTSHELL:
In the first release, tasks are introduced to our library with a difficulty level based on our internal standards. This involves team members reviewing and beta-testing the task, and this is essential to our task release process. Depending on the internal results that we get, the difficulty level of the task is adjusted accordingly.
OUR INTERNAL THRESHOLDS:
A task's difficulty level is set based on a 90% passing rate. Typically, we assign them as seen below:
- Elementary - if 50% or more of candidates score 90%.
- Easy - if 25% - 50% of candidates score 90%.
- Medium - if 12.5% - 25% of candidates score 90%.
- Hard - if 2.5% - 12.5% of candidates score 90%.
- Very Hard - if 2.5% or less of candidates score 90% - this is currently shown as a Hard task with a "very-hard" tag assigned.
HOW WE UPDATE THIS:
We understand that some of our easy tasks may seem hard for some candidates, and, vice versa, some medium tasks may turn out to be easier than we predicted. That’s why we review the pass rates of our tasks regularly and adjust their difficulty level if we deem it necessary. In this case, we inform you about the changes, and, if you’re actively using the task, let you know so you can adjust your existing tests appropriately.
If you have any additional questions about task difficulty level or would like to leave your feedback, please reach out to your Customer Success Manager or contact us at support@codility.com.