Our development environment incorporates a chat-based AI coding assistant known as Cody, powered by popular state-of-the-art LLM models.
Cody serves two primary purposes:
- Facilitating AI assistance within our IDE, simulating access to a modern development tool while upholding anti-plagiarism and cheating standards.
- Providing the opportunity to assess prompt-writing, output validation, and other AI collaboration skills.
Cody plays a crucial role in helping candidates understand the task at hand. For instance, candidates can seek clarification on task requirements, inquire about specific details related to elements of the task, or request examples illustrating task requirements.
Candidates also have the option to ask Cody for publicly available information, such as references from documentation (e.g., "Demonstrate how a Lambda function works in Python").
Clients have the ability to review the complete transcript of the candidate's interaction with Cody, enabling hiring teams to evaluate the candidate's prompt-writing skills and proficiency in utilizing AI for development. The assessment of prompt-writing skills holds significant importance for chat-based generative AI tools.
We believe in an AI-assisted future, where engineers do their job in collaboration with AI assistants. Today’s examples are CoPilot and ChatGPT.
More realistic assessment - Cody within the candidate assessment makes the evaluation more realistic. In most real-life cases, engineers would do their job with AI assistant(s) available.
Insights on Engineer-AI interaction - The transcript of a candidate and Cody’s conversation allows the hiring manager to gain more insights on whether the candidate asks reasonable questions (i.e. knows the field) or attempts to get a complete solution from the AI.
Cody can be used within both Screen and Interview:
1. Screen Test - Create a new test and under Settings step, toggle AI Assistant switch. This can also be done in Settings after the test is created prior to sending invites to candidates.
2. Interview - Create an interview. An interviewer will need to toggle on AI Assistant in the interview settings:
During Interview sessions, tasks will feature the AI Assistant button. Once the interview begins, candidates can utilize the assistant, while the interviewer has access to a read-only version of the conversation.
If you have any additional questions about Cody or would like to leave your feedback, please reach out to your Customer Success Manager or contact us at support@codility.com.
Important Note: Our AI Assistant Cody is currently not available for custom-created content.