Introduction
A job interview is a crucial step in the hiring process, shaping the foundation of the relationship between employers and employees. While it’s an undoubtedly stressful experience for candidates, it also holds significant value for companies. Choosing the right personnel is essential, as a poor hiring decision can lead to financial losses due to onboarding time, the probation period, and the search for a replacement. Furthermore, a mismatched hire can disrupt team dynamics and affect project outcomes. In this article, we will take a closer look at the job interview, specifically from the perspective of hiring a DevOps Engineer with a focus on the intricacies of the technical interview. We will explore the process of technical interviewing, the issues and challenges that arise for the interviewer and an organization.
How Do We Hire?
The complete Klika Tech interview process is detailed in the How We Hire article. As outlined in the article, the interview flow is fairly standard and consists of three main stages:
- Interview with a recruiter
- Technical interview
- Project interview
Having served as a technical interviewer for the past 1,5 years, I will delve into the Technical Interview process to shed light on elements the technical interview should consist of from the interviewer’s perspective.
Technical Interview
So what is the technical interview?
If you were to ask yourself or your colleagues for a definition of a technical interview, you would likely hear something like: “A technical interview is a structured process designed to assess a candidate’s practical skills, problem-solving abilities and overall suitability for the technical demands of the role. During the technical interview candidates have the opportunity to demonstrate their actual knowledge and experience”.
While this definition sounds convincing and comprehensive, it’s somewhat vague, isn’t it?
From this description, we don’t get a clear picture of what the “structured process” actually entails. How is it structured and what components does it consist of? When we say “designed to assess skills,” how exactly do we measure these skills? What principles guide us in the assessment? And as for “overall suitability for the technical demands” – what exactly do we mean by “suitability,” and what are these technical demands?
Without answering these questions and, consequently, having a clear and precise understanding of what a technical interview is, why it’s necessary, how it should be conducted and what outcomes we expect, the technical interview risks becoming ineffective and time-wasting for both the interviewers and the candidates.
So, let’s dive deeper and clarify the technical interview process to eliminate vague definitions, uncertainty and confusion.
Why?
Let’s start with the general purpose of an interview – specifically, why it is needed. In simple terms, when a company opens a position, it aims to hire someone who will perform the job effectively and thereby bring value to the organization. Consequently, a technical interview, among other interviews, is a process designed to assess a candidate’s ability to meet the criteria for a job well done.
This leads us to the first question – what are the criteria that define a job as successfully done and how can we measure a candidate’s ability to meet those standards? The rule of thumb here is that, in most cases, the criteria for a job well done correlate with a candidate’s experience, knowledge and skill set. In other words, if a candidates possess the necessary knowledge and skills, there’s a high likelihood that they will perform the job successfully. Thus, the required knowledge, skills and competencies for a particular job serve as a baseline for evaluating a candidates.
So, here is the second question – how can a candidate’s skills and competencies be effectively managed and assessed?
Competency Matrix
One effective tool for managing them is the Competency Matrix.
A competency matrix is a structured tool used by organizations to assess, track and manage the skills, knowledge and competencies of their employees and candidates. It might be represented in a grid or table format, where various skills, competencies or knowledge areas are listed along with corresponding proficiency levels. Each proficiency level is defined with specific indicators, outlining the knowledge and expertise required to attain a particular level of seniority.
For example, take a look at the limited snippet of the “Competency Matrix” and “Proficiency Level Indicators” tables for the DevOps engineers.
In the “Competency Matrix” table you could find two types of competencies (Linux and Infrastructure as Code) along with the seniority levels (DevOps Trainee, DevOps Engineer, Senior DevOps Engineer). The “ Proficiency Level Indicators“ table highlights the essential skills required for the each level of seniority.
Competency Matrix
| Competency | DevOps Trainee | DevOps Engineer | Senior DevOps Engineer |
|---|---|---|---|
| … | … | … | … |
| Linux | Level 1 | Level 1 | Level 2 |
| Infrastructure as Code | Level 1 | Level 2 | Level 3 |
| … | … | … | … |
Proficiency level indicators
| Proficiency level indicators | Comments | Level 1 | Level 2 | Level 3 |
|---|---|---|---|---|
| Linux | ||||
| Infrastructure as Code | ||||
| etc |
Now that we have the Competency Matrix, how can it be used during the technical interview? The answer lies in its purpose – to compare specific skills and competencies of candidates.
With this in mind, let’s refine the definition of a technical interview:
A technical interview is a process of gathering information about candidates’ competencies for its effective comparison afterwards.
Now it becomes a bit clearer what interviewers should focus on during the interview – we need to prepare a list of specific competencies, along with proficiency level indicators, beforehand and gather information about those competencies from the candidate during the interview.
Narrow down competencies
However, merely having a Competency Matrix is not sufficient. One of the challenges with a Competency Matrix arises from the rapidly evolving IT landscape, which ensures a constant influx of new tools, techniques, technologies and approaches to the market. As a result, the list of competencies can become overwhelmingly extensive, making it nearly impossible for a single candidate to cover all areas.
Let me elaborate on this. Currently, the DevOps landscape encompasses a wide range of areas, including Linux, shell scripting, Infrastructure as Code, software development methodologies, cloud development principles, software programming, architectural patterns, versioning and branching strategies, networking, web servers and load balancing, containers, Kubernetes, serverless architecture, databases and storage, IoT, high availability and disaster recovery, authentication and authorization, infrastructure security, cloud application security, continuous integration, continuous delivery, logging and monitoring, and others. Although the list is quite extensive, if you’re a consulting company with a mission to help clients worldwide uncover the connection between technology and business goals, you’ll likely want to include all these competencies in your Competency Matrix. However, even though I’ve had the opportunity to work with brilliant and highly talented engineers, I tend to believe that most engineers have their own specializations, allowing them to excel in certain areas and possess foundational knowledge in related fields. This makes it unfeasible to blindly follow the Competency Matrix during a technical interview. Additionally, attempting to cover the entire list of competencies would likely require far more than a single technical interview.
So, how do we narrow down the scope of competencies during the technical interview? The answer lies in aligning our focus with the specific technologies and skills that are either explicitly required or anticipated to be heavily utilized in the particular project or position. So, rather than attempting to cover all DevOps areas from the Competency Matrix in a single technical interview, it’s better streamline the interview by focusing on the core technologies that the candidates will actually work with daily in the specific role they are applying for.
For instance, if a project’s primary technology stack involves AWS, Kubernetes, Terraform, OpenSearch/Kibana, Prometheus/Grafana, Jenkins and ArgoCD, these should naturally form the core competencies to explore during the interview. Questions and discussions should center around the candidate’s experience and proficiency with these tools, as they are directly relevant to the day-to-day work they will be performing. Diving deeply into these areas will give a clearer picture of the candidate’s readiness to contribute effectively to the project from day one.
On the other hand, if the position does not require in-depth knowledge of, say, Serverless architecture, Linux internals, Shell scripting, or advanced Software Programming, then it makes little sense to treat gaps in these areas as deal-blockers. A candidate might not be particularly skilled in these topics, but this does not diminish his potential to excel in a role that doesn’t require them. By narrowing our focus in this way, we avoid penalizing candidates for lacking skills that are irrelevant to the position at hand. This approach allows interviewers to make informed decisions based on the competencies that truly matter, without covering all the competencies from the Competency Matrix. The matrix itself remains a valuable guide, providing a comprehensive view of desirable skills, but we adapt it to serve the needs of each unique position.
Moreover, when hiring for different positions, the core competencies will shift in line with the role’s requirements. A DevOps role focused on data pipelines and storage might prioritize knowledge in databases, data streaming and infrastructure resilience, while one centered on microservices might emphasize containerization, service mesh and observability. This dynamic approach ensures that each technical interview is tailored and relevant, yet still remains within the larger framework of the Competency Matrix.
Personal experience
So far, we have defined what a technical interview is and established why it is an essential part of the hiring process. Now, let’s shift our focus to the practical aspects of conducting a technical interview, aiming to make it both efficient and productive.
Below, I’ve outlined the core principles that I believe interviewers should adhere to in order to conduct an effective and insightful technical interview.
Consistency in questions
To conduct a fair and objective comparison between candidates, it is essential to maintain consistency in the questions asked during technical interviews. By asking the same set of core questions to each candidate, interviewers establish a reliable basis for evaluation. This approach ensures that all candidates are assessed on the same competencies and criteria, making it easier to draw meaningful comparisons.
Conversely, if each candidate is asked different questions, there would be no common ground for comparison. The lack of a standardized question set would create ambiguity in the evaluation process, making it difficult to measure one candidate’s response against another’s.
Scenario-based questions
In addition to asking general questions about technology (such as “What is a load balancer?” or “What is Terraform?”), it is highly beneficial to incorporate scenario-based questions into the technical interview. These types of questions go beyond mere definitions and instead focus on real-world problem-solving skills and practical application. For example, asking, “How would you distribute requests across multiple backends?” or “How would you provision 1,000 EC2 instances in AWS?” requires candidates to think through specific situations they might encounter on the job.
The objective here is not to conduct a theoretical exam where there is one “right” answer. Instead, scenario-based questions give insight into the candidate’s hands-on experience and approach to problem-solving. By presenting scenarios, interviewers can better understand how candidates apply their knowledge in practical situations, make decisions under constraints, and approach complex problems. This type of questioning provides a more accurate view of how a candidate would perform in the role and whether they possess the experience and critical thinking needed to succeed.
Real-task questions
In addition to scenario-based questions, it’s beneficial to design questions that closely resemble the real tasks, architecture and infrastructure that the candidate would encounter in the actual role. These questions could even be inspired by current tasks in the Sprint Backlog, providing a more practical and relevant context for the interview.
This approach has a dual benefit. First, it allows the interviewer to gain a deeper understanding of how the candidate would handle real-world tasks and challenges, giving insight into their practical problem-solving abilities and familiarity with the required tools and environments. Second, it enables the candidate to gain a clearer picture of what the job will entail, making it easier for them to assess if the role aligns with their expectations and strengths.
Structure of the Interview matters
I believe that structuring the interview logically and coherently is crucial for an effective assessment. The interview should progress in a natural flow, starting with foundational topics and gradually moving towards more complex and specialized questions. This approach not only helps the interviewer assess the candidate’s depth of knowledge across various areas but also gives the candidate a clear direction, reducing the potential for confusion.
For example, the interview could begin with basic Linux questions to evaluate the candidate’s understanding of operating systems, then progressively dive into topics related to infrastructure, high availability, monitoring, and configuration management. Here’s an example of how this structured approach might look:
- Linux Basics
Begin with simple Linux questions to establish the candidate’s familiarity with the operating system, such as:- What are some common Linux commands you use for system monitoring and troubleshooting?
- Infrastructure as Code (IaC)
Introduce the concept of Infrastructure as Code, to understand the candidate’s knowledge of automated provisioning:- What IaC tools can be used to provision machines in, for instance, AWS, and why would you choose one over another?
- High Availability (HA) and Fault Tolerance (FT)
Now move to more advanced infrastructure topics, focusing on resilience and reliability:- Once the machines are provisioned in AWS, how can we ensure High Availability (HA) and/or Fault Tolerance (FT) for those machines?
- Configuration Management
Cover configuration management, exploring methods to install and manage applications on the provisioned instances:- How can we install applications on those machines? What configuration management tools like Ansible, Chef, AWS SSM would you consider, or would you prefer using AWS AMI or even containerization?
- Monitoring
Delve into monitoring concepts, exploring how the candidate ensures system health and performance:- How would you set up monitoring for the provisioned machines in AWS? What tools would you use?
- Logging
Discuss logging strategies to see how the candidate approaches observability and troubleshooting:- How would you collect and manage logs from the applications running on those machines? What logging tools or services would you utilize?
By following this structured approach, interviewers can effectively cover a wide range of topics without overwhelming the candidate. This method not only assesses the candidate’s technical knowledge but also evaluates their ability to approach complex systems in a structured and logical manner.
Independent interviewers
In some cases, a single technical interview might not be sufficient to assess a candidate’s capabilities thoroughly. According to the How We Hire article, there might be an additional technical component in the project interview. To ensure a well-rounded evaluation, it’s beneficial to have a second, unbiased technical interviewer conduct this part as well. This approach allows the company to gather feedback from at least two independent interviewers, providing a more comprehensive and balanced assessment of the candidate’s skills and suitability for the role.
Conclusion
In this article, we explored the concept of a technical interview, its importance and the core principles that contribute to making it an efficient and productive process. We also delved into the Competency Matrix, discussing its role and limitations as a tool for assessing candidates.
To sum up, here’s an outline of how an technical interview should look:
- Competency Matrix Development – start by creating a broad Competency Matrix, listing the essential skills and competencies required for engineers in a specific IT field.
- Narrowing down – refine this list based on the unique requirements of the particular position.
- Interview Process – during the interview, gather information about the targeted competencies by:
- asking the same questions to all candidates to ensure consistency and comparability
- using scenario-based questions to understand how candidates approach practical challenges
- ensuring that scenario-based questions align with the current infrastructure or solutions in place
- Independent Assessments – in cases where multiple interview stages are necessary, conduct independent interviews with different interviewers to avoid bias.
- Comparison and Selection – finally, compare the gathered information across candidates, choosing the one who aligns most closely with the desired seniority level as defined in the Competency Matrix and Proficiency Level Indicators.
By following these steps, candidates can benefit from a structured, fair and transparent technical interview process that allows them to showcase their skills and suitability for the job effectively.
By Pavel Luksha, Senior DevOps Engineer, Klika Tech, Inc.