Gayane Harutyunyan on Measuring Programmers Labor Efficiency

Gayane Harutyunyan, Head of Information Technology at OPEN Group, wrote an article for the “Modern HR Technologies” journal sharing her knowledge on how to measure the programmer’s work efficiency.

Any organisation with one or more software developers on its staff sooner or later faces the question of how to evaluate their effectiveness, which KPIs can be applied and who should evaluate them.

One of the most obvious criteria for the employee’s work efficiency is the number of tasks he solved per unit of time (day, week, month). However, each task has different complexity: while one can take an hour to solve, the other takes a week to localise the error.

Moreover, what if one employee solves a big challenging task while another deals with many small tasks? On practice, one big task is divided into several smaller ones. It might turn out that the work efficiency of an employee who solved multiple sub-tasks is higher than of the one who addressed small tasks individually. Obviously, the assessment of labour by the number of tasks solved will not accurately describe the current situation.

It is essential to address the question which units of task assessment could be applied. There are two approaches: first, it can be asserted in person-hours which the “ideal developer” will spend on solving the task; secondly, the evaluation can be made in abstract units. In the literature on team development organisation, the standard units of the cost of the assignment are called “Story Points”.

Consider the advantages of evaluating tasks with Story Points compared to person-hours:

  • simple tasks require significant investments in time, while complex tasks require specialised knowledge and competencies of the employee. Such tasks might not have the same cost, but the “man-hours” measurements could match;
  • the system of assessing tasks in standard units allows to get away from personal estimates: the assessment of the problem in person-hours and the actual time spent on the solution can vary greatly. The difference might turn out to be in favour of the executor, or vice versa, causing executor’s dissatisfaction or risks to become an object for client’s manipulations.


Our development team uses task assessments with Story Points. As elements of the evaluation, we take the powers of 2 (1/2, 1, 2, 4, 8). The simple tasks have an agreed rating of 1/2 or 1; more difficult assignments - 2 or 4. The most complex and extensive tasks are rated with 8 points. If the problem cannot be calculated with this scale, we break it into smaller tasks or to move it in the project development category.

There is another nuance to consider when evaluating tasks with Story Points. If the tasks require new competencies that no one or a few members of the team have then higher rating will become an additional incentive for team members to pay attention to their training and professional development. As a result, the team members obtain various knowledge and the capability of jointly solving a wide range of tasks, and even can become the basis for a team of cross-functional specialists.

Who Should Evaluate the Tasks?

Obviously, the complexity of specific tasks should be assessed by a qualified person, as the executors can’t always present their work correctly. In most cases, the client does not have the necessary knowledge to assess the complexity of the task accurately.

The client can see the quality of work - the functionality is not interrupted with errors, it facilitates client’s needs and speeds up their tasks; deadlines – the job is completed within the agreed time, and it is valuable to the customer at the time of delivery. But the complexity of each task can only be determined by a qualified specialist.

The following are the most common options of task assessment: the task is evaluated by the responsible person (team leader, team leader, leading specialist); or the entire development team evaluates the task. Each method has its pros and cons. If the responsible person assesses the complexity of the task, the final assessment will always be subjective and based on the experience of a particular specialist. However, during the team assessment, one or more team members may know a faster way to solve a problem than the one responsible for the task. And it can be vice versa: when addressing a similar problem, one of the team members might have encountered pitfalls others are unaware of. However, the assessment of tasks by the team has one significant disadvantage: the time needed for the whole team to understand the essence of the problem and discuss its solution. Is each task worth the cost?


Evaluation of Performance Results by Period

Based on the results of the period we are interested in, we analyse our specialists’ work and summarise the ratings of all completed tasks.

Using this approach, we determined the effectiveness of each member of the team, as well as such a quantity as the power (productivity) of the group- the number of Story Points that the team can make per unit of time. After that, the positive results can be encouraged with bonuses in monetary value or intangible items for employees.

News on this topic