Improving Pull Request Process with Complexity Labels

Written by Sigute

Sticky notes and pens by Maria Tyutina

Does your team struggle with completing pull requests? Does there never seem to be enough time to review others’ code, or do you feel the need to continually ask your teammates for code reviews? Pull requests are difficult to manage, but labeling them according to complexity may present a solution to a broken process.

What is Complexity?

In general, the smaller the pull request, the simpler and faster it should be to review. However, one line change is not always straightforward when it comes to impact. Fixing a typo is easy; updating an external dependency may only change a few characters, but it could break an entire app.

Considering only the number of lines of code changing, though, can be misleading if it is the sole way to define complexity because large pull requests are not necessarily complex. Renaming something might affect hundreds of files, but the risk is low (especially with modern tools that reduce the likelihood of forgetting files). Adding new translations could also generate a large changeset, but is unlikely to produce new bugs. We need a new way to accurately denote how complicated a pull request will be.

Complexity Matrix

Use this matrix as a way to imagine complexity and roughly estimate where a pull request might fall into:

Easy — Small Size and Low Difficulty

Medium-Low — Large Size and Low Difficulty

Medium-High — Small Size and Hard Difficulty

Strategies such as A/B testing, feature flags, and staged releases could be applicable to this category. Changes are easily revertable in case of any issues, whether with code or user impact (e.g., if the new UI does not perform as well as expected).

Large — Large Size and Hard Difficulty

Be careful about initiating pull requests in this category. If the number of code changes increase, could they be divided across multiple code reviews? For example, database changes could be reviewed separately from the UI layer in a new app feature. Using feature flags to hide code that is unready for production helps with this process, so the feature branch does not become too far out of sync with the main branch.

Putting It Into Practice

S -> M -> L -> XL -> … -> XXXXXXL?

Some examples of pull request titles would look like this:

[S] Fixed accessibility bug in X screen

[M] Updated library Y dependency, please test

Alternative labels could include emojis:

🐇 -> 🦘 -> 🦙 -> 🦖

😀 -> 🙂 -> 😐 -> 😭

Remember that emojis appear differently between devices and platforms, so make sure that the ones you select look right on your teammates’ computers.

If you want to use animal emojis, you can use the following size categorizations:

Besides using the label in the title, you can implement labels in code review tools. GitHub, for instance, allows you to create custom labels (including emojis and colors) and use them to filter pull requests.

Impact of Changes

  • Determining how much time it might take to review changes at a glance. This process helps engineers anticipate which pull requests will require more time to review and which they can review with a few spare minutes.
  • Keeping complexity in mind when creating pull requests and strategies to mitigate risk.
  • The pull request list becomes more manageable.

Though issues were not significant, the Microsoft team discovered that because their release notes make use of pull request titles for draft versions, they had to add an additional script to strip the labels as part of their continuous integration process.

Pull requests can improve applications, but neglecting to give them due time and attention can cause further headaches. Labeling pull requests according to how complex they are helps engineers anticipate the amount of work they will require and plan their time properly.


This article was originally published by a WWCode member, Sigute, on their blog.

We are a 501(c)(3) non-profit organization dedicated to inspiring women to excel in technology careers.