Improving Pull Request Process with Complexity Labels

Written by Sigute

Sticky notes and pens by Maria Tyutina

Does your team struggle with completing pull requests? Does there never seem to be enough time to review others’ code, or do you feel the need to continually ask your teammates for code reviews? Pull requests are difficult to manage, but labeling them according to complexity may present a solution to a broken process.

What is Complexity?

First, let’s define complexity. Consider the following: what makes a pull request easier or more challenging to review? Perhaps there is a type of pull request that merges right away, but another that sits there for a long time. What’s the difference between them?

In general, the smaller the pull request, the simpler and faster it should be to review. However, one line change is not always straightforward when it comes to impact. Fixing a typo is easy; updating an external dependency may only change a few characters, but it could break an entire app.

Considering only the number of lines of code changing, though, can be misleading if it is the sole way to define complexity because large pull requests are not necessarily complex. Renaming something might affect hundreds of files, but the risk is low (especially with modern tools that reduce the likelihood of forgetting files). Adding new translations could also generate a large changeset, but is unlikely to produce new bugs. We need a new way to accurately denote how complicated a pull request will be.

Complexity Matrix

Factoring in size and difficulty, you can use complexity as a way to measure pull requests.

Use this matrix as a way to imagine complexity and roughly estimate where a pull request might fall into:

Easy — Small Size and Low Difficulty

These pull requests are your small and simple varieties. This category is the quickest pull request type to review and the easiest to test. Changes are unlikely to cause further issues and tend to merge very quickly.

Medium-Low — Large Size and Low Difficulty

These are large pull requests, but the changes themselves are trivial. They can take longer to review because of their size, but are also less likely to introduce new bugs. When possible, it’s practical to divide these pull requests into multiple smaller ones.

Medium-High — Small Size and Hard Difficulty

These are pull requests that do not alter many lines of code, but could pose a significant impact. Examples include changing business logic, UI elements, updating dependencies, and more. These changes can potentially cause unforeseen effects, so they require careful reviewing and testing.

Strategies such as A/B testing, feature flags, and staged releases could be applicable to this category. Changes are easily revertable in case of any issues, whether with code or user impact (e.g., if the new UI does not perform as well as expected).

Large — Large Size and Hard Difficulty

Our final category contains large and complex pull requests. Such requests could be adding a new screen to the app, changing one dependency to another, refactoring, etc. The larger a pull request becomes, the harder it is to keep track of changes and conduct a thorough code review.

Be careful about initiating pull requests in this category. If the number of code changes increase, could they be divided across multiple code reviews? For example, database changes could be reviewed separately from the UI layer in a new app feature. Using feature flags to hide code that is unready for production helps with this process, so the feature branch does not become too far out of sync with the main branch.

Putting It Into Practice

You can label your pull requests as part of the title and use t-shirt sizes to represent their complexity:

S -> M -> L -> XL -> … -> XXXXXXL?

Some examples of pull request titles would look like this:

[S] Fixed accessibility bug in X screen

[M] Updated library Y dependency, please test

Alternative labels could include emojis:

🐇 -> 🦘 -> 🦙 -> 🦖

😀 -> 🙂 -> 😐 -> 😭

Remember that emojis appear differently between devices and platforms, so make sure that the ones you select look right on your teammates’ computers.

If you want to use animal emojis, you can use the following size categorizations:

Besides using the label in the title, you can implement labels in code review tools. GitHub, for instance, allows you to create custom labels (including emojis and colors) and use them to filter pull requests.

Impact of Changes

The Android Microsoft “To Do” app team uses this complexity-labeling technique for pull requests. The benefits of this approach include:

• Determining how much time it might take to review changes at a glance. This process helps engineers anticipate which pull requests will require more time to review and which they can review with a few spare minutes.
• Keeping complexity in mind when creating pull requests and strategies to mitigate risk.
• The pull request list becomes more manageable.

Though issues were not significant, the Microsoft team discovered that because their release notes make use of pull request titles for draft versions, they had to add an additional script to strip the labels as part of their continuous integration process.

Pull requests can improve applications, but neglecting to give them due time and attention can cause further headaches. Labeling pull requests according to how complex they are helps engineers anticipate the amount of work they will require and plan their time properly.

_________________________________________________________________

This article was originally published by a WWCode member, Sigute, on their blog.