Research on code coverage
By Jaime Jorge, CEO and Co-founder of Codacy
January 10, 2017
Best practices dictate that you should track code coverage. Code coverage helps you assess the proportion of code that was tested and (more importantly) not tested.
Why is it important to track coverage? Our users report that coverage is important because it is the confidence level of code being correct and with less bugs. Weíve written previously about code coverage, this is a follow up based on industry data.
In this article we share some evidence of how different companies do code coverage, based on a survey of 680 companies about their code quality practices. Weíll answer three questions:
Without further adoÖ
Who tracks code coverage? What happens when you donít?
Letís first look at some of the numbers from our research.
Half of the companies track code coverage. Weíll see in a second how that impacts code quality and other parts of development.
This means that half of the companies cannot trust their code to be without bugs.
The bigger the company the more likely it is to track code coverage, as shown in the chart below.
You can see that the larger the company the more likely it is to be tracking code coverage.
Not surprisingly, the older the company the more likely it is to track code coverage too.
So the first major question we need to ask is: what is the impact of not tracking coverage?
Here is a quick summary of the impact:
subtle, there are benefits to tracking code coverage.
Code coverage thresholds?
Code coverage at any given time is really just a number. Itís the evolution of that number that matters. Code coverage that goes down is an indication that new code is not being tested therefore the confidence level is going down. If itís going up, normally itís a good sign.
Only 34% of companies require a minimum code coverage threshold to accept a pull request.
What is surprising is the difference between those 34% and 65%.
Here is the real difference between those two groups:
Another important question is: what is the ideal threshold?
Conclusion: having code coverage thresholds increased perceived code quality and decreased time spent on tackling technical debt. Companies that required this threshold set the bar, on average, at 80%.
Code coverage workflow rules?
consideration is about code coverage
First question is: how many companies have code coverage workflow rules?