Using (Bio)Metrics to Predict Code Quality Online — Sebastian C. Müller

How to write good code? We have code reviews, automatic quality approaches, but reviews take a lot of time and effort and automatic approaches can result in many false positives. What to do? What about measuring a developers cognitive load? From previous research, we know that cognitive load can be estimated using biometric sensors.

Sebastian did a study with 10 developers, for two weeks. The devs wore 1 or 2 biometric sensors (chest and wrist band). Furthermore they collected code difficulty ratings by sometimes interrupting them and asking them to rate their code, and they measured a variety of code metrics for all the code devs touched when working on something.

This lead to 41 million biometric data points, and perceived difficulty for 1480 classes and 1511 methods. Code reviews after the experiment resulted in 162 quality concerns in 1109 code elements.

A few cool findings

If you interact with code more, you perceive it as simpler, even though the metrics do not change:

Capture

The biometric data can be used to predict difficulty, outperforming other classifiers, both at the time of coding and afterwards:

Capture

 

The same holds for the bug prediction, the biometric classifiers is the best predictor of future quality concerns, better than code metrics or other metrics.

2016-05-19 11.52.53

Sebastian repeated the study with 5 developers in Switzerland and obtained similar results. A short video with Sebastian:


 

Full paper here: http://www.smueller.li/icse.pdf
Slides here: http://www.slideshare.net/s-mueller/using-biometrics-to-predict-code-quality-online