To study novice software teams, Raphael used students, so he starts by explaining how software testing is taught at the Leibniz university. After an introductory software engineering course, in which the needs and benefits of testing were explained, all software engineering students have to participate in a software project during their BSc. It lasts from 4 month and has three phases: requirements, design and implementation and is executed by 5 students for a real client.
Last year, Raphael did a Grounded Theory study where 97 students were interviewed in groups of 5, where they were interested in whether the lessons on testing from the course were implemented in real life. The students had to had in tests at the end of the project, but no info was provided on what precisely had to be done. Then, Raphael tried to understand
- How did they test?
- What did they test?
- What made it easier and what made it harder?
Students only test at the end of the projects (each line is one group, each row in one of the three phases)
So, why does this happen? Students were explicitly taught to test early. So, Raphael of course asked the students why they did not test earlier. One of them answered:
“The things we were testing were changing heavily. Because everything was so dynamic. At the end, there was no time left”
So, it was not the case that they forgot, or did not see the need. They just did not know how to test a changing system, they saw automated tests like technical debt. Students therefore mainly opted for manual labor, even though they knew that this was not really considered testing. “we just executed it, that is not really testing, but we did not have time” As reason for doing it manually, students lists this was faster, easier and did not involve a learning curve for new tools. Some groups did try GUI testing frameworks, but stopped this because there was no good tutorial available.
Even worse: students felt that writing tests was unproductive, and they felt it would be better for their projects to add more features than to write tests.
At the end of the projects, students were disappointed with the effects of the test suite, because (probably) since it was added at the end, the worst bugs were already found by manual execution, and there were no benefits of regression testing.
Since there was no time for my question during Q&A I am posting it here and I hope Raphael will respond 🙂
I am curious as to how the testing was taught. When I was a student, it was explained like something you just should do because if it inherently good. Like eating vegetables or stretching before a workout. It wasn’t until I had developed a huge code base with multiple people working on it, I began to see the value of tests. If I do not want other breaking my code, for example, I add a test. If I want to be sure a new optimization won’t mess up existing code, I add a test. Shouldn’t we try to have students experience how testing can be beneficial rather than just explaining them?
Not that I would know how to do this, by the way.