I’ve been interested to see the ways that the scientific method hits me in the face in this course, and these readings more than any other so far have hit home. The central message of testing seems to be that it is difficult and extremely important to apply the scientific method to practically invisible and infinitely malleable software systems.
I really identified with Phase 0 of the testing mindset as well as the kiddie-testing stage of software testing, because it’s sort of what I do when I’m programming little projects for myself or for homework. Most of my programming experience has dealt with very small, single-use programs that do not require a structured testing framework. Spending a bit of time with the IDE’s debugger tool seems to do the trick for the really difficult assignments! Throwing some haphazard input at the debugger while I search for a bug is solidly in Phase 0 of kiddie programming. But I’m a scientist! I know how to ask questions, form hypotheses, deductively predict experimental results, and inductively refine my hypothesis! But I haven’t ever used these skills in the few months that I’ve been programming! It’s somewhat intimidating to read about the really complex analytical framework of a solid and reliable testing process, but I’m at least more confident knowing that I am fluent in the scientific foundation of what is going on. You learn more with one solid testable hypothesis than with a hundred drunken-walk intuitive guesses, and the time spent up front conceptualizing the problem can save so much time and produce much better results.
I also liked some of the analogies used in Chapter 1. I really appreciated the use of growing pesticide (and antibiotic) resistance to illustrate the increasingly unmanageable complexity that arises when throwing everything you can think of at a problem without really thinking about the whole system. I have a good grasp of this ecological concept, so it helps me start to understand the software point that is being made.
Finally, I am definitely humbled by my first attempt to digest some real-deal computer science journal articles. There aren’t too many microbiological, molecular, physiological, or ecological scientific papers that go over my head, but by the time the software analysis paper got to the algorithm for computing postdominance, I felt once again like a newb. I believe I understand the idea of basic blocks and control flow edges, and how to identify these vertices and edges in a simple algorithm, but I didn’t understand how to input a CFD into an algorithm in order to compute the postdominance relationships. Regarding the mutation testing paper, I think I got the gist of the paper but definitely didn’t grasp the details. It seems like engineers would like to know whether their test suites are doing a good job of testing, and use known mutations of the code to see how often their suites successfully find the faults. I don’t quite understand what manual-seeding would be…isn’t seeding always replicable? Anyway, it’s good to be blown over by science every now and then in case one has gotten to confident.