Wanna know why I am context-driven? Published on the DEWT blog: Why I am context-driven!
“Because testing (and any engineering activity) is a solution to a very difficult problem, it must be tailored to the context of the project, and therefore testing is a human activity that requires a great deal of skill to do well. That’s why we must study it seriously. We must practice our craft. Context-driven testers strive to become the Jedi knights of testing”. James Bach
This are my sketchnotes from DEWT4. The central theme of the fourth DEWT peer conference was “Teaching Software Testing”.
Click the images to view them full size.
Recently I saw this tweet: “a lot of testers don’t consider alternatives because they don’t know them”. It was a reaction in a discussion about a Dutch article with the title „The days of the ‘Dutch school of testing’ are over”. Jan Jaap claims that Dutch testers suffer from “Law of the handicap of a head start“. Really? I don’t think it is the handicap of a head start. Did we (Dutch testers) ever had a head start? I think it is something that is called “The Dunning-Kruger effect“.
The Dunning-Kruger effect, named after David Dunning and Justin Kruger of Cornell University, occurs when incompetent people not only perform a task poorly or incompetently, but lack the competence to realize their own incompetence at a task and thus consider themselves much more competent than everyone else. Put more crudely, they’re too stupid to realize they’re stupid. The inverse also applies: competent people tend to underestimate their ability compared to others. (Source: Rational Wiki)
In the words of Dunning and Kruger: this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it.
I think many testers don’t realize that they actually know very little about testing at all. And that is why they don’t do anything to get better. They are also not encouraged much by their colleagues. Testing is often underestimated. I guess everyone has examples of managers who do not value testing as much as we would want them to. Testing is often devalued as “pushing buttons” and “everybody can test” is believed by many. I have seen companies who use testing as education, a first step in an IT career…
But there is more… I have worked as a selling and hiring manager and I experienced that there is a lack of competence in spotting talented testers. If you have the right certificates, people believe that you are a good tester. Because they simply do not know how to spot an excellent tester. In my recent job search nobody asked me to SHOW them my testing skills. We only talked about it.
October 23 DEWT and James Bach met in Dordrecht to talk about testing. The subject was “What is context-driven testing?” and more specific “How do you recognize a context-driven presentation?”.
They came together to prepare for the TestNet Autumn Event which was fully dedicated to context-driven testing and had the theme: “Exploring context-driven testing – a new hype or here to stay?”.
DEWT decided earlier this year to write an article about the event to reflect on context-driven testing in the Netherlands. What does the TestNet community have to say about context-driven testing? What can we learn from the event? What can we do to help the community learn? How context-driven is the Dutch Testing community? While discussing this the DEWTs found it hard to find heuristics to “measure” (maybe “recognize” is a better word here) the presentations at the TestNet event.
What is context-driven?
Often context-driven testing is “only” seen as an approach, but it can be more. Actually there are 3 different things that are called context-driven. Testers can be part of one, without necessarily being part of the other.
- Paradigm (world view)
For example: you can use an context-driven approach without being part of the context-driven community. To be in a paradigm you need to have a world view of testing.
||People evaluating a product by learning about it through experimentation
||is a matter organized and motivated by a systematic consideration of
||all the factors that significantly influence the problems and solutions that lie within the scope of their mission
- Attitude: context-driven testing allows to change the approach. An example is “Huib’s Rapid Software Testing”. This Huib’s way of doing testing, inspired by Rapid Software Testing, but he changed his approach to his context. Changing anything he sees fit. Often factory school testers do not want to change their approach and apply their approach the same in every situation.
- Principles: context-driven testing is NOT (only) the seven basic principles, but there is much more. Look what lies behind them, learn about the implicit principles. See slide 9 of this presentation by James Bach.
- Mentality: context-driven testing is about skills, humanity, science, critical thinking, problem solving, non-linearity, investigation, learning, etc.
- Metrics: you can use only those metrics if you understand them and if they solve a certain problem.
- Critical thinking: do you know how? How do you know that you know? How do you get better?
“Science is the Belief in the Ignorance of Experts” — Richard Feynman
Good stuff to look for:
- Heuristics over commandments
- Learning curves
- Compare alternative methods, trying other methods
- To be truly context-driven you have studies and tried the practices you say you do not like
- Discussion of causes and effects assuming open systems. Reject the believe that a project a well defined game that is predictable. Also reject the assumption that a process is unhackable
- Acknowledgement there are people with different opinions, not speaking from authority
- Social science
- Practitioner responsibility
- Refuse to do bad work
- Problem solving
- Skill based work
Be aware of people:
- talking about ‘success’ and ‘failure’ without explaining why and describing the context
- talking about ‘structure’ and ‘chaos’ (often meaning: I am out of control)
- showing no evidence (or using sentences like: “research shows that…”)
- relying on folklore
- using numbers without context
- who are ignorant of social science
- uncritically apply the manufacture metaphor (IT is like a factory)
- apply premature automation
- assuming other people read and follow what is written
- contempt humanism
- trying stuff once and over generalize that to the whole company/world
- who fear variation
- over simplify (approach the world as if it was linear)
- misuse of statical analysis
- who use averages without variance
- who do narrow research
- say or do stuff because their clients want it
- demonize tacit knowledge and idealize explicit knowledge
This is the summary I have made of what we discussed. You could consider this as a heuristic for context-driven presentations. I used this heuristic in my presentation “What is context-driven testing?” at the TestNet event.
||Success / Failure
||Chaos / Structure
|Compare alternative methods
||Lack of evidence / Narrow research
|Cause and effect
||Numbers without context
||Ignorance of social science
||Contempt of humanism
||IT & testing is like manufacturing
||No context mentioned
|Worldview of testing
||Generalizing after one try
||Use of averages without variance
|Allows to change approach
To help the Dutch community evolve to learn, think and do more skilled testing I want to advocate interesting stuff to read to the community. But what should a tester read? Earlier I published a list of popular books for testers. In my search for good books to read, some people replied that they only read blogs and other on-line content. On this blog I recommend great articles and on-line stuff in my great resources list. For testers blogs you can check my colleagues list.
But what do other testers recommend? My curiosity kicked in again and I sent out another email to my tester friends around the world to ask for their favourite blogs.
This is the “top 19″ of most mentioned blogs. In the list are all blogs that got more that 2 votes. I am thrilled to be in this list at all. But it must have been the “availability heuristic” that got me on the 3rd place! Thanks guys
180 different blogs where mentioned by 41 participants. The full list of blogs can be found here. An overview of all participants and their personal lists can be found here.
If this isn’t enough, you can check these listings of tester blogs:
Eurostar Conference – Gothenburg Sweden
6th November 2013 – 11:45 uur
How To Become A Great Tester
Agile Testing Days – Potsdam Germany
28th October 2013 – 9:00 uur
The International Society for Software Testing has launched – lets start putting the common sense & humanity back into testing!
I was fortunate enough to be asked to be one of the founding members of the ISST. Looking down the list of members, I know most of them personally & can vouch for all of them.
The society is still in its infancy & there are many questions to be answered but one thing is for certain, there are many exciting times ahead & I am really glad to be onboard.
Go & check it out for yourself & let me know what you think:
I love to read and I own many books on testing, software, management and other stuff that relates to my work.
But what should a tester read? On this blog I recommend several books in my great resources list. And what do other testers recommend? My curiosity kicked in and I sent out an email to my tester friends around the world.
Every year the Dutch association for software testers TestNet organizes two one-day conferences. This year TestNet has chosen context-driven testing as the theme for their autumn event in October (call for papers is here). To help the Dutch community evolve to learn, think and do more skilled testing I want to advocate some interesting books to the community. Here I need your help! Please send me your personal top 10 of best books testers should read. It can be any book, it doesn’t have to be a book on testing… I will collect the submissions and create a list of most popular books amongst testing professionals. Please send me your list of favourite books! Hope to hear from you soon.
One of the testers replied that he could not send a list. “What you should be reading depends on what you are ready to learn about next, and that varies from person to person”. And I agree with this statement. This list of books can be useful when used as a list to inspire. Another tester replied: “the reading that has been most helpful in my career has been centered around blogs and twitter far more than it has been around books”. For testers blogs you can check my colleagues list. Maybe creating a list of most popular testers blog will be my next project
Anyway, this is the top 10 of most mentioned books:
- Lessons Learned in Software Testing – Cem Kaner, James Bach, Brett Petticord (31 votes)
- Perfect Software and other Illusions about Software Testing – Gerald M. Weinberg (19 votes)
- Agile Testing – Lisa Crispin and Janet Gregory (14 votes)
- Thinking fast and slow – Daniel Kahneman (12 votes)
- How to Break Software – James A. Whittaker (11 votes)
- Tacit and Explicit Knowledge – Harry Collins (10 votes)
- Explore It! – Elisabeth Hendrickson (9 votes)
Secrets of a Buccaneer-Scholar – James Bach (9 votes)
A Practitioner’s Guide to Software Test Design – Lee Copeland (9 votes)
- An introduction to general systems thinking – Gerald M Weinberg (6 votes)
Peopleware: Productive Projects and Teams – Timothy Lister & Tom DeMarco (6 votes)
Quality Software Management Vol. 1 Systems Thinking – Gerald M. Weinberg (6 votes)
Secrets of Consulting – Gerald M. Weinberg (6 votes)
Testing Computer Software – Cem Kaner, Jack Falk, Hung Q. Nguyen (6 votes)
The Black Swan – Nassim Nicholas Taleb (6 votes)
180 different books where mentioned by 43 participants. 15 (!) different books by Jerry Weinberg where mentioned. The full list of books can be found here. An overview of all participants and their personal lists can be found here.
Normally, I do not blog about personal, non-testing related stuff. But this blog post is different. If you are not interested in my personal emotions and private life, no problem at all, I understand. Stop reading now and come back at another time.
My mother (http://ineterhorst.wordpress.com/) was diagnosed with lung cancer in June 2011. It didn’t look good and we all feared that she would not be with us anymore within a year. She got radiation and chemo treatment and a little medical miracle happened: the cancer disappeared! Almost four months ago I was with her at the doctor for the Quarterly checkup when he told us that “there was no reason to assume there are any active cancer cells in her body”. We celebrated and life went back to normal after almost 2 years of “hassle”. Then she got terrible headaches two months ago that didn’t go away for weeks and just to be sure the doctor wanted to make an MRI scan of her head. Her head was okay, but they found a tumor on her upper cervical spine. This time the cancer had spread and a PET scan showed cancer at several places in her body. This news was given to her on her birthday in April. It was devastating news and the prospects are unknown and not good, but it is a matter of months the doctor said…
My world stopped turning for a moment…
On Dutch television I saw this commercial titled “live today” by a funeral insurance and services company, which inspired me to write this blog post. It’s message is: “Why wait with saying something nice if it can be done today?”.
This blog is to tell you are the best mother I could have wished for. Although I didn’t make it easy for you when I was younger, you always supported me in everything I did. Even when things weren’t easy for yourself during your divorce or other setbacks, a simple request for help was enough to trigger you helping me over and over again. It didn’t matter if it was in the middle of the night, at the other side of the country or you had to prematurely terminate your holiday, you always came to my rescue! You did all that was in your power to give me and my brother everything we needed. Even when I started working and earning money, you kept buying stuff for us as presents or because you thought we might need it … Nothing was more important in the world than me and my brother. That might be a normal thing for many but I feel it is a very special thing to do and I want to thank you for that! There is so much more I want to say to you and I hope you stay around long enough to talk about all those things. You are the best mother I could ever wish for. Thank you for being my mother!
I love you!