Category: Learning (page 1 of 2)

Does certification have value or not?

I read a blogpost in Dutch named “Does certification have value or not?” by Jan Jaap Cannegieter. I wanted to reply, but there was no option to reply, so I decided to turn my comments into a blogpost. Since the original blogpost is in Dutch I have translated it here.

The proponents claim that you prove to have a foundation in testing with certification, you possess certain knowledge and it supports education.” (text in blue is from the blogpost, translated by me).

Three things are said here:

  1. prove to have foundation
    Foundation? What foundation? You learn a few terms/definitions and an over-simplified “standard” process? And how important is this anyway? Also the argument of an common language is nicely debunked by Michael Bolton here: “Common languages aint so common
  2. possess certain knowledge
    When passing an exam, you indicate to be able to remember certain things. It doesn’t prove you can apply that knowledge. And is that knowledge really important in our craft? I think knowledge is over appreciated and skills are undervalued. I’d rather have someone who has the skills to play football well instead of somebody who knows the rules. From a foundation training, wouldn’t you at least expect to learn the basic testing skills? In no ISTQB training, students use a computer. Imagine giving someone a driver’s license without having ever sat in a car …
  3. supports education
    Really? Can you tell me how? I think the opposite is true! As an experienced teacher (I also did my share of certification training in the past), my experience is that there is too much focus on passing the exam rather than learning useful skills. Unfortunately, preparing the students for the exam takes a lot of time and focus away from the stuff that really matters. Time I would rather use differently.

Learning & tacit knowledge

So how do people learn skills? There are many resources I could point to. Try these:

In his wonderful book “The psychology of software testing” John Stevenson talks about learning op page 49:

The “sit back and listen” approach can be effective in acquiring information but appears to be very poor in the development of thinking skills or acquiring the necessary knowledge to apply what has been explained. The majority of trainers have come to realise the importance of hands on training “Learn by doing” or “experiential learning”.

John points to resources like: Learningfromexperience.com and the book “Experiential learning: experience as the source of learning and development” by David Kolb. Also Jerry Weinberg has written books on experiential learning.

The resources on learning skills mentioned by me earlier, will tell you that experienced people know what is relevant and how things are related. Also practice, experimentation and reflection are important parts of learning. Learning of a skill  depends heavily on tacit knowledge. On page 50 in his book John Stevenson writes:

Pakivi Tynjaklak makes an interesting comment in the International Journal of Educational research: “The key to professional development is making explicit that which has earlier been tacit and implicit, and thus opening it to critical reflection and transformation” – This means that what we learn may not be something we can explain easily (tacit) and that as we learn we try to find ways to make it explicit. This is the key to understanding and knowledge when we take something which is implicit and make it explicit. Therefore, able to reflect on what is learned and explaining our understanding.

And since testing is collecting information or learning about a product, the importance of tacit knowledge also applies to testing: John writes in his book on page 197:

However testing is about testing the information we do not know or cannot explain (the hidden stuff). To do this we have to use tacit knowledge (skills, experience, thinking) and we need to experience it to be able to work it out. This is what is meant by tacit knowledge“.

Back to the blogpost:

The opponents say certification only shows that you’ve learned a particular book well, it says nothing about the tester’s ability and can be counterproductive because the tester is trained to a standard tester.

  • Learned a particular book
    Agree, see arguments 1 and 2 above.
  • it says nothing about the tester’s ability
    Agree, see my argumentation on skills in point 2 above: “knowledge is over appreciated and skills are undervalued”. To learn we need practice and reflection. Also tacit knowledge is an important part of learning.
  • Trained to a standard tester
    Agree. No testing that I know of, is standard. Testing is driven by context. And testers with excellent skills have the ability to work in any context without using standards or templates. Have a look at the Ted Talk by Dr. Derek Cabrera “How Thinking Works“. He explains that critical thinking is a skill that is extremely important. Schools (and training providers)  nowadays are over-engineering the content curriculum: students do not learn to think, they learn to memorize stuff. Students are learned to follow instructions, like painting by the numbers or fill in templates. To fix this, we need to learn how to think better! Learning to paint by numbers is exactly what certification based on knowledge does with testers! Read more about learning, thinking and how to become an excellent tester in one of my earlier blogpost: “a road to awesomeness“.

Comparison with driving license
Does a driving license show anything? Well, at least you have studied the traffic rules well and know them. And, while driving, it is quite useful if we all use the same rules. If you doubt that, you should drive a couple of rounds in Mumbai.

In testing we should NEVER use the same rules as a starting point. “The value depends on the context!” . Driving in Mumbai or anywhere by strictly adhering to the rules, will result in accidents and will get you killed. You need skills to drive a car and be able to anticipate, observe, respond to unexpected behaviour of others, etc. This is what will keep you out of trouble while driving.

As I explained earlier on the TestNet website: this comparison is wrong in many ways. For a driver’s license, you must do a practical exam. And to pass the practice exam, most people will take lessons! You will be driving for at least 20 hours before your exam. And the exam is not a laboratory: it means you go on the (real) road with a real car. A multiple-choice exam does not even remotely resemble a real situation. That’s also how pointless ISTQB or TMap certificates are. Nowhere in the training or the exam, the student uses software nor does the the student has to test anything!

This is the heart of the problem! People do not learn how to test, but they learn to memorize outdated theory about testing. Unfortunately in many companies new and inexperienced testers are left unattended in complex environments without the right supervision and support!

So what would you prefer in your project: someone who can drive a car (someone who has the basic skills to test software), or someone knows the rules (someone who knows all the process steps and definitions by heart?). In addition, ISTQB states that the training is intended for people with 6 months of experience here.  So how are new testers going to learn the first 6 months?

The foundation for a tester?

The argument that the ISTQB foundation training provides a basis for a beginner to start is nonsense! It teaches the students a number of terms and a practically unusable standard process. In addition, there is a lot of theory about test techniques and approaches, but the practical implementation is lacking. There are many better alternatives as described in the resources earlier in this blogpost: learning by doing! Of course with the right guidance, support and supervision. Teach beginners the skills to do their work, as we learn the skills to drive a car in driving lessons. In a safe environment with an experienced driver next to us. Until we are skilled enough to do it without supervision. Sure, theory and explicit knowledge are important, but skills are much more important! And we need tacit knowledge to apply the explicit knowledge in our work.

So please stop stating that foundation training like TMap and ISTQB are a good start for people to learn about testing. There aren’t. Learning to drive a car starts with practicing actually driving the car.

Jan Jaap states he thinks a tester should be certified: “And what about testers? I think that they should also be certified. From someone who calls himself a professional tester we may expect some basic knowledge and knowledge about certain methods?“.
I think we may expect professional testers to have expertise in different methods. They should be able to do their job, which demands skills and knowledge. We may expect a bit more from professional testers than only some basic knowledge and knowledge about methods.

Many of the well-known certification programs originated when IT projects looked very different and, in my view, these programs did not grow with the developments. So they train for the old world

Absolutely true.

Another point where the opponents have a point is the value purchasing departments or intermediaries attach to certificates. In many of the purchasing departments and intermediaries, the attitude seems that if someone has a certificate, it is also a good tester. And to say that, more is needed.

It is indeed very sad that this is the main reason why certificates are popular. Many people get certificated because of the popular demand of organisations who do not recognise the true value of these certificates. Organisations are often not able (or do not what to spend the time needed) to recognise real professional testers and so they rely on certificates. On how to solve this problem I did a webinar “Tips, Tricks & Lessons Learned for Hiring Professional Testers” and wrote an article about it for Testing Circus.

Learning goals & value

On the ISTQB website I found the Foundation Level learning goals. Let’s have a look at them. Quotes from the website are in purple.

Foundation Level professionals should be able to:

  • Use a common language for efficient and effective communication with other testers and project stakeholders.
    Okay, we can check if the student knows how ISTQB defines stuff with an exam. However, understanding what it means or how to deal with it in a daily practice is very different. Also, again, common language is a myth.
  • Understand established testing concepts, the fundamental test process, test approaches, and principles to support test objectives.
    Concepts and test process: okay, you can check if a student remembers these. However, the content is old and outdated and in many places incorrect! I think understanding approaches cannot be checked in a (multiple-choice) exam. Maybe some definitions, but how to apply them? No way.
  • Design and prioritize tests by using established techniques; analyze both functional and non-functional specifications (such as performance and usability) at all test levels for systems with a low to medium level of complexity.
    Design and prioritize tests? Interesting. Where is this trained? Or being tested in the exam? Analyse specifications? That is not even part of the training. Applying some techniques is, but there is a lot more to designing and prioritize tests and analysing specifications.
  • Execute tests according to agreed test plans, and analyze and report on the results of tests.
    Execution of tests nor analysis or reporting of test results is part of the exam. In class only the theory about test reporting is discussed but never practiced.
  • Write clear and understandable incident reports.
    How do you check this with a multiple choice exam? And how you train this skill without actually testing software in class? No exercises in class that actually ask you to write such reports.
  • Effectively participate in reviews of small to medium-sized projects.
    The theory about reviews is part of the class. To effectively participate in reviews, you need to do it and learn from experience.
  • Be familiar with different types of testing tools and their uses; assist in the selection and implementation process.
    Some tools and their goals and uses are mentioned in class. So I will agree with the first part. But to assist in selection and implementation, again you need skills.

So looking at the learning goals above, I doubt if the current classes teach this. The exam for sure doesn’t prove that a foundation level professionals is be able to do this things. A lot of promises that are just wrong! Certificate training like ISTQB-F and TMap as they are now are simply not worth the money! The training and exam are mostly 3 days and cost around 1.700 euro in the Netherlands. I think that is a crazy investment for what you get in return… There are better ways to invest that money, time and effort!

I think that a more valuable 3 day foundation training is doable. But surely not the way it is done now by TMap or ISTQB. I’ve written a blog post about it years ago: “What they teach us in TMap Class and why it is wrong!“.


More blogs / presentations about certification:

A Road to Awesomeness

At TestBash Manchester I did a presentation titled “A road to awesomeness”. In this talk I tried to explain how testers can become awesome. I talked about learning and testing skills. To prepare this talk I created a mind map to list skills and ways to learn. The mind map is here and will be continuously updated in the future. Let me know if you have things to add.

The 4-hour tester

In another talk at TestBash Manchester Helena Jeret-Mäe and Joep Schuurkes talked about the 4-hour Tester Experiment.  They took the the challenge of teaching new testers useful skills in only 4 hours. They focus on 5 skills:

  • Interpretation
  • Modeling
  • Test design
  • Note taking
  • Bug reporting

Their experiment is an interesting try to teach software testers essential skills in only 4 hours. Of course it is impossible to teach anyone to test in 4 hours.  But the 4-hour tester has simple exercises that take at most one hour to complete. Interesting approach to help inexperienced testers to get started.

Learning

To become awesome, you have to learn a lot. But how do we learn? I like the 10/20/70 model: 70 percent of learning comes form job-related experiences, 20 percent from interactions with others, and 10 percent from formal educational. So training is only a small part of a learning journey. Interaction with others, like communities and networking, coaching and mentoring are a part of it as well. But the biggest part of learning takes place from work related experience. This doesn’t mean just working. Many years of working doesn’t let you necessarily learn effective if you do not get feedback, reflect on what you do in the right environment. Doing the same work for 10 years doesn’t give you 10 years of useful experience. If you never retrospect, reflect or get any feedback, I’d say you have 10 times 1 year of experience. To make learning effective in your job you need:

  • Concrete, challenging and achievable tasks
  • Realistic application of skills, processing and reflection
  • Personal interpretation, exchange with others and constructive feedback
  • A safe environment to experiment and make mistakes

But there is more to learning. In her TED Talk “Learning how to learn” Barbara Oakley talks about two fundamental learninglearning-how-to-learn modes of our brain: focused mode and the more relaxed diffused mode. When your are learning, you want to go back and forth between these two modes. When you are stuck, you defocus going into the diffused mode, generating new ideas. If you want to learn more about this, I recommend the coursera MOOC “Learning How to Learn: Powerful mental tools to help you master tough subjects“. Two of the things I took from that course is that active listening (e.g. by asking questions) is far more effective. Also learning by doing is a great way to learn fast.

Learning is the most important skill for a tester. And learning is closely related to thinking skills. I’ll come to that later.

What makes an awesome tester?

An awesome testers has many skills. In my blogpost “Heuristics for recognizing professional testers” I described 18 heuristics to recognize professional testers. This was the starting point for my talk at TestBash Manchester. The first steps to help me think about aspects to become awesome, was creating a mind map with a couple of things to focus on: what makes an awesome tester: who (characteristics), what (skills) and how (ways to learn and how to become an expert).

awesome-testers-mapThere are many skills that make an awesome tester. I think the most important skill is the ability to learn! Remember the definition of testing we use in Rapid Software Testing: “Testing is evaluating a product by learning about it through exploration and experimentation”. Besides learning I identified  5 categories of skills testers need:

  • Thinking skills
  • Testing skills
  • Technical skills
  • Domain skills
  • Soft skills

And of course there are many, many skills that make these categories. Have a look at the mind map to learn more about these skills. It is very important to recognise what skills are involved in being a great tester. If you want to learn, you need to know what to focus on. Not knowing which skill to train, will result in unfocussed and ineffective learning. I see many testers apply test techniques without knowing what skills are used and which methods are underneath the technique. This makes them apply techniques as recipes or trick. Being able to apply approaches, tactics and techniques effectively in any situation requires the right skills.

Thinking skills

Learning and thinking are closely related. While researching thinking and thinking skills, I stumbled upon a great TED talk by Dr. Derek Cabrera called ”How Thinking Works”. He talks the schooling system teaching about 4 universal thinking skills. Schools nowadays are over-engineering the content curriculum: students do not learn to think, they learn to memorize stuff. Kids are learned to follow instructions, like painting by the numbers. To fix this, we need to learn how to think better and for that he suggests four thinking skills: DSRP. A simple set of rules to become better in systems thinking.

  • Making Distinctions – Any idea or thing can be distinguished from other ideas or things
  • Organizing Systems – Any ideas or thing can be split into parts or lumped into a whole
  • Recognizing Relationships – Any idea or thing can relate to other ideas or thing
  • Taking Perspectives – Any idea or thing can be the point or the view of a perspective

In his book “Systems thinking made simple“, Dr. Cabrera talks metacognition. “Systems thinking is a particular type of metacognition that focuses on and attempts to reconcile the mismatch between one’s mental models and how the real world works”, he continues to describe acts of metacognition: “Awareness that everything you experience is a mental model that approximates (either poorly or well) the real world”, “The ability to distinguish among cognition (thinking), emotion (feelings), and conation (motivations) and the awareness of how these different human capacities influence our mental models and behavior”. Recognizing that models are a big part of our thinking makes you a better tester. But also that your biases and emotions influence your thinking are great insights we have to take into account in our testing.

A second TED talk I recommend is “How to think, not what to think” by Jesse Richardson. He says: “The way we approach education needs to adapt! What’s different in teaching children how to think, we are involving them in the process of their own learning. Instead of just telling them to memorize the right answer, we ask them to engage their own minds, their own awareness by questioning things, attaining understanding not just knowledge. And that involvement, that engagement, is so important, because it keeps the spark of curiosity alive.”

He mentions the famous TED Talk by Ken Robinson: “Do schools kill creativity?“. One of my all time favorites.

Activities in testing
To fully understand what skills we use in testing, I made a list of testing activities. In Rapid Software Testing we teach the 9 Elements of testing and I took these as a starting point.

  1. Model the test space and risks
  2. Determine coverage
  3. Determine oracles
  4. Determine test procedures
  5. Configure the test system
  6. Operate the test system
  7. Observe the test system
  8. Evaluate the test results
  9. Report test results

The next step was to zoom in on the different elements in the list. Zooming in on “Model the test space and risks” we can distinguish:

  • Context Analysis
  • Creating a product Coverage Outline
  • Test Plan
  • Test scope
  • Risk & Value Analysis

I did this with all 9 elements and came to a pretty long list of activities. Maybe you can think of more activities testers do, let me know if you do.

This (making distinctions or factoring) exercise resulted in a huge list of things we do in testing and products we make. The next step was to lists the skills that we use, doing these activities. Obviously the first thing I thought of was learning and thinking. Researching this, I stumbled upon a lot of interesting stuff, of which part I have described earlier. But there was more, way more as you can see in my mind map. And this mind map expands every time I visit it.  Factoring the activities and skills helps me to see what is really going on while we are testing.

So how do you become awesome?

So now we have a map of activities and skills. But how do we learn them? And where do we start?

The most important part of becoming awesome is knowing what to learn and how. The first step is to know what to learn and focus on that. Stop trying to learn 10 things at the same time, it doesn’t work! A way to get to know what you want to learn is writing a Personal Development Plan answering 5 simple questions: Who am I? What are my skills? What do I want? What do I need?  How do I get there? This can give you great insight in your skills, ambition and learning needs. The second step is to find mentors who can help you and give you feedback. After that it is “just” a matter of doing it and practice!

I learned to be awesome by doing many things:

  • Observing others and myself: by becoming manager and a coach I found out that observing others gives you great insight in how testing is done and what the common problems are for people who are learning how to test. Observing myself, which is quite difficult in the beginning, I learned what I was doing and I found out that this was a big eye-opener and learning enabler. Using a journal a writing down stuff every day on my way home, gave me great insights in what my problems were. Knowing your problems, is the first step to solving them.
  • Explaining, presenting, teaching & coaching: by explaining stuff to others, you learn to structure your thoughts. To be able to present or  teach you really have to know your stuff. And still every time I teach, I learn new things about what I teach. New examples, new problems, new ways of doing things. Also answering difficult questions is very helpful learn deeply about topics. That is way I think we should encourage debate and questioning at conference way more.
  • Pairing: pairing is a fun and nice way to learn from others. Seeing how others do things is helpful and has many learning opportunities. Also the feedback you get from your pairing-partner is valuable.
  • Writing my blog: same thing as explaining and teaching. Writing this blog post made me think about what o write and how to explain. I needed to structure my thoughts. Great exercise.
  • Keeping a journal: see observing other and myself.
  • Always having a notebook with me: to practice note taking and it also helps me not to forget interesting stuff that others tell me. It is great to read back notes from conferences and courses from way back. It refreshes my memory and I often see new insights I didn’t see before.
  • Discussing & debating testing: see explaining.
  • Trying new stuff: experimenting is fun and also a way to learn about new techniques, tools, approaches, etc.
  • My coaches and mentors: I have had and still have many mentors. All these people help me to become better every day. It is great to have many mentors, each with their own expertise and specialties. I have a big international network of people I know and who I work with. So many sources of knowledge and skill help me quickly find an answer to almost every difficult testing problem I have.

My advise is to try the same exercise I did creating the mind map. Create your own model of test activities and skills. Observe what you do, how you do it and make your own mind map (or any other way you would like to model it). Come back and compare yours to my mind map.

So why is this important?

I see may similarities with the problems in the current schooling system and in testing:  over-engineering the content curriculum, students do not learn to think, they learn to memorize stuff and to follow instructions (like painting by the numbers). In most testing classes testers learn tricks, procedures and the use of template like approaches in stead of the required skills to do an excellent job. In Rapid Software Testing (RST) we teach thinking skills. We learn testers how to effectively think and solve problems instead of using tricks, templates and standards to do their work. By learning to think better, you learn better. And isn’t testing all about learning? Also, if you are able to think, you are able to do better work, apply your skills in the right way. Adapt to changing context, find your own solution to problems that occur. We also teach testers to dynamically manage their focus: focus/defocus is similar to fundamental learning modes Prof. Dr. Barbara Oakley talks about in her TED Talk.

The DSRP model (distinctions, systems, relationships, perspectives) is very useful in testing. In RST we talk about models and learn our students how to model (distinctions, systems and relationships) and to think from different perspectives using a diverse strategy using heuristics. Great to see that the stuff we teach is actually backed up by scientists who study metacognition (thinking about thinking) and epistemology (the study of knowledge). Making a product coverage outline for instance helps you see distinctions and relations and helps systems thinking.

Good luck on your journey to awesomeness!

References

The slides I used in Manchester are here: slideshare.
The updated slides are here: pdf.

All talks at TestBash are recorded. My talk is here.

Why testers are not taken seriously…

5784a3e8c82ffdc1da395f1ded31eab6Some time ago I was invited to talk to a group of testers at a big consultancy firm in the Netherlands. They wanted to learn more about context-driven testing. I do these kind of talks on a regular basis. During these events, I always ask the audience what they think testing is. It surprises me each time that they cannot come up with a decent definition of testing. But it gets worse when I ask them to describe testing. The stuff most people come up with is embarrassingly bad! And it is not only them, a big majority of the people who call themselves professional testers are not able to explain what testing is and how it works…

How can anybody take a tester serious who cannot explain what he is doing all day? Imagine a doctor who tells you he has to operate your knee.

Doctor: “I see there is something wrong there
Patient: “Really? What is wrong doctor?
Doctor: “Your knee needs surgery!
Patient: “Damn, that is bad news. What are you going to do doctor?
Doctor: “I am going to operate your knee! You know cut you with a scalpel and make it better on the inside!
Patient: “Okay… but what are you going to do exactly?
Doctor: “Euh… well… you see… I am going to fix the thingy and the whatchamacallit by doing thingumabob to the thingamajig. And if possible I will attach the doomaflodgit to the doohickey, I think. Get it?
Patient: “Thank you, but no thanks doctor. I think I’ll pass

But it is much worse… Many testers by profession have trouble explaining what they   are testing and why. Try it! Walk up to one of your tester colleagues and ask what he or she is doing and why. 9 out of 10 testers I have asked this simple question begin to stutter.

How can testers be taken seriously and how they learn a profession when they cannot explain what they do all day?

albert-einstein-if-you-cant-explain-it-simply-you-dont-understand-it-well-enoughOnly a few testers I know can come up with a decent story about their testing. They can name activities and come up with a sound list of real skills they use. They are able to explain what they do and why. At any given time they are able to report progress, risks and coverage. They will be happy to explain what oracles and heuristics they are using, know what the product is all about and practice deliberate continuous learning. In the Rapid Testing class (in NL) we train testers to think and talk about testing with confidence.

How about you? Can you explain your testing?

How I became a Rapid Software Testing trainer

TestingTrapeze2015OctoberIn the October edition of Testing Trapeze my experience report “How I became a rapid software testing trainer” is published. It has been an amazing journey! When I started this journey, I thought it would be much easier. It was a lot of hard work, a struggle sometimes, but it was totally worth it! A journey in which I learned a lot about myself and the testing craft. I am looking forward teaching the next class in December in the Netherlands this year! See you there?

Testing Trapeze is a free high quality testing magazine from Australia and New Zealand, advocating good testing practices.

 

 

Stay Hungry. Stay Foolish.

Yesterday the wonderful people from Lucky Cat Tattoo put a piece of art on my arm.

Stay Hungry. Stay Foolish.
This was the ‘farewell message’ of the whole earth catalog. It was placed on the back cover of the final edition in 1974. Steve Jobs used this quote in his famous commencement speech in 2005 on Stanford University. While writing this blog post I found this article by a neuroscientist explaining what the quote means…

Hungry
Hungry points to always looking for more, striving to improve, being ambitious and eager. Everything I do, I do with passion. What keeps me moving is energy and passion. I need challenges to feel comfortable. I want to be good in almost everything I do. Not just good, but the very best. All that makes that my surroundings sometimes suffer from me because I always want to do more and do better. Fortunately, I have an above average energy level and that helps me do what I do. This video of Steve Jobs summarizes how I work.

Foolish
Foolish points to taking risks, feeling young, being daring, exploratory and adventurous. Like a child learning how the world works by trying everything. It also reminds us not always do what people expect us to do and not always take the traditional paths in life.

I’m curious. This is an important characteristic in a software tester. Richard Feynman, the Nobel Prize winner was a tester, though he was officially natural scientist. In this video, “The pleasure of finding things out” he talks about certainty, knowledge and doubt (from 47:20). Critical thinking about observations and information is important in my work! Richard Feynman never took anything for granted. He took the scientific approach and thought critical about his work. He doubted a lot and asked many questions to verify.

Because of my curiosity, I want to know everything. This has one big advantage: I want to develop and practice continuous learning. The great thing about my job is that testers get paid for learning: testing is gathering information about things that are important to stakeholders to inform decisions. I love to read and I read a lot to discover new things. I also ask for feedback on my work to develop myself continuously. Lately, experiential learning has my special attention. I wrote a column about why I like this way of learning. When it comes to learning, two great TED videos come to mind: “Schools kill creativity” and “building a school in the cloud“. These videos tell a story about how we learn and why schools (or learning in general) should change.

Think different!

Here’s to the crazy ones. The misfits. The rebels. The troublemakers. The round pegs in the square holes. The ones who see things differently. They’re not fond of rules. And they have no respect for the status quo. You can quote them, disagree with them, glorify or vilify them. About the only thing you can’t do is ignore them. Because they change things. They push the human race forward. And while some may see them as the crazy ones, we see genius. Because the people who are crazy enough to think they can change the world, are the ones who do.

jediJedi
Besides I am a huge fan of Star Wars, the Jedi sign has a deeper meaning:
“Because testing (and any engineering activity) is a solution to a very difficult problem, it must be tailored to the context of the project, and therefore testing is a human activity that requires a great deal of skill to do well. That’s why we must study it seriously. We must practice our craft. Context-driven testers strive to become the Jedi knights of testing.” (source: “The Dual Nature of Context-Driven Testing” by James Bach).

I believe that learning is not as simple as taking a class and start doing it. To become very good at something you need mentors who guide you in your journey. I strongly believe in Master-Apprentice. Young Padawans are trained to become Jedi Knights by a senior (knight or master) who learns them everything there is to know. The more they learn, the more responsibility the student gets. That is why I am happy that I have mentors who teach, coach, mentor, challenge and guide me. And that is why I am a mentor for others doing the same. Helping them to learn and become better.

bearBear
In 2013 I took the awesome Problem Solving Leadership aka PSL workshop facilitated by Jerry Weinberg, Esther Derby and Johanna Rothman. This amazing six day workshop gave me many valuable insights in myself and how to be a better leader by dealing effectively with problems (or as Jerry says it: enhancing the environment so that everybody is empowered to contribute creatively to solve problems). During the social event we visited the Indian Pueblo Cultural Center which also made a big impression on me. Here I bought a talisman stone with a bear engraved. In native American beliefs the bear symbolizes power, courage, freedom, wisdom, protection and leadership (more info on bear symbolic: herehere and here).

A new level of testing?

Yesterday I saw this awesome video of Lars Andersen: a new level of archery. It is going viral on the web being watched over 11 million times within 48 hours. Now watch this movie carefully…

The first time I watched this movie, I was impressed. Having tried archery several times, I know how hard it is to do. Remember Legolas from the Lord of Rings movie? I thought that was “only” a movie and his shooting speed was greatly exaggerated. But it turns out Lars Andersen is faster than Legolas. My colleague Sander send me an email telling me about the movie I just watched saying this was an excellent example of craftsmanship, something we have been discussing earlier this week. So I watched the movie again…

Also read what Lars has to say in the comments on YouTube and make sure you read his press release.

This movie is exemplar for the importance of practice and skills! This movie explains archery in a way a context-driven tester would explain his testing…

0:06 These skills have been long since been forgotten. But master archer Lars Andersen is trying to reinvent was has been lost…

Skills are the backbone of everything being done well. So in testing skills are essential too. I’ll come back to that later on. And the word reinvent triggers me as well. Every tester should reinvent his own testing. Only by going very deep, understand every single bit, practice and practice more, you will truly know how to be an excellent tester.

0:32 This is the best type of shooting and there is nothing beyond it in power or accuracy. Using this technique Larsen set several speed shooting records and he shoots more than twice as fast as his closest competitors…

Excellent testers are faster and better. Last week I heard professor Chris Verhoef speak about skills in IT and he mentioned that he has seen a factor 200 in productivity difference between excellent programmers and bad programmers (he called them “Timber Smurf” or “Knutselsmurf” in Dutch).

0:42 … being able to shoot fast is only one of the benefits of the method

Faster testing! Isn’t that what we are after?

0:55 Surprisingly the quiver turned out to be useless when it comes to moving fast. The back quiver was a Hollywood Myth…

The back quiver is a Hollywood myth. It looks cool and may look handy on first sight, since you can put a lot of arrows in it. Doesn’t this sound like certificates and document-heavy test approaches? The certificates looks good on your resume and the artifacts look convenient to help you structure your testing… But turn out to be worthless when it comes to test fast.

1:03 Why? Because modern archers do not move. They stand still firing at a target board.

I see a parallel here with old school testing: testers had a lot of time to prepare in the waterfall projects. The basic assumption was that target wasn’t moving, so it was like shooting at a target board.  Although the target proved always to be moving, the testing methods are designed for target boards.

1:27 Placing the arrow left around the bow is not good while you are in motion. By placing your hand on the left side, your hand is on the wrong side of the string. So you need several movements before you can actually shoot..

Making a ton of documentation before starting to test is like several movements before you can actually test.

1:35 From studying old pictures of archers, Lars discovered that some historical archers held their arrow on the right side of the bow. This means that the arrow can be fired in one single motion. Both faster and better!

Research and study is what is lacking in testing for many. There is much we can learn from the past, but also from social science, measurement, designing experiments, etc.

1:56 If he wanted to learn to shoot like the master archers of old, he had to unlearn what he had learned…

Learning new stuff, learning how to use heuristics and train real skills, needs testers to unlearn APPLYING techniques.

2:07: When archery was simpler and more natural, exactly like throwing a ball. In essence making archery as simple as possible. It’s harder how to learn to shoot this way, but it gives more options and ultimately it is also more fun.

It is hard to learn and it takes a lot of practice to learn to do stuff in the most efficient en effective way. Context-driven testing sounds difficult, but in essence it makes testing as simple as possible. That means it becomes harder to learn because it removes all the methodical stuff that slows us down. These instrumental approaches trying to put everything in a recipe so it can be applied by people who do not want to practice, make testing slow and ineffective.

2:21 A war archer must have total control over his bow in all situations! He must be able to handle his bow and arrows in a controlled way, under the most varied of circumstances.

Lesson 272 in the book Lessons Learned in Software Testing: “If you can get a black belt in only two weeks, avoid fights”. You have to learn and practice a lot to have total control! That is what we mean by excellent testing: being able to do testing in a controlled way, under the most varied of circumstances. Doesn’t that sound like Rapid Software Testing? RST is the skill of testing any software, any time, under any conditions, such that your work stands up to scrutiny. This is how RST differs from normal software testing.

2:36 … master archers can shoot the bow with both hands. And still hit the target. So he began practicing…

Being able to do the same thing in different ways is a big advantage. Also in testing we should learn to test in as many different ways as possible.

3:15 perhaps more importantly: modern slow archery has led people to believe that war archers only shot at long distances. However, Lars found that they can shoot at any distance. Even up close. This does require the ability to fire fast though.

Modern slow testing has led to believe that professional testers always need test cases. However, some testers found that they could work without heavyweight test documentation and test cases. Even on very complex or critical systems also in a regulated environment. This does require the ability to test fast though.

Lars Andersen: a new level of archery3:34 In the beginning archers probably drew arrows from quivers or belts. But since then they started holding the arrows in the bow hand. And later in the draw hand. Taking it to this third level. That of holding the arrows in the bow hand, requires immense practice and skill and only professional archers, hunters and so on would have had the time for it. … and the only reason Lars is able to do it, is he has been years of practicing intensely.

Practice, practice, practice. And this really makes the difference. I hear people say that context-driven is not for everybody. We have to accept that some testing professional only want to work 9 to 5. This makes me mad!

I think professional excellence can and should be for everyone! And sure you need to put a lot of work in it! Compare it to football (or any other good thing you want to be in like solving crossword puzzles, drawing, chess or … archery). It takes a lot of practice to play football in the Premiership or the Champions League. I am convinced that anyone can be a professional football player. But it doesn’t come easily. It demands a lot of effort in learning, drive (intrinsic motivation, passion), the right mindset and choosing the right mentors/teachers. Talent maybe helps, and perhaps you need some talent to be the very best, like Lionel Messie But dedication, learning and practice will take you a long way. We are professionals! So that subset of testers who do not want to practice and work hard, in football they will soon end up on the bench,  won’t get a new contract and soon disappear to the amateurs.

Lars Andersen: a new level of archery4:06 The hard part is not how to hold the arrows, but learning how to handle them properly. And draw and fire in one single motion not matter what methods is used.

Diversity has been key in context-driven testing for many years. As testers we need to learn how to properly use many different skills, approaches, techniques, heuristics…

4:12 It works in all positions and while in motion…

… so we can use then in all situations even when we are under great pressure, we have to deal with huge complexity, confusion, changes, new insights and half answers. 

5:17 While speed is important, hitting the target is essential.

Fast testing is great, doing the right thing, like hitting the target is essential. Context-driven testers know how to analyze and model their context to determine what the problem is that needs to be solved. Knowing the context is essential to do the right things to discover the status of the product and any threats to its value effectively, so that ultimately our clients can make informed decisions about it. Context analysis and modelling are some of the essential skills for testers!

There are probably more parallels to testing. Please let me know if you see any more.

 

”Many people have accused me of being fake or have theories on how there’s cheating involved. I’ve always found it fascinating how human it is, to want to disbelieve anything that goes against our world view – even when it’s about something as relatively neutral as archery.” (Lars Andersen)

Heuristics for recognizing professional testers

Via Twitter Helena Jeret-Mäe asked this question: “What are your criteria for professionalism for testers in CDT community?”. Later via Email she updated her question to: “So the updated version of my question is what are the heuristics for recognizing professional testers in your opinion? I changed “criteria” to heuristics… it’s less categorical. And I’ll leave the term “professionalism” up to you as well – I don’t know exactly what you meant by it.”

In my talk “How to become a great tester” at ContextCopenhagen last January I talked about testers and their skills. I said that most testers don’t know what they’re doing and can’t explain effectively what value they add. I have seen many testers who use the same approach over and over again. If I ask them to name test techniques, they can only name a few. If I ask them to explain techniques to me or show how they work, I get no answers. I find that shocking and I cannot understand why testers who call themselves professionals know so little about their craft and do not study their craft.

That is why I make a distinction between professional testers (of which I think there are only few) and testers by profession. Of course I know and understand that there will always be people who have a 9 to 5 mentality, do not read books or blogs and only want to do courses when the boss pays for them. I accept that reality, but that doesn’t mean I want to work with them!

Let me now answer the questions asked, have done enough ranting for now…

Professionalism is what it means to be a professional and what is expected of them. Professional testing is complex and diverse and has several dimensions: knowledge, skills, experience, attitude, ethics and values. I wrote a blog post “What makes a good tester?” in 2011 on this topic and I have a broader view on this topic now although everything I wrote then still counts. In my earlier blog posts I didn’t mention values and ethics and I now think they are extremely important. James Bach writes about them in his blog post “Thoughts Toward The Ethics of Testing”. A great practical example of ethics and values is Rapid Testing. Look at the “the premises of Rapid Testing” and “the themes of Rapid Testing” both can be found in the slides of Rapid Software Testing.

It is hard to recognize professional testers. Every tester is unique and brings different characteristics to the table. Every project is different too and to be successful in finding the right professional tester for your project different characteristics may be important. There are many characteristics to be considered so to be able to recognize professional testers heuristics can be used. Heuristics are fallible methods for solving a problem or making a decision, shortcuts to reduce complex problem or rules of thumb. They are used to determine good enough feasible solutions for difficult problems within reasonable time. On Wikipedia I found this definition: “Heuristics are experience-based techniques for problem solving, learning, and discovery that give a solution that is not guaranteed to be optimal. Where the exhaustive search is impractical, heuristic methods are used to speed up the process of finding a satisfactory solution via mental shortcuts to ease the cognitive load of making a decision.

My heuristics for recognizing professional testers:

1) They have a paradigm of what testing is and they can explain their approach in any given situation.
Professional testers can explain what testing is, what value they add and how they would test in a specific situation.

2) They really love what they do and are passionate about their craft.
Testing is difficult and to be successful, testers needs to be persistent in learning and in their work. Passion helps them to be become real professionals. Watch this video where Steve Jobs talks about passion

3) They consider context first and continuously.
To be effective testers need to choose testing objectives, techniques and by looking first to the details of the specific situation. They recognize that there are no best practices but only good practices in a given context.

4) They consider testing as a human activity to solve a complex and difficult problem that requires a lot of skill.
Testers recognize testing is not a technical profession. Testing has many aspects of social science since software is built for humans by humans.

5) They know that software development and testing is a team sport.
Collaboration is the key in becoming more effective and efficient in testing. Software development is a team sport: people, working together, are the most important part of any project’s context. A great tester knows how to work with developers and other stakeholders in any situation.

6) They know that things can be different.
Professional testers use heuristics, practice critical thinking and are empirical. They know that time to test is limited, systems are becoming increasingly complex and thinking of everything is hard. Therefore heuristics are a very helpful “tool” for testers. They also know that biases and logical fallacies can fool them. They practice critical thinking to deal confidently and thoughtfully with difficult and complex situations. Testers have to accept and deal with ambiguity, situational specific results and partial answers.

7) They ask questions before doing anything.
Testing depends on many things and what is the context of the stuff I am looking at? What is the information we need to find? What is the testing mission? Giving a tester an exercise in an interview can easily test this. If he or she starts working on it or gives you an answer without asking questions, this tells you something.

8) They use diversified approaches.
There is no approach or technique that will find all kinds of bugs or fulfil all test goals. Different bugs are found using different techniques. In order to be able to do this, testers need to know many techniques and approaches. This demands training, practice and some more practice.

9) They know that estimation is more like negotiation.
Have a look at some blog post by Michael Bolton:

10) They use test cases and test documentation wisely.
The context determines what test documentation you should make and what kind of documentation is useful. Quite recently a great (and long) article by James Bach and Aaron Hodder was published in Testing Trapeze “Test cases are not testing: Toward a culture of test performance”. Also Fiona Charles has written some interesting stuff about test documentation in her article the Breaking the Tyranny of Form.

11) They continuously study their craft seriously, practice a lot and practice “deliberate practice”.
Deliberate practice is a structured activity with the goal of improving performance. According to K. Anders Ericsson, there are four essential components of deliberate practice. It must be intentional, aimed at improving performance, designed for your current skill level, combined with immediate feedback and repetitious.

12) They refuse to do bad work, never fake and have the courage to tell their clients and the people they work about their ethics and values.
Testing is often underestimated and many believe “everybody can test”. I have experienced project managers who tell me how to do my work. Professional testers know how to push back and are able to explain why they do what they do.

13) They are curious and like to learn new things.
Testers find pleasure in finding things out. A good read is the curious behaviour by Guy Mason: “a curious mind is one that could be described as an active, engaged and inquisitive mind. Such a mind frequently seeks out new information, enjoys discovering what there is to discover and enjoys the process that comes along with this goal.“

14) They could have any of the important interpersonal skills mentioned in the list below. As stated earlier it depends on the context which skills are most important.

  • Writing skills (like reporting, note taking and concise messages)
  • Communication skills (many different like listening, story telling, presentation, saying no, verbal reporting, arguing and negotiating)
  • Social and emotional skills (like empathy, inspiring, networking, conflict management and consulting)
  • Problem solving skills
  • Decision making skills
  • Coaching and teaching skills
  • Being proactive and assertive

15) They have excellent testing skills:

  • Thinking skills (critical, lateral, creative, systems thinking)
  • Analytical skills
  • Modelling
  • Risk analysis
  • Planning and estimation
  • Applying many test techniques
  • Exploring
  • Designing experiments
  • Observation

Have a look at the Exploratory Testing Dynamics in the RST appendices where several lists of skills are listed.

16) They have sufficient technical skills.
There are many technical skills a tester needs like being able to use tooling, coding skills or willingness to learn what they need to know about the technical structure of the application they are testing. Test automation skills like scripting and SQL skills to work with databases, being able to configure and install software, knowledge and skills to work with the platform the system under test is on (Windows, Linux, Mobile, etc.).

This list is probably not complete. It is very well possible that I have made some mistakes. So please let me know if you have any contributions or improvements. Also: being a professional doesn’t mean you are an expert on all mentioned skills. Have a look at the Dreyfus model to learn more about expert level. A real professional knows what he or she can do and when to ask for help. They do not fear to learn and they are not afraid to make mistakes. I guess that would be the 17th heuristic in the list.

BTW: I googled “code of ethics software testing” and found the ISTQB Code of Ethics for test professionals. I wonder if people who pass the exam know about these AND more important practice them… What about you?


More information

Some more posts that discuss great testers:

And last but not least: have a look at the Testers syllabus by James Bach. An awesome document which lists many important skills and areas of knowledge. It inspired me read about and study different areas.

DEWT4 sketchnotes

This are my sketchnotes from DEWT4. The central theme of the fourth DEWT peer conference was “Teaching Software Testing”.

Click the images to view them full size.


My Tools

So you think you can test ….

Recently I saw this tweet: “a lot of testers don’t consider alternatives because they don’t know them”. It was a reaction in a discussion about a Dutch article with the title „The days of the ‘Dutch school of testing’ are over”. Jan Jaap claims that Dutch testers suffer from “Law of the handicap of a head start“. Really? I don’t think it is the handicap of a head start. Did we (Dutch testers) ever had a head start? I think it is something that is called “The Dunning-Kruger effect“.

Dunning KrugerThe Dunning-Kruger effect, named after David Dunning and Justin Kruger of Cornell University, occurs when incompetent people not only perform a task poorly or incompetently, but lack the competence to realize their own incompetence at a task and thus consider themselves much more competent than everyone else. Put more crudely, they’re too stupid to realize they’re stupid. The inverse also applies: competent people tend to underestimate their ability compared to others. (Source: Rational Wiki)

In the words of Dunning and Kruger: this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it.

I think many testers don’t realize that they actually know very little about testing at all. And that is why they don’t do anything to get better. They are also not encouraged much by their colleagues. Testing is often underestimated. I guess everyone has examples of managers who do not value testing as much as we would want them to. Testing is often devalued as “pushing buttons” and “everybody can test” is believed by many. I have seen companies who use testing as education, a first step in an IT career…

But there is more… I have worked as a selling and hiring manager and I experienced that there is a lack of competence in spotting talented testers. If you have the right certificates, people believe that you are a good tester. Because they simply do not know how to spot an excellent tester. In my recent job search nobody asked me to SHOW them my testing skills. We only talked about it.

 

Popular blogs for testers

blogsTo help the Dutch community evolve to learn, think and do more skilled testing I want to advocate interesting stuff to read to the community. But what should a tester read? Earlier I published a list of popular books for testers. In my search for good books to read, some people replied that they only read blogs and other on-line content. On this blog I recommend great articles and on-line stuff in my great resources list. For testers blogs you can check my colleagues list.

But what do other testers recommend? My curiosity kicked in again and I sent out another email to my tester friends around the world to ask for their favourite blogs.

This is the “top 19” of most mentioned blogs. In the list are all blogs that got more that 2 votes. I am thrilled to be in this list at all. But it must have been the “availability heuristic” that got me on the 3rd place! Thanks guys 😀

Rank # votes Name URL
1 26 Michael Bolton http://www.developsense.com/blog
2 23 James Bach http://www.satisfice.com/blog
3 12 Huib Schoots http://www.huibschoots.nl/wordpress/
4 11 Cem Kaner http://kaner.com/
5 10 Michael Larsen http://www.mkltesthead.com/
6 9 Markus Gartner http://www.shino.de/blog/
  9 Elisabeth Hendrickson http://testobsessed.com/
8 8 Alan Page http://angryweasel.com/blog/
9 7 Andy Glover http://cartoontester.blogspot.com/
  7 The Test Eye http://thetesteye.com/blog/
  7 Zeger van Hese http://testsidestory.com/
12 6 Pradeep Soundararajan http://testertested.blogspot.com/
  6 Matt Heusser http://xndev.com/creative-chaos/
14 5 Iain McCowatt http://exploringuncertainty.com/blog/
15 4 Jurgen Appelo http://www.noop.nl/
  4 Keith Klain http://qualityremarks.com/
  4 Pete Walen http://rhythmoftesting.blogspot.com
  4 Rob Lambert http://thesocialtester.co.uk/
19 3 Adam Goucher http://adam.goucher.ca
  3 Adam Knight http://www.a-sisyphean-task.com
  3 Anne-Marie Charrett http://mavericktester.com
  3 Bob Marshall http://flowchainsensei.wordpress.com
  3 Eric Jacobson http://www.testthisblog.com
  3 Gojko Adzic http://gojko.net
  3 James Christie http://clarotesting.wordpress.com/
  3 James Lyndsay http://workroomprds.blogspot.com/
  3 Ministry of Testing http://www.ministryoftesting.com/testing-feeds/
  3 Parimala Hariprasad http://curioustester.blogspot.in/
  3 Seth Godin http://sethgodin.typepad.com/

180 different blogs where mentioned by 41 participants. The full list of blogs can be found here. An overview of all participants and their personal lists can be found here.

If this isn’t enough, you can check these listings of tester blogs:

Older posts