Category: DEWT

The art of reflection

“Once is coincidence
Twice is striking
Three times is pattern”

October 19-21 2018 DEWT held their 8th annual peer conference with “Developing expertise in software testing” as theme. I had the honour to open the conference with my experience report called “Mentoring and coaching to develop skills”. In the open season after my experience report and during the rest of the peer conference we talked about reflection on several occasions. I think one of the most important skills in learning is reflection.

My vision on learning
Learning is the process of acquiring new or adapting existing knowledge, behaviours, skills, values ​​or preferences. Learning is much more than knowing: putting the learned into practice and gaining experience with it is important to truly internalise real knowledge and to gain skill. Learning must be linked to experiences from daily practice. By reflecting on knowledge and skills, so-called learning loops are created.

Learning is an ongoing process: the world is changing fast and to be excellent in your role requires many skills. So it is important to keep up! To learn effectively, learning should come from yourself, with intrinsic motivation and personal responsibility.

Learning requires a positive and open learning environment in which you can safely come out of your comfort zone to try new things. The safe environment gives confidence to make mistakes and to experiment with new knowledge and skills. It also demands a certain degree of challenge. How big the challenge is, is different for everyone. It is not that you always have to come out of your comfort zone radically. Right outside of your comfort zone, in your stretch zone,  you learn best.

Effective learning requires focused learning with clear learning objectives and preferably evaluation criteria. It should be clear what you want to learn and how you can measure that. Where do you want to grow? And how do you know that you have made a step? By setting clear learning objectives, you focus on learning.

Levels of learning
There are different levels of learning: (ref: Joris van de Griendt)

  1. In single-loop learning (improvement, behavioural improvement), it is about the visible and concrete behavioural level (what): you do something and that has a certain effect. This effect may be desirable or not, and based on that assessment, you can adjust your behaviour or not.
  2. Double loop (reframing, behavioural renewal) If you want to change your behaviour permanently, there is double-loop learning: researching which patterns the behaviour involves (how). Which patterns and mechanisms are behind the behaviour? Which helpful or obstructing thoughts are involved? Insight into this can provide a more well-founded choice to change behaviour.
  3. Triple-loop learning (transformation, behavioural development) goes even deeper. You involve your values, your purpose, the why question: why do I really want to change this behaviour? What important values ​​in me support this and what stops me?

What is reflection?
Reflection is a process of exploring and examining ourselves, our perspectives, attributes, experiences and actions / interactions. It helps us gain insight and see how to move forward. (ref: University of Edinburgh).

When we reflect, we deeply consider something that we might not otherwise have given much thought to. This helps us to learn. Reflection is concerned with consciously looking at and thinking about our experiences, actions, feelings, and responses, and then interpreting or analysing them in order to learn from them. (ref: The Open University).

Reflection is looking back at your behaviour in a certain situation. You reflect on that situation by asking yourself meaningful questions to make you think about the situation. The difference between just thinking and reflection is the intention to learn. There is a difference between evaluation and reflection. Evaluation means making a judgement about something you did. For example: did you reach your goal? Did I do the right thing? While reflecting means creating a safe space to investigate behaviour without making a judgement with the intention to grow.

Like learning, there are different levels of reflecting:

  1. Single-loop reflection focuses on behaviour and actions and is very close to evaluation.
  2. Double-loop reflection means trying to get hold of underlying convictions that interfere with the adjustment of interaction and behaviour.
  3. Triple-loop reflection is about motives and matters that touch on their own identity. There is often a relationship with issues at a higher level, that of the organisation or even of an entire system.

Iceberg

(image credit: Dutch Vision Institute)

Above the waterline behaviour is perceptible and statements are audible. But opinions, beliefs, feelings and emotions are not visible; they are below the waterline. These invisible elements, however, are often the motives for visible behaviour. An important part of the reflection will therefore consist of researching these deeper layers of the Iceberg.

By not addressing all layers within one’s competence management, you allow the coachee to act for incongruously (say A and do B, or vent a belief that contradicts his own motivation); just like external fragmentation (non-integration in the context), internal fragmentation (in the context of do-thinking motives) also leads to a real risk of energy loss.

Korthagen

(ref: How do I use the Korthagen reflection circle diagram?)

Korthagen’s reflection cycle is a tool or a strategy to be followed for learners to gain insight into their educational performance and to improve this. By applying this cycle step-by-step, one learns to systematically reflect a skill to be learned.

Phase 1: Describe the experience/situation you wish to reflect upon. What was the actual situation? You can do this by using the STARR(S) method: Situation-Task-Action-Result-Reflection-Strengthen (see appendix).

  • What did I have to do in this situation?
  • What action did I actually take?
  • What was the outcome of this action?

Phase 2: Looking back: What exactly happened?

  • What did I see?
  • What did I do?
  • What did I think?
  • What did I feel?

Phase 3: Awareness of essential aspects

  • What does that mean to me now?
  • What is the problem (or the positive discovery)?
  • What has all that caused? What does it involve?

Phase 4: Alternative methods

  • What alternative methods do I see (solutions or ways of making use of what I have discovered)?
  • What are their advantages and disadvantages?
  • What will I remember for next time?

Phase 5: Trial/action

  • What do I want to achieve?
  • What should I watch out for?
  • What do I want to try out?

Danger of thinking too much

Reflecting is an active activity that demands skills. It often happens that professionals think they reflect, while they are actually worrying.

This points out the important differences:

Worry

  • Involved in itself, looking from our own perspective, alone
  • Focused on mistakes
  • Focused on judging and condemning
  • Global approach
  • Mono causal approach

Reflection

  • Involved in the problem, also looking at a perspective outside of oneself, in contact with others
  • Focused on solutions
  • Focused on understanding
  • Analytical approach
  • Multi causal approach

Tips for reflecting

You can reflect on every situation and every problem that concerns you. You can learn a lot from that, but the pitfall is that you will be overwhelmed by the information and will keep looking back endlessly. Another danger is that you may feel that you are actually doing a good job – the work is going well, there is no criticism from colleagues or your supervisor – and so you see no reason to reflect. Yet it can also be very instructive to reflect on yourself and your way of acting.

The following tips can help you reflect:

  • Choose a concrete situation and look back on that specific moment and your course of action
  • Reflect regularly and ‘schedule’ at least once a week a reflection moment, preferably at a fixed time
  • Ask yourself open questions
  • Explain judgments about yourself; first see what really happened before judging yourself
  • Reflect in a methodical way, for example by going through a list of questions or use a reflection model
  • Reflect not only on problem situations but also on success experiences
  • Use feedback from others to reflect from that point of view
  • Read more about reflection to get inspired. This is a nice blogpost about reflection: How Self-Reflection Gives You a Happier and More Successful Life. For more inspiration read this: Tools to help you with self-reflection

Together you learn even more

It is already very instructive to consider your own functioning in this way, but it can be even more profitable if you do this together with others. Do not try to judge here either. Not about others, nor about yourself: you feel free to tell everything. For example, start doing intervision with peers or colleagues. More about intervision here: Intervision: what is it about?

Start a journal

Writing in a journal regularly (preferably daily on a specific time) helps to analyse your professional and personal growth. Journaling can give you a different perspective on things. Writing in your journal is a very useful tool to help you understand yourself and the world around you. Write down activities, thoughts, ideas, reasons, actions, techniques and reflections on specific topics or skills you want to improve. By writing in a journal you get an overview of your thoughts in which you can identify patterns. Journaling helps you to get thoughts and ideas out of your head but more important it enables making sense of things that happened. After doing something related to your learning goal, take notes on your observations, summarise facts and experiences. Also write down how it makes you feel.

More about reflective journaling in this article: Reflective Journals and Learning Logs.

 

Here are two helpful checklists to help you reflect:

Why I am context-driven!

Wanna know why I am context-driven? Published on the DEWT blog: Why I am context-driven!

“Because testing (and any engineering activity) is a solution to a very difficult problem, it must be tailored to the context of the project, and therefore testing is a human activity that requires a great deal of skill to do well. That’s why we must study it seriously. We must practice our craft. Context-driven testers strive to become the Jedi knights of testing”. James Bach

DEWT4 sketchnotes

This are my sketchnotes from DEWT4. The central theme of the fourth DEWT peer conference was “Teaching Software Testing”.

Click the images to view them full size.


My Tools

Context-driven (presentation) heuristics

James BachOctober 23 DEWT and James Bach met in Dordrecht to talk about testing. The subject was “What is context-driven testing?” and more specific “How do you recognize a context-driven presentation?”.

They came together to prepare for the TestNet Autumn Event which was fully dedicated to context-driven testing and had the theme: “Exploring context-driven testing – a new hype or here to stay?”.

DEWT decided earlier this year to write an article about the event to reflect on context-driven testing in the Netherlands. What does the TestNet community have to say about context-driven testing? What can we learn from the event? What can we do to help the community learn? How context-driven is the Dutch Testing community? While discussing this the DEWTs found it hard to find heuristics to “measure” (maybe “recognize” is a better word here) the presentations at the TestNet event.

DEWTs

What is context-driven?

Often context-driven testing is “only” seen as an approach, but it can be more. Actually there are 3 different things that are called context-driven. Testers can be part of one, without necessarily being part of the other.

  1. Paradigm (world view)
  2. Community
  3. Approach

For example: you can use an context-driven approach without being part of the context-driven community. To be in a paradigm you need to have a world view of testing.

Testing People evaluating a product by learning about it through experimentation
driven by is a matter organized and motivated by a systematic consideration of
context all the factors that significantly influence the problems and solutions that lie within the scope of their mission
  • Attitude: context-driven testing allows to change the approach. An example is “Huib’s Rapid Software Testing”. This Huib’s way of doing testing, inspired by Rapid Software Testing, but he changed his approach to his context. Changing anything he sees fit. Often factory school testers do not want to change their approach and apply their approach the same in every situation.
  • Principles: context-driven testing is NOT (only) the seven basic principles, but there is much more. Look what lies behind them, learn about the implicit principles. See slide 9 of this presentation by James Bach.
  • Mentality: context-driven testing is about skills, humanity, science, critical thinking, problem solving, non-linearity, investigation, learning, etc.
  • Metrics: you can use only those metrics if you understand them and if they solve a certain problem.
  • Critical thinking: do you know how? How do you know that you know? How do you get better?

Science is the Belief in the Ignorance of Experts” — Richard Feynman

Presentations

Good stuff to look for:

  • Heuristics over commandments
  • Learning curves
  • Compare alternative methods, trying other methods
  • To be truly context-driven you have studies and tried the practices you say you do not like
  • Discussion of causes and effects assuming open systems. Reject the believe that a project a well defined game that is predictable. Also reject the assumption that a process is unhackable
  • Acknowledgement there are people with different opinions, not speaking from authority
  • Social science
  • Humanism
  • Practitioner responsibility
  • Craftsmanship
  • Refuse to do bad work
  • Problem solving
  • Skill based work

Be aware of people:

  • talking about ‘success’ and ‘failure’ without explaining why and describing the context
  • talking about ‘structure’ and ‘chaos’ (often meaning: I am out of control)
  • showing no evidence (or using sentences like: “research shows that…”)
  • relying on folklore
  • using numbers without context
  • who are ignorant of social science
  • uncritically apply the manufacture metaphor (IT is like a factory)
  • apply premature automation
  • assuming other people read and follow what is written
  • contempt humanism
  • trying stuff once and over generalize that to the whole company/world
  • who fear variation
  • over simplify (approach the world as if it was linear)
  • misuse of statical analysis
  • who use averages without variance
  • who do narrow research
  • say or do stuff because their clients want it
  • demonize tacit knowledge and idealize explicit knowledge

This is the summary I have made of what we discussed. You could consider this as a heuristic for context-driven presentations. I used this heuristic in my presentation “What is context-driven testing?” at the TestNet event.

Heuristics Success / Failure
Learning Chaos / Structure
Compare alternative methods Lack of evidence / Narrow research
Cause and effect Folklore
Open systems Numbers without context
Social science Ignorance of social science
Humanist view Contempt of humanism
Craftsmanship IT & testing is like manufacturing
Explain context No context mentioned
Worldview of testing Generalizing after one try
Different opinions Use of averages without variance
Allows to change approach Linearity

DEWT3 Sketchnotes

Here are the sketchnotes I made during DEWT3. Click the images to view them full size.

Software Quality Characteristics in Dutch

At EuroStar 2012 in Amsterdam, Henrik Emilsson did a talk about the Software Quality Characteristics poster made by The Test Eye. After his talk he asked if someone was interested in translating this poster into other languages. Me and my DEWT colleagues happily picked up this gaunlet and we proudly present the Dutch translation: Software Kwaliteit Kenmerken. I use this checklist often when preparing my testing. Now available in the Dutch language!

 

Why use checklists?

The modern world has given us stupendous know-how. Yet avoidable failures continue to plague us in health care, government, the law, the financial industry—in almost every realm of organized activity. And the reason is simple: the volume and complexity of knowledge today has exceeded our ability as individuals to properly deliver it to people—consistently, correctly, safely. We train longer, specialize more, use ever-advancing technologies, and still we fail. Atul Gawande makes a compelling argument that we can do better, using the simplest of methods: the checklist (source: Amazon.com)

DEWT2 was awesome!!

Last Friday (October 5th) a bunch of software testing nerds and one agile girl gathered in Hotel Bergse Bossen in Driebergen talk about software testing with the central theme “experience reports: implementing context-driven testing”. Ruud published almost all the photos I took on the DEWT website, so I have to write a report in text here. Thanks dude 😉

After some drinks and having dinner we gathered in the conference room called the chalet. I think they call it the chalet because of the humid smell inside because it certainly didn’t look like a chalet.

But never mind, the room was good enough to do a peer conference. Lighting talks were on the program for Friday and Jean-Paul started with a talk in which he asked the question: “Is the context-driven community elitist?”. Jean-Paul sees a lot of tweets and blogs from people in the context-driven community who seem to look down on the rest of the testing community sending the message “look how great we are!”. Is this effective behaviour given the fact that the context-driven community wants to change the way people test, he asked himself.

Joris had a short and clear answer to the first question: “yes!” (the context-driven community is elitist). We had a long discussion that went everywhere but never close to an answer to question about effectiveness and how it can be done better. Never the less it was a valuable and fun discussion. I will blog about this later this year.

We had a great evening/night with home brew beer “Witte Veer”, Belgium beer brought by Zeger and a bottle of Laphroaig quarter cask brought by Joris. Sitting at the fire place a lot of us stayed up until late

Saturday morning the last DEWTs and guests arrived. With 23 people in the room we had a great group. Our guest were: Markus Gärtner, Ilari Henrik Aegerter, Tony Bruce, Gerard Drijfhout, Pascal Dufour, Rob van Steenbergen, Derk-Jan de Grood, Joep Schuurkes, Bryan Bakker, Lilian Nijboer and Philip-Jan Bosch. The DEWTS: Adrian Canlon, Ruud Cox, Philip Hoeben, Zeger van Hese, Jeanne Hofmans, Joris Meerts, Ray Oei, Jeroen Rosink, Peter Schrijver, Jean-Paul Varwijk and myself.

Peter was our facilitator and for the first time we used k-cards at DEWT to facilitate the discussion. I really like this method, but Lillian didn’t. She had an interesting discussion on twitter with some DEWTs.

Jean-Paul: I like the LAWST format we are using for #DEWT2
Lilian: @Arborosa i don’t. impersonal and inhibits useful discussion
Markus: @llillian @Arborosa What’s the improvement you’re suggesting? 🙂
Lilian: @mgaertne @Arborosa but i am invited and feel i at least have to try it this way
Jean-Paul: @llillian @mgaertne Invite us to an agile event to learn alternatives
Markus: @Arborosa @llillian Funny thing, I would like to introduce more focused-facilitated sessions quite often in agile discussions using k-cards.
Lilian: @mgaertne @Arborosa different ppl prefer different things 🙂

Ilari kicked of with a presentation about the introduction of context-driven testing at eBay. He had two very interesting ideas: the first is “Monday Morning Suggestion” a short email to his team with tips, tricks or interesting links. The second was that he supports his team in getting better and learning by paying for their books and conference visits. Great stuff!

Second was Markus Gärtner with a story about him coaching a colleague to become a better tester and more context-driven. He talked about his lessons learned coaching where teaching gradually turned into coaching. He also gave some insight in the transpection sessions he did using the socratic method.

The third presentation was about changing people to become more context-driven by Ray Oei. An interesting discussion developed about how to get people to change. We focus too much on the people who do not want to change instead of working with the people who do want to change. Pascal asked an interesting question which I put on twitter: “If all testers become context-driven overnight, are we happy?” James Bach replied: “Yes, we are happy if all testers become CDT” and “We are happy if all testers take seriously the world around them, and how it works, and dump authoritarianism”. To me it doesn’t really matters, I do hope that more people become CDT, but I prefer good software testing no matter how them call it.

After the discussion Joris said he was attending the wrong peer conference since he expected a lot of discussion about testing instead of what he calls “people management”. An interesting question, but isn’t testing also a lot about people? Just an hour late we went for lunch and a walk in a wet forest. After the walk the group photo was taken.

The forth talk was by Ruud Cox who talked about testing in a medical environment. He described the way he has been working. He and his colleague tester use exploratory testing to learn and explore. Scripted testing was used to do the checking. Ruud explained that exploratory testing fits very well in an R&D environment. After his talk we had an interesting discussion about auditors and their role in testing.

Around 16:00 it was my turn to do my talk about implementing context-driven testing within Rabobank. I did a short version of the talk I will do at EuroStar in November telling about the challenges we had and what we did to change our context. What did we do? What worked and what didn’t? Some interesting questions were asked. And we had a nice discussion about the personal side of becoming more context-driven where Joep explained how he became context-driven. The first stage was being interested and he went looking for a book. The second stage took him over a year where he struggled with the stuff he learned at RST and trying to adapt his way of working. The third and last stage is where he doesn’t need anything anymore because he will always find a way to make things work.

After my talk and the discussion the day ended. We did a quick round to discuss our experiences and thanked the people for attending, organizing and facilitating. Our agile girl still doesn’t know what to think about the facilitation with the k-cards, she obviously had to get used to discussing with the cards. I am curious how she thinks about them when she had the chance to think about it some more.

All together I had a great time. Pity we didn’t play the Kanban game on Friday. But I hope Derk-Jan will give me another chance at the game night at TestNet in November. Thanks all for the awesome weekend!!

DEWT3 is already being planned for April 20 and 21 next year and James Bach will attend. Let me know if you are interested in joining us in April 2013. The topic hasn’t been decided yet, but if you have great ideas please share them.

Standing left to right: Ray Oei, Jean-Paul Varwijk, Adrian Canlon, Markus Gärtner, Ruud Cox, Joris Meerts, Pascal Dufour, Philip Hoeben, Gerard Drijfhout, Bryan Bakker, Derk Jan de Grood, Joep Schuurkes, Lilian Nijboer, Philip-Jan Bosch, Jeroen Rosink, Jeanne Hofmans
Kneeling left to right: Tony Bruce, Zeger van Hese, Ilari Henrik Aegerter, Huib Schoots, Peter Simon Schrijver

First DEWT peer conference

Pre-conference drinks

After a long and instructive week of RST, I picked up Michael at his hotel on Friday evening and we drove to Driebergen for the first DEWT peer conference (#DEWT1). We checked in and went to our rooms for a small powernap waiting for the others to arrive. Since the conference would only start the next day, we had the inevidiable drinks and fun. You can imagine that putting 7 testgeeks in the same room will end up in enjoyable stories about our craft. The bar closed at 23:00, but we found the night porter happy to bring us more drinks. Somewhere around 2 a.m. we went to bed after a very entertaining evening/night.

Opening by morning chair Ruud Cox

On Saturday Michel and Jeanne joined us after breakfast and our DEWT conference could finally start. Ruud being the chairman in the morning kicked off thanking everybody for being there.

Artful Testing
Zeger did his “Artful Testing” talk. A very well done talk using a amazingly beautiful prezi. Although he had 15 minutes according to the program he managed to keep us facinated for triple the amount of time. He must have paid Ruud to get his extra time 🙂 Or was it because his talk kept us all from looking at the clock?

Zeger Artfully in action

Zeger’s talk was about (without giving away too much) the connection between testing and art. The ingredients were:

  • how testers can benefit from arts
  • some good comparisons between art and software, testers and art critics
  • some nice “artful testing” heuristics, too bad they are still without a nice mnemonic, but Zeger might change that over time.
  • a good testing challenge
  • nice photo’s and other pieces of art
  • Ambiguities, mystification and anagrams (nope no Da Vinci Code here)

I don’t know what Zeger loves more: art or testing, but he sure did a good job combining them. This talk is on the program of the Belgium Testing Days and I suggest you go and see this talk! I hope Zeger will also send in his talk for the TestNet fall event as well.

 

Michael Bolton on Transpection

Transpection
Michael Bolton was up next on the topic transpection. We asked Michael to help us explore the intruiging “tool” of transpection. After a short introduction, referring to several post by James Bach and himself we started an exercise. We did the exercise in pair, where we were to collect information on the subject. Michel and I searched some articles and then tried a little transpection of our own. In the round up we collected all information in a nice mindmap.

BTW: See Zeger’s blog for more information on how we studied transpection.

Lighting Talks

After lunch Jeanne took over as conference chair and lighting talks were on the program.

Jeroen Rosink

Jeroen Rosink: Testing Pyramid, about testers in the classic career path (test analyst – test coordinator – test manager) getting more senior in their job and there are only a few places at the top of the pyramid.

Me: The power of knowing nothing, about people without knowlegde asking lots of good questions, not being biased. And the usefulness of telling people why instead of how. Inspired by my first days in a new job and this video of Simon Sinek.

Zeger van Hesse: Bader Meinhoff-phenomenon, about the advantage of knowing a lot. Since the phenomenon says that when you have seen it once, you start recognizing it more often. A powerfull thing for testers.

 

Introducing RST in Dutch projects
Ray talked about his experience on how he tried to introduce his take aways form the RST training in his projects. An interesting discussion followed Ray’s short talk and we made a mindmap on how to make the case for Rapid Testing. At the end a discussion from the RST class in the week before the conference about giving release advices popped up again.

Credibility

The last talk was done by Ruud on Credibility. Ruud referred to a presentation on this subject by Randy Rice which inspired him. Credibility is build on trust and is one the most valuable assests a tester has. If you lose your credibility you will have a though time doing your job. Ruud showed a cool heuristic he uses to keep reminding himself on the key factors of credibility. He also made a nice design he can put on his monitor so he is reminded whole day.

 

STYLE by Ruud Cox

STYLE

  • Safety language
  • Two ears one mouth
  • Yes but
  • Lighten up a little
  • Empathy

After this topic the conference was officially over and we had some drinks (with nice testing games), dinner together (with some yes/no question mysteries and lifting my parrot jokes to perfection) and we left for home. This conference was excellent!! Why? It was great fun, I learned a lot, got to explore some really interesting topics with cool colleagues and went home with a lot of new ideas. Thanks guys!

Ray Oei, our technician, at work

This conference was attended by: Jeroen Rosink, Ray Oei, Jeanne Hofmans, Michel Kraaij, Huib Schoots, Jean-Paul Varwijk, Ruud Cox, Zeger Van Hese, Michael Bolton. Peter “Simon” Schrijver and Anna Danchenko could not attend.
Ray filmed the whole day, I’m curious about the movies he made….