Is the Context Driven School of Testing - Dead?

Earlier this week, Scott Barber pointed out an update to the Context-Driven-Testing page which has served as a starting point for people who want to learn about Context Driven Testing. In this post Kem Caner, one of the 'big four' of the Context Driven School gives an impression that the idea of Context Driven Testing being a 'School' has perhaps reached a point where it no longer exists.  He goes on to cite differences in vision among the founders of the movement, and I'll let you read his post above to draw your own conclusions.  However, when I was asked this question, I felt that I didn't agree.   I've seen places where friends or close colleagues go their separate ways because of a polarizing my way or the high way sort of mentality.  I'm not sure that's quite what Kem is referring too.

I debated whether to post my response or thoughts to this here at all as no doubt everyone involved or outside this movement will have their own opinions and beliefs.   I decided that for posterity's sake that I would indeed.  I have discussed this a bit with some fellow testers in the MiagiDo group, and Michael Larsen another MiagiDo member recently made a post giving his view on things (In Support of Context Driven Testing).  It's worth a read, and his thoughts were a bit similar but also slightly different from my own initial response to this question which I gave a bit earlier.  What follows was my initial response to this question of whether Context Driven School of Testing is Dead as it was originally shared with others in MiagiDo.  The only changes are the addition of Italics, and  a few grammatical improvements.

I'll have to quote Monty Python on this one. 

"I'm not dead yet!" 

I read Scott Barber's post, and I read the information there.  I really chaffe a bit and comparing this to the polarization of a few figure heads within the American republican party.  Kaner mentions Gingrich, who no doubt is a polarizing figure. If you're an American, you likely either like him, or you hate him, there probably isn't  a middle ground for most Americans.  However, I think perhaps the 4 Founders have gotten a bit too caught up in their own mystique. 

I see the same thing within our American political discourse.  People at the so called top seem so polarizing, and the media tries to frame discussions a certain way, and yet most of us at the bottom, outside that top so called 'chosen' echelon are not polarizing, we are not so closed minded or resistant to having honest discussions about things.   

I think this is true also for the Context-Driven Community, whether we call it a School or not, to me is irrelevant.   The point is we are making a fundamental distinction in where our thinking begins.  I see this same problem in the US Education system.  A lot of people expect school, especially college to prepare them for everything they need to succeed in a career.  The reality is quite opposite, it's a base, a starting point.  If you get that strong base, I believe a strong education helps you succeed no matter what you apply your life too.  I feel the same here with Context-Driven School of testing. 

Listen, so the four founders of the movement disagree.  There are four gospels in the New Testament of the Christian Bible, (I use this as an example), and all four a bit different, varied, and share some similarities too.   Those guys were alive at the time right?  Yet they were not the same as each other, they had differences and disagreements no doubt. So let's be clear the entire point of our School (in my estimation) is to not have Drone or Fake testing right?  I believe the point, as I understand it, is to encourage people to use and develop their critical thinking skills, and to not be satisfied with testing that feels lacking in covering the real risks of a project. 

Let's look at this another way.  James Bach gave a talk (I believe this was at Oredev, see James videos posted at his site at www.satisfice.org if interested) about Renaissance thinking a few years ago.  He describes what was fundamentally different about it, and how it wasn't just what was happening In Italy, or Scotland, or wherever else, that there were little movements pretty much elsewhere.  Think about the artists themselves, they all approached art a bit differently, yet they don't look like carbon copies even if they've learned some of the same skills and techniques.  This is no different than how we each approach testing in the Context Driven School of testing.  We are still different then what has come before.  We are still different than the more agile-centered testers. 

We cannot deny who we are.  We've been enlightened, we've grown as a community, a school.  It would be impossible for us to try to put the genie back in the bottle now.  What's done is done, the shot has been fired and heard around the world now.  The question is, in any revolution, what will each of us do because of it.

That's my feelings on it anyways.
Hope that wasn't too preachy,
-Tim Western

So in conclusion, while there may clearly be differences between the founders of the Context Driven moment, its principals and the School of thinking that it began, I must stand up and say, hey I'm still Context Driven, I still belong to that School of testing.   Now I do think there is a risk here.  There's a risk in perhaps holding too close to the fundamentals of the school.  We need to be aware of other ways of doing things, even if we may find that our way is better.  As an engineer, a tester, or a scientist I would be looking for ways to prove my own methods wrong, just as so many other scientists have done for centuries.  Yet for now, as a Dynamic and not a Static individual, my understanding of testing, and Context Driven Testing will only grow from here out.  It isn't a one time thing to learn, but a thing I will continue to learn and grow and mature as a tester as I put the ideas I've encountered into practice, as I test them, and work through the problems I face professionally.

So what about you?  What do you think is the Context Driven School of Testing really dead?

Note:  Since I set this post up for posting Scott Barber has added a part II on his thinking With the Context-Driven School "closed" what's next? He brings up a very good point about the need for testers to focus on the value they need to provide within their contexts, which I can totally agree with as I am driven to do whatever I can to provide value on a project.

Also I will note that James Bach has also put up a response (which I just now read) If you want to know James Bach's side, I suggest you read Context Driven Testing at a Crossroads.


Edit:

Since posting this, I've found Markus Gärtner blog about this at Lessons Learned from Context-driven testing  And note that Cem Kaner has given his comments on this as well.  It's definitely worth a read.


Additionally, Cem Kaner has added another response (blog) entry at http://context-driven-testing.com/?p=23 that I encourage everyone to read.   This certainly sheds some light on some of the issues that Cem is concerned about, and feels a bit clearer to me about what the issue as he saw it is.

Stepping in the shadow of a monument of process, or standing up?

Somewhere in an old and run down railroad boom town, stands a hundred churches.  Most of the churches were built nearly a century before the Americans with Disabilities Act was even contemplated on the floors of the US Congress.  Some have been updated with ramps or elevators, but most like this one old red brick church have only concrete stairs that lead up and into the hall where the people meet each Sunday morning.  Every Sunday a hundred people or more, women, men, children, and elderly walk up those old concrete steps before that service begins and then, retreats back down those stairs again after the service has concluded.

On this particular Sunday morning, many feet trod up and down those stairs. Each pair of feet appear only conscious of their forward motion, an attempt to keep things moving forward, progress, a process.   So one step after another and soon they reach the destination on another iteration of that Sunday morning hour or more of fellowship and learning.  These little feet may be blissfully unaware, or perhaps they know all too well  that the church is a hundred years old, and has stood tall for longer than those feet have been able to move.  Furthermore, it might seem to those feet, as the creep up those stairs in the shadow of that hundred year old steeple that those feet will continue to visit the same pattern of stairs until the trip can no longer be made.

Of course, then there will be other feet will take its place, and as many feet will continue this pattern to mark up and down those stairs, with their purpose focused on the zenith  where the door opens into the terrace of worship.  So common and repetitive might this routine become that these feet may pay little attention to anything except to each footfall after footfall.  While the conditions of the stairway may sometimes be traveled in the dry, the wet, or perhaps with salt or kitty litter spread about to enhance traction when snow or ice threaten the weekly ascent, but always these feet work their way upward, looking towards the doorway at the top, and yet despite the variety caused by the cycle of seasons, still these feet remain oblivious, giving less and less attention to the grind, the steps each foot falls upon as the habit of the process over takes the analysis and observation that once, during those first few trips up and down those stairs, where the mind noticed the grade, the hardness, and sound of those foot falls on their ascent.

Yet then one Sunday Morning, a half hour before the celebration of this years Scout Sunday a lone boy finds his way up these stairs.  He has walked these stairs before, but it has been months since he last made that climb.  While others around him may feel this is nothing but a normal routine, he takes in the moment, the smell of the cold snowy air, the feel of the chill wind against his cheeks, and the sound of each footfall as he climbs step by step, turning at its half way point to the side, and then up one more step to the doorway. That's when his senses detect it.  A sound altogether different from the sounds of all the previous steps he has heard ring out.

The sound catches his attention, and he lingers for a moment on that step, noting that it is different, but not so sure about it, he opens the door and enters the old church.   Others behind walk up the same step, and not one notices, or comments on the sound he heard as he stands back from the doorway and ponders, waiting for the latest bunch of congregational attendants to finish their ascent.   This boy doesn't attend this church regularly, he goes to another church nearby and across town.  He was only visiting on this occasion in support of the church that had graciously helped support his Cub Scout unit.  Yet a question forms in his mind, unlike all the other habitual comers and goers, going through the usual and traditional process of that weeks iteration that raises his curiosity.

Weekend Testers Americas #24 - Black Box Software Testing: Practice 3

Today I participated in Weekend Testers Americas Session #24.  The mission for this session was fairly open ended.

Mission: 

* In today's mission we would like you to frame your exploration on your 

own. 

* We are about to give you a 'black box machine'. 

* You will have 60 minutes. 

* At the end of the timebox present your notes (no more than page long) to the group chat.

* What to report and in what format - we leave it up to your decision. 

* We hope everybody will bring a unique bit of experience we all can learn from.

Ultimately it boiled down to an exploratory testing session on a practice  flash applet.  Because of the way the mission was framed, each individual tester, or testing pair defined the mission for themselves. Since the general feel of the mission was so open, and my instincts were to work the practice as an exploratory session, I felt there was a lot of room to try things, perhaps things I wouldn't normally have done.   While some of the testers paired, I elected to go it alone this time though.

Before I get into my synopsis of the exercise and my conclusions of how that session went, I'd like to give credit to James Lyndsay who I am told is the one who produced this particular practice over at his site http://www.workroom-productions.com/.  There are a number of practice exercises that could be used for some general and basic practice of exploratory testing skills.  From what I've read he also has some more in depth training as well, which could be a useful if exploratory testing is an area you feel could help you in your testing efforts.  I'll leave it to my readers to decide that for themselves.    Now, if you'd like to try the exercise before reading my report, head on over to http://www.workroom-productions.com/papers/puzzle_03.swf and explore it to your hearts content for up to an hour.  Otherwise, what follows would be considered a 'Spoiler'.   So when you are ready to continue reading, please click 'Read More' below.

Reflections on 2011, a year of trial, growth, and questions

It has been a while since I've had time to pick up my bloggers pen.   October is traditionally a hellish month for me, even when work isn't trying, the Cub Scouts have kept me busy all but a single weekend that I recall, and then on Sundays our son plays soccer.   However, things quickly went off the rails this year.   The whole family came down sick over the span of a month, and one week, we were all sick at the same time.  Yikes!  Praise God, we are better now, but that isn't the only change.

My duties on my project have shifted back to more coder oriented tasks, and less focused on testing.  While I enjoy both programming and testing pursuits, I'll admit, I miss the testing aspects of what I was doing before.   It is funny in a way, when I first got approached about a 'Automation testing' position in 2009, I was worried about being pigeonholed as a tester and excluded from some kind of elite club of programmers.  Yet that was a bit a naive thing to worry about in hindsight.  Testing brought back my love of learning in a way I had not felt since college.  It returned to me part of who I always was, but had kept silent in order to make ends meet.  I've learned a lot as I ran that course, and I wouldn't trade the decision for the world.

My current role on project has me pondering though.  I've heard it debated around twitter, about whether you can be both a programmer and a tester.   I know I can do either, but at some point do you not need to decide which to specialize in?  The reality is there are only so many hours in the day for study and growth, and the opportunity cost of each new learning investment, in effect is at a loss for learning something else.  This is a reality that I've now come face to face with in the last two months.   I still have the knowledge from what I learned as a tester, but it has been hard to try and keep up on my learning where testing is concerned, especially when my current responsibilities require me to act in a more code-centric role.

This feeling has left me feeling a bit lost internally because I know I can succeed at anything I choose to focus my efforts upon, it wouldn't matter if it was testing and programming, or some other group of tasks from which I must choose. I have the drive to do what is necessary to succeed.   Still, I find myself at this cross roads because I enjoy doing things that provide value to the teams that i work with, no matter how small or great the achievement may be.  Up till now it hasn't mattered whether I was a tester, programmer, or performing some other service to the project team.  As long as it was value that I added, I was happy, content and felt fulfilled inside,  Yet I find myself feeling as though I am stuck at a fork in the road.

I feel as if I have paused at a great fork where two rivers meet.  One is a possible focused career on testing, the other, a continued focus on programming, its methodology, and potential as a generalist programmer.  Either fork in the river looks potentially enjoyable from a learning stand point, with its opportunity to pause to fish, relax, or just skip a rock to the other side.  Like most rivers though, I realize that I can only paddle up one stream at a time.  Although the left fork might be easier, the right might be the more fulfilling, or the converse could be true.

For nine years, I have worked professionally to develop, test, and support various software efforts.  I have learned something from every experience that I have been fortunate to endure.  I wouldn't trade those experiences away as they define a bit of who I am personally and professionally.  As I enter my tenth year of service in software development, I find myself looking back over the peaks and valleys behind, and ahead up the forks in the river, yet seeing behind the first bend of either is impossible.  So I am presently anchored, where I am at this fork, pausing to consider and reflect upon what my dreams are for the next ten years.  Where do I want to be?  What roads will I need to travel to get there?  These are questions that I have no answer for currently.   So given that the new year is around the corner, I can see myself at least initially focusing greatly upon what exactly it is that I most want to do, and the realities of that choice which may require not just myself, but my whole family to adapt as well.

It may take some time for me to come to some answers, and the pot has clearly become foggy and hard to see how its contents will turn out when I finally reach that conclusion, but I want to consider things more closely, set a plan and then rush after it to attain it.  Perhaps it is the nature of how I 'fell into' my current assignment that is at the heart of this muddled mind of mine.  At least I know it is something I can do for now, while I sort through my feelings and make what could possibly be the biggest personal, and professional decision I have made in my life thus far.

But enough about me, as you read this and other blogs, I imagine you may be reflecting on recent events, just as I have been.  Where do you stand?  What's your dream?  How will you decide what to focus upon this year, and as a result, what areas that get left behind will you perhaps miss when we reach this point a year from now?

Diary of a Soccer Coach: Week 4

You've been there before a well worn meeting room with your team gathered around a table going over a list of action items related to the project you've been working.   Sometimes they are new requirements, perhaps they are refinements of existing functionality, or tweaks of the deployment procedures taking into account lessons learned in that first deployment of the software.   The first practice after that first game,  is very much like this.   Often there isn't enough time to discuss defensive or offensive tactics with the players before the first game.

It is sometimes a matter of perspective, a coach after the first game of the season, or a manager or lead on a team reviewing the steps they took on that first ever critical deployment, the first game, the first actions of substance as far as the customer might see.  So in hindsight, that first practice, that first meeting, is often a discussion of the aftermath.  For our kindergartners, we discuss the issues I noticed during that first game.  There are almost always a few areas to correct, and they aren't always the same from Game 1 in one year versus any of the others.

Typically a reminder of the rules is necessary.  A reminder about which goal we are attacking, which one we defend, a reminder not to use hands except for the Throw-ins, and an encouragement to stop play when the whistle blows and quickly bring the ball to the referee when there is a stoppage of play for going out of bounds.  While errors can occur in any game, I try to point out the mistake, and correct the behavior without singling out any particular team member.  The point after all is not that someone erred, but that we play the game correctly to reduce stoppages of play.

With the instructional league sometimes this is difficult.  Some players have an over arching desire for the ball, and they may indulge this by diving at the ball.  This is a behavior we try to discourage.  For one thing, falling to the ground is as bad as waiting flat footed for the ball.  They aren't upright able to move with the ball, and if they are down where the ball is, there's a higher possibility of injury as other players go for the ball around them.   Sometimes they fall down, and stay on the ground, and again even if the ball isn't near them, this isn't behavior we want to encourage.  Sometimes its a sign that the kid is tired, but they rarely will get into shape if they sit down when they are supposed to be in the game, and the ball could come at any time too.

Ever been on a team where similar behavior happens?  Where they get tunnel vision, seeing only the one task before them at the expense of what is going on around them on the field?   How can we avoid this behavior?  What's worse is what if a team mate ends up blocked or stuck in some area and doesn't realize it?    As professionals we can try to encourage, give a second set of eyes to these issues, but in the end its really up to the individual to get themselves back on track.  We can encourage that team mate to come back on board, but honestly, if that person cannot take the initiative, there may be little we can do to really fix an issue that is internal to them.

Just like with my soccer players, I take them back to basics, breaking down the basics of the pass, of corner kicks and goal kicks, of throw-ins and kick-offs.  Only so much time can be spent on correcting the past, as new challenges and new games await.  So before the second game we spend a little time talking about defense.  Of reminding the kids that in our league, there are no goalies and therefor noone should go into the goal arc even to go after the ball, but more importantly we teach them what to do to protect their own goal.

First involves positioning, if a player is following an opponent bringing the ball up, we show them how they can move their feet with out crossing them, using the balls of their feet to have better response time in their jockeying back and forth.  We show them how to encourage the ball handler to dribble a particular direction, to funnel them away from a straight shot on goal, or to where we hope additional team mates can cut off their lane of advance.  We also try to show them that having everyone covering one person leaves open lanes of passing to the opposition, it leaves area of the field uncovered, and opens up easy attacks on their teams goal.

In software development, testers play a part of defense, not from bugs scoring on them, but from preventing threats to the value of the product.  If the goal as a team is to release a product with value that's usable by the client, then anything that allows the product to be misused, leaves features less than fully implemented, or just plain not covered is a threat that we as testers try to find.   The one difference here is that unlike in soccer where we can see the ball coming many times before it arrives near our zone of defense, in testing we don't have the ability to look at the software and say a bug is coming from here or there.  We have to instead visualize it with our mind.

How can we visualize where bugs might be?  One way is to be involved early in the process, be in with the conversations with the customer or client and helping to determine how the software may be used.  We also must consider the negative, the view of what invalid data, or improper operations might do to the software.  What if a file consumed is missing settings, does the software resort to a default and store that in the configuration for next time?   If you start typing before the software can fully load, will it cause an unexpected behavior?   We can brain storm a horde of test ideas to try to cover the entire areas of the application, but the reality is just like in soccer, we are just one tester, we can only cover so much ground in eight hours of work time.

What about opposition tendencies? It may be possible in soccer to see that certain players tend to favor an attack on goal from the right or left side.  Some players may prefer passing the ball forwards, or looping back rather than continuing forward at a bad angle.  As testers, we can evaluate the software for tendencies, are there certain areas that seem more bug prone, are there areas that are more critical, or more likely to be highly used and thus could cause more risk?  Is there a particular feature set which sets your software apart from another, then that is an area I'd be sure to test.

Then a foul may be called.  Maybe one player pushed or tripped another, maybe it was a hand ball.  Maybe there's an area of your software that is of particular risk to the customer.  They need that feature to work, quickly, to solve a time critical problem.  Whatever the case may be, we try as hard as we can to find every single bug there may be, but the reality is we can't cover the whole of a software that's anything but trivial.  The nature of software and the myriad of systems it may be installed upon create such a large volume of possibilities that we cannot test it all, so we use techniques to break the software down into areas that we can cover.  We find ways to distill problems to a range of possible outcomes, and we try to think of new ways to test old functionality, because you just never know when a new feature may impact an old one.