tag:blogger.com,1999:blog-27265207689829438352024-03-08T13:50:02.984-05:00Discovered TesterContemplation, introspection, transpection, and other thoughts related to testing and life long learning.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.comBlogger31125tag:blogger.com,1999:blog-2726520768982943835.post-37259429810255151602013-03-16T11:00:00.000-04:002013-03-16T11:00:02.761-04:00On the Stoplight Heuristic, and 'Mindless' TDD and Test Automation<br />
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
I read an interesting article this morning about TDD, and one possible outcome where developers are so focused at keeping things 'green' that they may get a bias around their code which drives them to jump to alter the code to make the tests green. It's an interesting read, and a cautionary warning to developers who may too easily fall into the Test Driven Development groove. Please read it before making the jump</div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
<br /></div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
<span style="font-size: x-small;"><a href="http://stoneship.org/essays/tdd-bypasses-your-brain/">TDD Bypasses Your Brain - by Denis Defreyne</a> </span></div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
<br /></div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
</div>
<a name='more'></a><br />
<br />
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
This is a risk I've been keenly aware of in my role as a Software Developer in Test, but it applies equally to Developers whose main priority is line code for applications. Let's consider for a moment, what are the three main statuses we typically see when any automation, be it unit, GUI driven, API, or integration type tests that run and report. It's that stoplight heuristic, Red, Yellow, and Green. </div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
<br /></div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
A lot of people strive to keep things Green, because Green is good, it means you can push forward with more work, it means the tests (or automated checks if you've read some of Michael Bolton's work) are 'passing', and that's a Good thing right? However, that Green could be a false positive. A false positive occurs, when something passes in a test, because, it either isn't checking the right thing, or the right attribute of something in the software, when in fact there is perhaps some defect there that lurks unknown, because we did not think to test for it.</div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
<br /></div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
False Positives are tricky, and hard to test for, because in a fast paced agile world, the pressure to keep pace with your team, to keep progressing the software often puts us into something of a 'cruise control' sense of development, where we're propelled forward at the same velocity, sometimes, without fully considering the implications of any new feature. What's more, something could have failed during tear down in your cleanup or after hooks, and it may have failed silently, giving you no indication that anything was wrong, since the 'expectations' or 'assertions' passed and the test is reported as green.</div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
<br /></div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
Yellow in the stoplight heuristic, often, means inconclusive, or pending. It can be used to signify work that's in progress, but not completed. Or it could be as simple as being marked as pending in RSpec. Yellow readouts can be good, because they remind us we still have work on any given test scenario or example. However, dealing with Yellow, also has its risks. It's easy to delete or comment out the 'pending' line that generates the pending or inconclusive check report. It might even let the test be green. Watch out though, with RSpec, an example can be green, and not have really asserted, (expected) anything. Thus if no errors are thrown, you might have an example which turns green, yet you've not really evaluated anything at all. </div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
<br /></div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
Then there is Red, the bane of CI/CD systems. We don't like to see Red. Like a bull, we often want to rush to attack fixing the issue. But it's important to try to understand why a given check failed first, before making a decision to add a bug fix to the underlying code, or a maintenance story for the test to your backlog. Sometimes Red means, a change we were waiting for has finally happened, so we must now test this feature differently. Sometimes during a sprint, you may expect a change in a test, and wait for it to happen, before merging in a branch of Tests which will now test that new thing. Other times it may mean that the test caught a legitimate failure, or something odd went on in the environment.</div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
<br /></div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
Then there are False Negatives. Tests which sometimes fail, not because the test is wrong, or because the application failed, but because some assumption we used when writing the tests was not true, this one instance when the tests ran. This implies a number of different things. Was the data we ran against clean? Did something change or go out of scope in the course of running the test? Or as, I have personally experienced, maybe it's a timing issue. We love that Automation at times can be very fast, but sometimes, it may be faster than the underlying system, and a race condition ensues. Perhaps in this case we should poll for a brief time to see if the status changes to a correct one.</div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
<br /></div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
We as Testers, Software developers, and Test Engineers must be careful when chasing after the Green bar. The readout of a stoplight is just one thing we can see when looking ahead at our software, but we must not forget that Green simply means, what we checked passed.</div>
<div style="font-family: arial; font-size: 10pt; padding: 0px;">
<br /></div>
<div style="padding: 0px;">
<span style="font-family: arial; font-size: 10pt; padding: 0px;">I've always understood that green, doesn't mean, zero defects are present in the application. It means nothing we checked acted out of the manner in which we asserted parts of the code. Unfortunately, even a red status does not always mean there's a failure in the code. An open/shared environment can quickly become 'unclean' and a failure could result because of something, someone else did, which your tests couldn't account for at the time.</span></div>
<div style="padding: 0px;">
<span style="font-family: arial; font-size: 10pt; padding: 0px;"><br /></span></div>
<div style="padding: 0px;">
<span style="font-family: arial; font-size: 10pt; padding: 0px;">Let's also remember that we can't guarantee defect free software, because there will always exist the possibility that there's something we miss checking, or some variable in the system which we might fail to take into account when we do testing or write automation checks. That's why you'll never take the human being out of testing for evolving systems, because the ability to analyze and dig deeper is something that no machine can do as effectively as the human brain, and even the best Test Automation, like any good software, will only test or check, what you tell it to check.</span></div>
DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com4tag:blogger.com,1999:blog-2726520768982943835.post-90880559237699083372012-07-30T20:45:00.000-04:002012-07-30T20:45:00.879-04:00Tester Reading Challenge: Read Markus Gärtner's ATDD by Example in 15 Days or less!Are you ready for a Tester Reading Challenge? Read Markus Gärtner's ATDD by Example in 15 Days or less!<br />
<br />
Okay, one of my favorite testers, Michael Larsen, AST Board Member, This Week In Software Testing producer, a Boy Scoutmaster, and Weekend Testing America's driving member is often found saying, hey make a bold prediction, and then try and meet that prediction, you may just surprise yourself. Well in the aftermath of CAST 2012 (Conference for Association of Software Testing), I noticed that he and a couple of my favorite testers and MiagiDo members went to by Markus Gärtner's ATDD by Example. My copy arrived just today, and I've been looking forward to reading it during the next couple of weeks. I was admiring the cover, and the size of the book is not too hefty, and thought hey, it's a shame I couldn't get Markus's picture with it. Well actually I did thanks to tweetdeck (see here: <a href="http://twitpic.com/adnenn">http://twitpic.com/adnenn</a>) As tasteful a picture of a book with my notebook computer as it will come. The thought then occurred to me, this book is 200 hundred pages. <br />
<br />
I began to wonder, how long would it take me to read through this, at ten pages a day that would be 20 days. No, that sounds too easy, and with some free time to read padded in my schedule, I decided to challenge myself, and anyone else so motivated to read Markus's book in 15 days or left. For myself, I've set myself a challenge of August 12th to finish the reading. (see: <a href="https://twitter.com/Veretax/status/230093387708125186">https://twitter.com/Veretax/status/230093387708125186</a>)<br />
<br />
I have no idea whether that's a reasonable challenge for me given my commute times to and from work at just over an hour and a half both ways. Nor do i know how much free time I'll have in the next few weeks, yet ten pages a day seems a decent morsel to both read and digest and it's high time I tried something bold. <br />
<br />
Not only that, I'm challenging every tester in the community, who has not already done so to purchase this book, and try to beat, match, or meet the goal of reading this 200 page tome in 15 days or less. If you take the challenge, I'd love to hear from you, and your impressions of the book as I'm fairly certain it's reading may help many testers look at their craft in new ways.<br />
<br />
<br />
So, will you take the challenge?<br />
<br />
Here's a quick link to the book on Amazon, should your local bookstore not carry it <a href="http://www.amazon.com/ATDD-Example-Test-Driven-Development-Addison-Wesley/dp/0321784154/ref=sr_1_1?ie=UTF8&qid=1343694590&sr=8-1&keywords=atdd+by+example">ATDD By Example: A Practical Guide to Acceptance Test-Driven Development</a><br />
<br />
<br />DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com3tag:blogger.com,1999:blog-2726520768982943835.post-18371819609054180212012-07-04T09:30:00.000-04:002012-07-04T09:30:03.374-04:00Are You Prepared for Crisis? Retailers Need a Back-Up Plan for OutagesAs I mentioned in my previous post, many around us have been dealing with the effects of the Summer <a href="http://www.wvmetronews.com/hoppy.cfm?func=displayfullstory&storyid=53594">Derecho storm</a> that has impacted much of the eastern US. Some of us are even effected when our businesses and homes are not located in the path of the storm as its affects no doubt impacted communications for the internet, hosting for many internet sites, and as Matt Heusser has written over at 21st Century It: <a href="http://www.21cit.com/author.asp?section_id=2063&doc_id=246826&">Cloud Computing: Keeping Things Simple, Except When It Doesn't</a>, an article which discusses how this emergency impacted Amazon's EC2 for several businesses and how some CEOs may react to the outage.<br />
<br />
I in turn, posted in the comments there a long comment about how people of non cloud hosted solutions should not feel themselves somehow safer either, but I wanted to also post that comment (edited a bit for grammar) for the rest of my readers to review. Please take a look and let me know in the comments, what areas you may have seen where companies could do more to avoid catastrophes like this, and in some cases, if it is just an 'Act of God' away from occurring, what can we do to minimize how this affects, not only our own corporate bottom lines, but the citizenry of an ever growing networked society? Note that I've added a lot to what may be a comment some of you will have already read, but I think it is worth reading a second time.<br />
<br />
In an era with ever increasingly severe, and unpredictable weather at times, I find that the story of Amazon's EC2 services outage, (a major outage to be sure) to be just one of many. Look at the recent storm which has impacted several million citizens in just one night from Indiana, all the way through Ohio, West Virginia, Virginia, Maryland, DC, etc. So, it isn't just cloud hosted solutions which are vulnerable to outages of this type. Non-cloud solutions are no safer at times. <br />
<br />
A great example are Point of Sale systems, such as those used by so many grocers. retailers, gas stations, convenience stores, restaurants, etc. When the power is out at many of these locations, the expectation is it will quickly turn back on. Yet as I a West Virginian look at the news I cannot help but look at major chains, like Krogers, Walmart, Food Lion, Foodland, Dollar General, and others who serve a large area and have now after nearly two plus days of little if any power have had to dispose of millions of dollars, <a href="http://www.wvmetronews.com/news.cfm?func=displayfullstory&storyid=53592">countless tons of perishable products</a> - everything from meats, cheeses, refrigerated juices, prepared meals, <a href="http://www.wvmetronews.com/news.cfm?func=displayfullstory&storyid=53597">milk</a> and deli meats and side items, and also highly time sensitive produce. <br />
<br />
It saddens me that in the effort to fine tune profit lines, companies like these which provide vital services are caught just as the local populace without any type of failover plan. At some stores even things that could be used and necessary in the short term like candles, matches, propane, grills, charcoal, and bottled water remained on shelves because stores had no fail over plan for how to handle inventory when the computers and power were down. Retailers of all sorts turned people away without cash as credit card and even check verifying machines remained unable to function or connect due to lack of power, or down telephone lines left many desperate consumers stranded without cash. Even if people had cash though, many stores had insufficient cash reserves, or lack of ability to process and sell merchandise absent the scanner based UPC code look up machines that so many retailers are dependent upon. <br />
<br />
What's worse? (For the corporate bottom line, that is.) People look far outside their normal shopping zones to shop at other stores of the chain, or (perhaps even more likely), customers take their dollars to spend at your competitors, who no doubt raked in cash hand over fist on simple commodities like generators, ice, bottled water, paper plates, cleaning supplies, and canned and other non-perishable items. Those retailers who had a plan, positioned themselves to not only help the people seeking to find these goods, but likely increased their profit margins in the short term, which no doubt investors of any major retailer will appreciate.<br />
<br />
For me at least, this is something that all companies need to address. Whether they are cloud or not cloud related seems irrelevant at this point. When I read that retailer X or Y contemplated buying a generator at some point but thought it would be used too infrequently to justify it's cost, I can't help but shake my head as more money is walking out the door in dumpster bins then in their bank accounts. For me, it was sad to see so much, could have been bought, or given away product (if you wanted a public relations effort in this environment), been cooked, and eaten by some who maybe had nothing for that day. With freak ice storms, out of control wildfires, rain storms, hurricanes and tornadoes, and other such weather phenomena becoming more common in occurrence if I were a CEO I'd be rethinking my fail over plans at the local level when connectivity and power goes down.<br />
<br />
Unfortunately, for those of us still stuck in this state of emergency, we are forced to pick up the pieces and begin the clean up as many more tons of lost food, and damaged homes and businesses will have to be performed before we can all get back to a state of normalcy. So if you happen to live in one of these areas, pay attention to your normal sanitation schedule, and watch for instructions on local news or radio broadcasts on how to <a href="http://www.wvmetronews.com/news.cfm?func=displayfullstory&storyid=53609">properly dispose of the spoilage</a> so as not to attract rodents, and further complicate an already abysmal situation for many of us.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-8096984362106000132012-07-04T09:10:00.000-04:002012-07-04T09:10:00.563-04:00After the July 2012 Derecho Storm: A Call for Service to allHello, again, I know it's been a while since I last picked up the blogger's pen. While the initial cause of my absence was do to many a real life concern that had taken most of my attention, most recently I have accepted a job with Rackspace in Blacksburg, VA starting soon. I can't wait to start my endeavors there, learning new stacks, new technology, and helping test the things they have coming down the pipe. I may post more about this transition at some time, but for now I'm playing the blogging idea by ear.<br />
<br />
However, before I put the pen down again for a bit as I transition to something new, I have one more thing I feel the urge to write about. For those not aware, this past Friday, the 29th of June, 2012, the eastern US was hit by a massive fast moving derecho storm which caused destruction on a vast scale and scope. A state of a emergency was declared in many states, including my home in West Virginia as high winds knocked down power transmission lines, felled trees and limbs, hindered communications, affected water supplies, and a host of other acts of destruction which are still impacting these areas. My home in Summers County, West Virginia was in an area heavily impacted by this storm, and we went nearly 72 hours without electricity, internet, television, or other modern conveniences like air conditioning. What's more many of my neighbors lost several hundred pounds of stored food ranging from meats to produce as the thermometer climbed to near 100 Fahrenheit on some days, with brief showers providing only a small respite. <br />
<br />
As work crews, continue into their fourth or fifth day of activities in areas where repair times for an entire county may be measured in days or weeks instead of hours, it is easy for many to get hot under the collar, angry at the slow turn around times, gas shortages in some places, hard to find ice or bottled water, so I ask that everyone in these affected areas look to your neighbors. Help each other, and try to be smart with any purchases you make during this crisis. This storm is at the least a storm of the decade in terms of damage, but may approach century levels when you consider the amount of Fuel, Food, and equipment deployed in this massive repair effort. So look to your fellow man or woman, and if it is in your ability to help out even in a small way, please do so.<br />
<br />
I have a post I'm working on to illustrate one major testing issue that came to my awareness this past weekend, but I want to take time to revise it a bit before posting it. So to those who cannot yet read this due to power outages, fear not, help is coming. For those in unaffected areas, consider donating to your local food banks, and the families of those brave workers who are travelling far from home to help those who are in need. A donation of food may be very helpful for a food bank or pantry already tapped and stretched to the limit as people in surrounding states tax a system that is already struggling to provide for so many unemployed families. I encourage you to consider donating a couple canned goods or non perishable items (as defined by your local pantry) in support of relief efforts, not just in these affected areas, but in your home towns and counties, as well as for the areas even now affected by the wild fires in Colorado. <br />
<br />
For those who have already given of their time and resources, I want to thank you for your help and offering, those of us who have experienced this crisis may never be able to thank you personally for your hard, tireless, and perhaps round the clock work to help those of us whom you may never even see face to face. Thank you all.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-44186003338151569872012-03-27T21:23:00.000-04:002012-03-27T21:23:34.819-04:00Elevator Optimization Challenge: Part 2Earlier I posted an Elevator Optimization Challenge and asked for feed back on what kinds of questions you might need to ask in order to begin suggesting a course of action to make the elevator's usage more efficient.<br />
<br />
If however you missed it, please return to <a href="http://discoveredtester.blogspot.com/2012/03/elevator-optimization-challenge.html">Elevator Optimization Challenge</a> for an overview.<br />
<br />
Once you've familiarized yourself, you can continue after the jump.<br />
<br />
<br />
<a name='more'></a><br />
<br />
For this exercise it becomes apparent that you must establish some details about the elevators in a given building. Specifically, what are their characteristics, how do they operate. You might even go to the abstract and consider the question of just what is an elevator. An elevator can be defined two ways:<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span><br />
1. A platform or compartment housed in a shaft for raising and lowering people or things to different floors or levels.<br />
<br />
2. A machine consisting of an endless belt with scoops attached, used typically for raising grain for storage.<br />
<br />
For the purpose of this exercise we will assume it is definition #1. An elevator being a platform or compartment housed in a shaft for raising and lowering people or things to different floors or levels. We can also assume that even if an elevator may possibly be used for transportation of cargo, it is designed with the idea that people may also ride along. This is an important distinction because now we know that safety issues for an elevator will be related to human tolerances under which the elevator may be operated safely. You can picture any typical type of elevator you might find in a building. <br />
<br />
Once we have an idea of the general function of an elevator, we can think about it's components, or key parts. Each elevator typically has a control interface, one internal to the elevator lift compartment which is used to select a number of floors (ignore other options for this exercise), and an interface external to the elevator doors which is used to call an elevator for pick up to take its passengers or cargo up or down to another level.<br />
<br />
Secondly, each elevator, sometimes pairs of elevators will have a mechanism which is used to raise or lower the elevator lift compartment to the desired level. While there are many different types of mechanisms used for raising and lowering elevators including but not limited to: Mechanical Gears, hydraulics, or cables, the exact mechanism really isn't of importance for this exercise. For our purposes, you may assume that all the elevators in a building are of a similar lift type.<br />
<br />
There is also a regulator interface which controls the speed and acceleration of the elevator as it accelerates up to its normal speed, and decelerates to stop at the next chosen level. As you can imagine, different types of elevators may raise or lower at different rates of speed, and may accelerate fast or slow. However, this difference really isn't going to have much bearing on this exercise because this speed is a function of the mechanism to lower and raise the elevator lift, and is also governed by a limiter which prevents each elevator from exceeding an acceleration or speed threshold that would be unsafe for human passengers. So you can in effect treat this as a constant for each building, even if elevators in another building might move at a different rate of speed. While it might be possible to 'boost' the mechanism for an elevator, we will assume that the owners of any given building will require us to remain within prescribed tolerances under which the elevator was installed and that tampering with this aspect of the elevator's motion mechanism could violate or void it's warranty or service agreements.<br />
<br />
Now, we may also wish to consider the elevator's size attributes. Each elevator also has a size factor, the volume or square foot area that encloses the people or cargo it can haul, and more importantly it has a maximum weight tolerance that it can permit before a warning sensor triggers an alarm. When the alarm is triggered, cargo or passengers must exit the elevator until the weight has fallen below that maximum weight threshold. The elevator will not move until this has been achieved.<br />
<br />
<br />
Those are the givens of this and any elevator mentioned in this exercise. So the question you ask was a very good one, how can you optimize an elevator? Because of the capital expenditures involved to install, or replace an elevator, and the time factor involved in replacement (and the annoyance to clients as a single or multiple shafts are out of commission), it seems an unreasonable possibility for our situation to increase the capacity either in terms of maximum number of persons carried or weight that the elevator can lift. Suffice it to say, that's just not reasonable for our clients. They are hoping we can offer a software solution, a patch to the computer which controls where the elevator goes. So while changing the physical size of the elevator's lift compartment and shaft could be done, we are not in the elevator 'building' or 'replacement' business. Think of the us in this exercise as an elevator software 'tuning' company.<br />
<br />
So given this detail, what questions need to be asked that will help identify how to 'tune' the elevator to increase its efficiency through out the day. This tuning won't be related to its hardware, or the parameters we established above, so what can we change? I think the next question is simple, how does the elevator move when it receives commands? That will spawn a number of questions I'm sure, but one that sticks out to me is what happens after the elevator has answered its last 'order' and the last passenger or cargo has been removed, and the elevator's doors have closed? What happens?<br />
<br />
In my mind there are a few possibilities:<br />
<br />
1. The elevator does nothing, it stays on the floor it last serviced until summoned by another call for service.<br />
2. It returns to where it started, perhaps the first floor, or some main entrance floor as defined by the building's business rules.<br />
3. It might return to some other standby floor where it can reduce its travel time to the next pick up floor upon being summoned. That standby floor could be a fixed position, or could it might change depending upon the time of day or recent elevator activity.<br />
<br />
Given this what other information would we need to cull from the usage of the elevator to attempt to suggest a pattern to better optimize it's usage? Then once you've gotten the answers to those questions, consider whether there are any special cases you might have forgotten. I'll add more later, because I don't want to give away what I think some of these questions are just yet, but this is how I'd frame the discussion to begin moving towards a solution that we can build and then verify via testing.<br />
<br />
It looks like a third post on this Testing Challenge may be forth coming. I hope this second part has helped peel back a few layers of the onion and get the mind thinking in terms of how you can accomplish the challenge.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com2tag:blogger.com,1999:blog-2726520768982943835.post-59502364378212079142012-03-09T14:15:00.001-05:002012-03-09T17:01:03.932-05:00Elevator Optimization ChallengeIn my life I've lived, worked, and visited many buildings with elevators. Some had a single elevator and a single digit number of floors. Some had multiple public, private, or service related elevators, twenty or more floors. Some count starting at Ground or 1 (First Floor), some have basements, sub basements, garages, or other floors outside the normal first floor up to the top floor. I once lived in a dormitory that had a ground and zero floor, consider that for possible context. <br />
<br />
Much has been written about elevator optimization, and I feel just as I would for many other problems that optimizing the elevator's function might not only depend on the context of the tenants and users of the building's elevator. The context could even change as tenants moved in and out of the building not just through the day, but throughout the life time of the building. <br />
<br />
So the thought occurred to me, what if a tester were approached to work for an elevator tuning company to help optimize the flow of traffic in the elevators. What common questions would need to be asked to figure out the building's context? So here's the challenge. Below I am presenting four different building possibilities. I'm less concerned about the optimization of each building in the future but how to currently maximize its efficiency. So here are the Buildings:<br />
<br />
<br />
Building 1: <br />
Office Building A: <br />
Floors: 5 Numbered 1 - 5<br />
Entry/Exit: Floor 1<br />
Elevators: 2: 1 Public<br />
Tenants:<br />
Floor 1: 2, Law Office, CPA's office<br />
Floor 2: File Storage, Break Room, Copy Center<br />
Floor 3: Conference Room, Reception area, Storage<br />
Floor: 4: Software Company: Management, Developer Offices<br />
Floor: 5: Training Room, Support Desk Cubes<br />
<br />
Building 2:<br />
Apartment Complex<br />
Floors: 12 Numbered B, G, 0-9<br />
Entry/Exit: Floors B and G<br />
Elevators: 2 Public Single controller<br />
Tennants:<br />
Floor B: Small Commerce, Post Office, Book Store, Weight Room<br />
Floor G: Managers Office, Cafe, Storage<br />
Floors 0-9: Tenants about 25 apartments per floor (for the purpose of the exercise assume an assortment of, 2, 3, Bedroom furnishings, with kitchen, living room, and a single bathroom (not shared)<br />
<br />
Building 3:<br />
Hospital:<br />
Floors: 15 Numbered, B, G1, G2, 1-13<br />
Entry/Exit: B, G1, G2, 1<br />
Elevators: 8 4 Public 4 Service/Maintenance (2 separate banks on separate controllers)<br />
Floors:<br />
B: Basement/Storage, limited diagnostic laboratories<br />
G1: Garage First Floor entrance/exit, Walk in Clinic<br />
G2: Garage Second Floor, Reception area, and Shop, Cafeteria.<br />
Floors 1-13 Various layouts for residents, surgery, etc Exact arrangement doesn't matter<br />
Visiting Hours are typically from 8-11 AM, and from 3-830 PM for the hospital<br />
<br />
Building 4<br />
Small Office Building<br />
Floors: 5 Numbered 1-5<br />
Entry/Exit: 1, 2, 3<br />
Elevators: 2, separate banks and controllers<br />
Floors 1 and 2 are parking garages, where Floor 1 is the least used of the two.<br />
Floors 3-5 are office spaces, most of the traffic is to floors 4 and 5 though <br />
<br />
<br />
So given these buildings, what questions would you first need to answer before you could begin to recommend suggestions for optimization? I know I have a few in mind myself, but I'll leave this post be for now, and then maybe add a continuation post later on. What questions would you ask to accomplish the optimization?<br />
<br />
<br />
Note: This is probably the first time I've proposed an idea for a Test Challenge, so if its formulation or presentation seem off, or missing something, please feel free to let me know. You can drop me an email if you'd rather not post in the comments.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com4tag:blogger.com,1999:blog-2726520768982943835.post-31457920155677308042012-03-01T12:05:00.004-05:002012-03-02T13:07:36.451-05:00Is the Context Driven School of Testing - Dead?<div class="tr_bq">Earlier this week, <a href="http://scott-barber.blogspot.com/2012/02/is-testing-dead-dunno-but-context.html?spref=tw">Scott Barber</a> pointed out an update to the <a href="http://context-driven-testing.com/?page_id=9">Context-Driven-Testing page</a> which has served as a starting point for people who want to learn about Context Driven Testing. In this post Kem Caner, one of the 'big four' of the Context Driven School gives an impression that the idea of Context Driven Testing being a 'School' has perhaps reached a point where it no longer exists. He goes on to cite differences in vision among the founders of the movement, and I'll let you read his post above to draw your own conclusions. However, when I was asked this question, I felt that I didn't agree. I've seen places where friends or close colleagues go their separate ways because of a polarizing my way or the high way sort of mentality. I'm not sure that's quite what Kem is referring too.</div><br />
I debated whether to post my response or thoughts to this here at all as no doubt everyone involved or outside this movement will have their own opinions and beliefs. I decided that for posterity's sake that I would indeed. I have discussed this a bit with some fellow testers in the MiagiDo group, and Michael Larsen another MiagiDo member recently made a post giving his view on things (<a href="http://www.mkltesthead.com/2012/02/in-support-of-context-driven-principles.html">In Support of Context Driven Testing</a>). It's worth a read, and his thoughts were a bit similar but also slightly different from my own initial response to this question which I gave a bit earlier. What follows was my initial response to this question of whether Context Driven School of Testing is Dead as it was originally shared with others in MiagiDo. The only changes are the addition of Italics, and a few grammatical improvements.<br />
<blockquote class="tr_bq"><i>I'll have to quote Monty Python on this one.</i> </blockquote><blockquote><i>"I'm not dead yet!"</i> </blockquote><blockquote><i>I read Scott Barber's post, and I read the information there. I really chaffe a bit and comparing this to the polarization of a few figure heads within the American republican party. Kaner mentions Gingrich, who no doubt is a polarizing figure. If you're an American, you likely either like him, or you hate him, there probably isn't a middle ground for most Americans. However, I think perhaps the 4 Founders have gotten a bit too caught up in their own mystique.</i> </blockquote><blockquote><i>I see the same thing within our American political discourse. People at the so called top seem so polarizing, and the media tries to frame discussions a certain way, and yet most of us at the bottom, outside that top so called 'chosen' echelon are not polarizing, we are not so closed minded or resistant to having honest discussions about things. </i> </blockquote><blockquote><i>I think this is true also for the Context-Driven Community, whether we call it a School or not, to me is irrelevant. The point is we are making a fundamental distinction in where our thinking begins. I see this same problem in the US Education system. A lot of people expect school, especially college to prepare them for everything they need to succeed in a career. The reality is quite opposite, it's a base, a starting point. If you get that strong base, I believe a strong education helps you succeed no matter what you apply your life too. I feel the same here with Context-Driven School of testing.</i> </blockquote><blockquote><i>Listen, so the four founders of the movement disagree. There are four gospels in the New Testament of the Christian Bible, (I use this as an example), and all four a bit different, varied, and share some similarities too. Those guys were alive at the time right? Yet they were not the same as each other, they had differences and disagreements no doubt. So let's be clear the entire point of our School (in my estimation) is to not have Drone or Fake testing right? I believe the point, as I understand it, is to encourage people to use and develop their critical thinking skills, and to not be satisfied with testing that feels lacking in covering the real risks of a project.</i> </blockquote><blockquote><i>Let's look at this another way. James Bach gave a talk (I believe this was at Oredev, see James videos posted at his site at www.satisfice.org if interested) about Renaissance thinking a few years ago. He describes what was fundamentally different about it, and how it wasn't just what was happening In Italy, or Scotland, or wherever else, that there were little movements pretty much elsewhere. Think about the artists themselves, they all approached art a bit differently, yet they don't look like carbon copies even if they've learned some of the same skills and techniques. This is no different than how we each approach testing in the Context Driven School of testing. We are still different then what has come before. We are still different than the more agile-centered testers.</i> </blockquote><blockquote><i>We cannot deny who we are. We've been enlightened, we've grown as a community, a school. It would be impossible for us to try to put the genie back in the bottle now. What's done is done, the shot has been fired and heard around the world now. The question is, in any revolution, what will each of us do because of it.<br />
<br />
That's my feelings on it anyways.<br />
Hope that wasn't too preachy,<br />
-Tim Western</i></blockquote><br />
So in conclusion, while there may clearly be differences between the founders of the Context Driven moment, its principals and the School of thinking that it began, I must stand up and say, hey I'm still Context Driven, I still belong to that School of testing. Now I do think there is a risk here. There's a risk in perhaps holding too close to the fundamentals of the school. We need to be aware of other ways of doing things, even if we may find that our way is better. As an engineer, a tester, or a scientist I would be looking for ways to prove my own methods wrong, just as so many other scientists have done for centuries. Yet for now, as a Dynamic and not a Static individual, my understanding of testing, and Context Driven Testing will only grow from here out. It isn't a one time thing to learn, but a thing I will continue to learn and grow and mature as a tester as I put the ideas I've encountered into practice, as I test them, and work through the problems I face professionally.<br />
<br />
So what about you? What do you think is the Context Driven School of Testing really dead?<br />
<br />
Note: Since I set this post up for posting Scott Barber has added a part II on his thinking <a href="http://scott-barber.blogspot.com/2012/03/with-context-driven-school-closed-whats.html">With the Context-Driven School "closed" what's next?</a> He brings up a very good point about the need for testers to focus on the value they need to provide within their contexts, which I can totally agree with as I am driven to do whatever I can to provide value on a project.<br />
<br />
Also I will note that James Bach has also put up a response (which I just now read) If you want to know James Bach's side, I suggest you read <a href="http://www.satisfice.com/blog/archives/724">Context Driven Testing at a Crossroads</a>.<br />
<br />
<br />
Edit:<br />
<br />
Since posting this, I've found Markus Gärtner blog about this at <a href="http://www.shino.de/2012/02/29/lessons-learned-from-context-driven-testing/">Lessons Learned from Context-driven testing</a> And note that Cem Kaner has given his comments on this as well. It's definitely worth a read.<br />
<br />
<br />
Additionally, Cem Kaner has added another response (blog) entry at <a href="http://context-driven-testing.com/?p=23">http://context-driven-testing.com/?p=23</a> that I encourage everyone to read. This certainly sheds some light on some of the issues that Cem is concerned about, and feels a bit clearer to me about what the issue as he saw it is.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com4tag:blogger.com,1999:blog-2726520768982943835.post-6129091801393173692012-02-17T17:32:00.004-05:002012-02-17T17:32:00.768-05:00Stepping in the shadow of a monument of process, or standing up?Somewhere in an old and run down railroad boom town, stands a hundred churches. Most of the churches were built nearly a century before the Americans with Disabilities Act was even contemplated on the floors of the US Congress. Some have been updated with ramps or elevators, but most like this one old red brick church have only concrete stairs that lead up and into the hall where the people meet each Sunday morning. Every Sunday a hundred people or more, women, men, children, and elderly walk up those old concrete steps before that service begins and then, retreats back down those stairs again after the service has concluded.<br />
<br />
On this particular Sunday morning, many feet trod up and down those stairs. Each pair of feet appear only conscious of their forward motion, an attempt to keep things moving forward, progress, a process. So one step after another and soon they reach the destination on another iteration of that Sunday morning hour or more of fellowship and learning. These little feet may be blissfully unaware, or perhaps they know all too well that the church is a hundred years old, and has stood tall for longer than those feet have been able to move. Furthermore, it might seem to those feet, as the creep up those stairs in the shadow of that hundred year old steeple that those feet will continue to visit the same pattern of stairs until the trip can no longer be made.<br />
<br />
Of course, then there will be other feet will take its place, and as many feet will continue this pattern to mark up and down those stairs, with their purpose focused on the zenith where the door opens into the terrace of worship. So common and repetitive might this routine become that these feet may pay little attention to anything except to each footfall after footfall. While the conditions of the stairway may sometimes be traveled in the dry, the wet, or perhaps with salt or kitty litter spread about to enhance traction when snow or ice threaten the weekly ascent, but always these feet work their way upward, looking towards the doorway at the top, and yet despite the variety caused by the cycle of seasons, still these feet remain oblivious, giving less and less attention to the grind, the steps each foot falls upon as the habit of the process over takes the analysis and observation that once, during those first few trips up and down those stairs, where the mind noticed the grade, the hardness, and sound of those foot falls on their ascent.<br />
<br />
Yet then one Sunday Morning, a half hour before the celebration of this years Scout Sunday a lone boy finds his way up these stairs. He has walked these stairs before, but it has been months since he last made that climb. While others around him may feel this is nothing but a normal routine, he takes in the moment, the smell of the cold snowy air, the feel of the chill wind against his cheeks, and the sound of each footfall as he climbs step by step, turning at its half way point to the side, and then up one more step to the doorway. That's when his senses detect it. A sound altogether different from the sounds of all the previous steps he has heard ring out. <br />
<br />
The sound catches his attention, and he lingers for a moment on that step, noting that it is different, but not so sure about it, he opens the door and enters the old church. Others behind walk up the same step, and not one notices, or comments on the sound he heard as he stands back from the doorway and ponders, waiting for the latest bunch of congregational attendants to finish their ascent. This boy doesn't attend this church regularly, he goes to another church nearby and across town. He was only visiting on this occasion in support of the church that had graciously helped support his Cub Scout unit. Yet a question forms in his mind, unlike all the other habitual comers and goers, going through the usual and traditional process of that weeks iteration that raises his curiosity. <br />
<br />
<br />
<a name='more'></a><br />
<br />
<br />
The young boy makes his way back down the steps, and this time he takes a slightly different path on the other side of the stairway. As he steps down the stairs, he notes that this time it sounded a bit different. He begins to question as he continues his way back down to the bottom of the stairway. Had he misheard? Did he really hear it wrong? Or had he stepped a bit different on that step and just didn't notice? Had he perceived a discrepancy where none existed? In that moment the boy considers whether it had all been in his mind, after all a hundred others had come and gone on these steps already that morning, surely at least one other person would have noticed something amiss. He begins to be skeptical of his own observation, thinking that because he is new here that maybe it has always sounded that way, or perhaps there was another sound at the moment which he had confused with his own footfall.<br />
<br />
That explanation could be the truth, yet he still feels that calm voice inside saying, "Hey, I did hear something, so what was it?" At this point he thinks about the difference between ascent and descent on the stairway and ponders whether the step may sound different depending upon which direction he traveled on the stairway. That alone could be sufficient answer to why the sound had been different, but how do you prove it?<br />
<br />
The young boy considers the time on his watch and then shrugs his shoulders. Then he begins to ascend the steps again. Step by step, they all sound the same, with each footfall, each sound just like the other he begins to think that maybe it was nothing after all. He smiles as his mind begins to drift from this curiosity to focus on the other tasks that morning. He turns and steps again on that last step, to hear a hollow 'thunk' sound. There it was again! That sound was the same as before. He had reproduced the effect, and now was confident it wasn't just something he had imagined.<br />
<br />
To prove the point, the boy begins stepping up and down off that single step repeating the sound over and over again. There can be no doubt to him now, that it does sound different than all of the other steps. Then another question comes to his mind: What had he heard when he came down though? He quickly realizes that he had ascended the stairway on its right side, but upon descending he had been on the left (his right side on the way down). Would it make the same sound he made on the right side if he stepped further to his left? So he formulates another experiment, slides to the left and he tries it, and as he suspected, sure enough the step doesn't make this sound on that side. He repeats this test over and over, then moves back to the right to confirm the sound is different there still. <br />
<br />
Now he has reproduced the defect, and shown a bit about its scope, and he has drawn the attention of one of the church's leadership, who are wondering, what this crazy boy is doing. The boy smiles politely, and then begins to explain his discovery. A hundred people before had passed that same step, perhaps for weeks, and not one of them had noted this problem. Yet this outsider, this newcomer of a boy had noticed it even though it really wasn't something that would necessarily be expected to concern this boy.<br />
<br />
Have you ever joined a software team that had been around for a long time? Sometimes teams get so entrenched into the ebb and flow of their processes that the processes themselves create blind spots. A latent defect or bug could be present in the product and go undetected for a long time, simply because no focus, thought, or imagination had brought it to anyone's attention before. Even with a large team, a small, perhaps tiny defect could be dismissed each in turn, yet that defect could be a sign of something about to go wrong with the system being produced.<br />
<br />
I've been in that situation, several times, and it can be a bit of a daunting task. If you are the new guy on the block, and are still learning the ropes of the teams process, learning how they interact and accomplish their daily tasks it can be such a trial to ask, and yet as a newbie that's part of your Job. You have to know how and why things work the way they are. Now imagine this had happened on a Software project. You as a tester or developer ask a more experienced development guy about it, and get a response. How do you respond?<br />
<br />
How would you respond if they say it has always been that way? Do you ignore it and flag it as just an annoyance that everyone knows about? Do you speak up and say hey this could be dangerous, and then articulate your feelings as to why it is a problem? What if noone had noticed it before, yet because you are still in your first month with the team, they discount what you say, perhaps fail to believe you? What do you do then? Would you have courage to confirm your observations, and then try to prove it to the team? Would you have the passion to see that the problem gets addressed, even if the people around you look at you like you've gone crazy?<br />
<br />
These are questions I've had to answer in the past. As a tester, a developer, a leader, a parent, at some point in your life you will find something that is out of place, something that is against the norm, and maybe you are so weary from the travel down life's path that your mind wants to simply dismiss it. If you do, does your mind realize the risks? The risks that threaten your Team, your project, your family or community, do you see them and just ignore them? <br />
<br />
It is too easy at times to fall into ruts of habit, running here and there all the while wearing out the same patch of carpet beneath our feet. How do we combat this apathetic attitude? How do we combat life's fire when just keeping it burning is all we want? <br />
<br />
For me, I do this by testing myself, by testing things around me, and trying things that at times will throw people off. My kids think it's just their father being goofy when I ask them questions about seemingly obvious things. They may even think I'm getting on their case as I ask them to observe something I've noticed. What I'm gauging is their reaction to their surroundings. How do they process things, and how would I respond if I was in their shoes? It isn't easy to be the squeaky wheel. Standing alone when the crowd is seated and content with things to be as they are. <br />
<br />
Yet, I'm not convinced that being silent is the answer. There are times when you may have to table an issue because of priority, or it could be this is not the right venue or time to raise it. It may even be that the thing you've noticed is far more complex than even you might imagine in your mind. What happens after you've raised the issue is something you and the team will have to manage, but do you have the courage to stand out and speak out when you see something that seems to be wrong? For me, as an Eagle Scout, and a member of the Association for Software Testing, and a person who believes integrity is very important for professionals, I would do my best to make the issue known. It might not be something that can be solved over night, and there might be intermediary steps that need to be crossed before it can happen, but I know I would certainly put myself out there to say what I had seen. Even if there's a small chance I might be wrong, that in and of itself is an opportunity for growth and learning. Can you say the same? So as you test your life around you, what kernels in the corner are you ignoring? What patterns have you fallen into that could be obscuring something that's a serious problem, and once you've identified them, what will you do to try to correct them?DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com3tag:blogger.com,1999:blog-2726520768982943835.post-75534039954185114972012-02-08T10:41:00.002-05:002012-02-08T11:30:06.682-05:00Weekend Testers Americas #24 - Black Box Software Testing: Practice 3Today I participated in Weekend Testers Americas Session #24. The mission for this session was fairly open ended.<br />
<br />
<blockquote class="tr_bq"><blockquote class="tr_bq">Mission: </blockquote><blockquote class="tr_bq">* In today's mission we would like you to frame your exploration on your </blockquote><blockquote class="tr_bq">own. </blockquote><blockquote class="tr_bq">* We are about to give you a 'black box machine'. </blockquote><blockquote class="tr_bq">* You will have 60 minutes. </blockquote><blockquote class="tr_bq">* At the end of the timebox present your notes (no more than page long) to the group chat.</blockquote><blockquote class="tr_bq">* What to report and in what format - we leave it up to your decision. </blockquote><blockquote class="tr_bq">* We hope everybody will bring a unique bit of experience we all can learn from.</blockquote></blockquote><br />
Ultimately it boiled down to an exploratory testing session on a practice flash applet. Because of the way the mission was framed, each individual tester, or testing pair defined the mission for themselves. Since the general feel of the mission was so open, and my instincts were to work the practice as an exploratory session, I felt there was a lot of room to try things, perhaps things I wouldn't normally have done. While some of the testers paired, I elected to go it alone this time though.<br />
<br />
Before I get into my synopsis of the exercise and my conclusions of how that session went, I'd like to give credit to James Lyndsay who I am told is the one who produced this particular practice over at his site <a href="http://www.workroom-productions.com/papers/puzzle_03.swf">http://www.workroom-productions.com/</a>. There are a number of practice exercises that could be used for some general and basic practice of exploratory testing skills. From what I've read he also has some more in depth training as well, which could be a useful if exploratory testing is an area you feel could help you in your testing efforts. I'll leave it to my readers to decide that for themselves. Now, if you'd like to try the exercise before reading my report, head on over to <a href="http://www.workroom-productions.com/papers/puzzle_03.swf">http://www.workroom-productions.com/papers/puzzle_03.swf</a> and explore it to your hearts content for up to an hour. Otherwise, what follows would be considered a 'Spoiler'. So when you are ready to continue reading, please click 'Read More' below.<br />
<br />
<br />
<a name='more'></a><br />
<br />
<br />
Before I begin my observations, I'm going to take a moment to be a bit critical of my attempt at this session. Admittedly, it's been a while since I've done focused exploratory testing. I have done some exploratory testing in the past, but my current tasks does not have exploration as a key component as I see it. So there is no established 'routine' for exploration that I might normally have fell back into using. This could have been a plus or a minus. Additionally, in some ways I was less than prepared. While I did use Shmuel Gershon's RapidReporter for note taking, at one point in, I realized I didn't have a notepad and pen handy, (something that normally I always have handy, but for some reason didn't think to grab before this session.) Since I didn't want to lose time to go hunt for a notepad in my other room, I proceeded with out it. In hindsight, if I were doing this again, I think I could have perhaps better organized my effort if I had taken time to use a notepad to organize my thoughts after the first few minutes of the session.<br />
<br />
<br />
What follows is my report as submitted to the session, plus later, notes as recorded into rapid reporter.<br />
<br />
<br />
Page: <a href="http://www.workroom-productions.com/papers/puzzle_03.swf">http://www.workroom-productions.com/papers/puzzle_03.swf</a><br />
<br />
Mission: Explore functionality.<br />
<br />
Page Load testing:<br />
<br />
Chrome: First Try Failed to Load - about:blank; Second Try: spun out but never loaded.<br />
Firefox: Inconclusive, wasn't setup for flash (odd thought it was) (and tried to download the swf)<br />
IE 9: Loads no problems. Will use IE 9 for testing<br />
<br />
Box is Win 7 64 Bit.<br />
<br />
First thought was what areas did the mouse change, or appear clickable. The screen has four buttons in blue and yellow in a 2x4 pattern on the left, a dial gauge that is circular, with a needle, green until like the last twenty percent. A label in the lower left hand corner that says Puzzle 3. A red and green LED below the dial, and a pinkish red image that I'll call a spool in the bottom right. It appears the 4 blue and 4 yellow buttons are clickable, as well as the red spool and ? mark in the bottom right.<br />
<br />
Hovering over the Spool also revealed the Test "work room productions; Test Strategy quality assurance' text. The spool surprised me a bit, because it pops up a message from the swf's creator, but only as long as you hold the mouse button down. Same with the ? mark which gives some hints. Oops noone told me it would give me hints, Spoilers!<br />
<br />
Observations, the text has a transparent background for the spool, and is a bit difficult to read due to this (for me anyways) It's not entirely illegible but someone with poorer eyesight might not be able to read it.<br />
<br />
Next try clicking buttons, initialy thought was to do a zig zag pattern from Blue 1 down. (wondering if order matters Permutations vs Combinations that is.) Tried a sequence going from left to right then down the left to right then down the left to see what happens when all buttons pressed. Noticed that at 7 or 6 buttons the red light may come on, but not sure again if order matters here or not. I tried a number of different combinations here, and then thought if trying some key combinations. A drag of finger across keyboard revealed that the buttons would get 'inset' if a key was pressed, seems that's the U key that causes it. IS this cosmetic or does it affect behavior would require a more detailed plan I think to figure out. <br />
<br />
Stepping back, noting to self that I forgot to have a notebook handy to write things on quickly between clicks. Would really like to to do a truth table for this thing if I had more time I would do so. <br />
<br />
Ooo Pressing S and T seems to cause a target to show up, and I notice that it seems to be clickable with data entered into it. (only numerics though) Interesting. <br />
<br />
<br />
Eventually I got the needle to move, stop, decrease, increase, halt with no lights on, green light on, both green and red, etc. So I wonder what the lights acutally indicate since they don't indicate which direction it's moving, which would have been my thought.<br />
<br />
Conclusion: If I had a note pad handy, a truth table would have been better to get what combinations of buttons did what. Thus more testing would be needed if it were just me to obtain that.)<br />
<br />
<br />
<br />
<br />
That's the short synopsis of the testing we did for about sixty minutes. You can see the reports of other testers at <a href="http://weekendtesting.com/archives/2393">http://weekendtesting.com/archives/2393</a> It is always interesting to look at the approaches others took, and what they discovered. In some cases, testers focused on that combination, and what combinations caused the needle to move, or the lights to come on. Some tried to visualize it as a control system for some system. While my report didn't reflect it, I was imagining it as an air gauge, or a switching system with a Volt or Amp Gauge where the possibility could be that going to far into the red would cause something 'bad' to happen. Nothing seemed to happen when we got there that I noticed though. <br />
<br />
Some of the testers tried to define what the states might mean for the buttons. Is a button up, considered on or off? I didn't really spend much time wondering what those buttons met. If I had access to the customer, which another tester did note, I'd probably have asked for some background for the control system, and that likely would have had a profound impact on how I viewed and proceeded with my testing. One tester, Pradeep Soundararajan, thought he'd seen the exercise before, and rather than just testing the device, spent time devising the possible repercussions of testing it. He spent time identifying useful heuristics that could be needed to properly test something like this. I actually found this refreshing, as he identified some potential biases, and problems you might have to answer when testing something like this.<br />
<br />
I'll let my readers go to the Experience Report over at <a href="http://weekendtesting.com/archives/2393">http://weekendtesting.com/archives/2393</a> to draw their own conclusions, but I definitely benefited from seeing how people in different backgrounds saw testing of this kind. Before I close this blog, I'd like to post my notes from rapid reporter, so you can see my thought process as I went about the testing.<br />
<br />
<br />
<h1 style="text-align: center;">Session Report | Powered by <a href="http://testing.gershon.info/reporter/">Rapid Reporter</a></h1><br />
<input checked="true" id="1" type="checkbox" /> Show autogenerated rows<br />
<div id="aroundtable"><table border="1" style="margin-left: auto; margin-right: auto;"><tbody>
<tr class="Type"><th>Time</th><th>Reporter</th><th class="notetype">Type</th><th>Content</th><th>Screenshot </th><th>RTF Note </th></tr>
<tr class="(Rapid Reporter version)"><td>2/4/2012 1:01:14 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">(Rapid Reporter version)</td><td>1.12.01.06</td><td></td><td></td></tr>
<tr class="Session Reporter" style="background-attachment: initial; background-clip: initial; background-color: #ffff99; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial; font-weight: bold;"><td>2/4/2012 1:01:14 PM</td><td>Tim Western</td><td class="notetype">Session Reporter</td><td>Tim Western</td><td></td><td></td></tr>
<tr class="Session Charter" style="background-attachment: initial; background-clip: initial; background-color: #ffff99; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial; font-weight: bold;"><td>2/4/2012 1:01:14 PM</td><td>Tim Western</td><td class="notetype">Session Charter</td><td>Weekend Testing Americas #24 Mission: TBD</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:01:21 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>Join Skype Converation</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:01:26 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>Say Hi and Check in.</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:03:26 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>Introductions</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:11:36 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>Mission statement:</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:11:53 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>In today's mission we would like you to frame your exploration on your own. We</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:12:04 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>are about to give you a 'black box machine'.</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:12:11 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>You will have 60 minutes. At the end of the timebox present your notes (no more</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:12:20 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>than page long) to the group chat.</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:12:28 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>What to report and in what format - we leave it up to your decision. We hope</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:12:34 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>everybody will bring a unique bit of experience we all can learn from.</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:12:59 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>[1:11:44 PM] Weekend Testers Americas: http://www.workroom-productions.com/papers/puzzle_03.swf</td><td></td><td></td></tr>
<tr class="Setup" style="background-attachment: initial; background-clip: initial; background-color: #e3e3e3; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:14:55 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Setup</td><td>Chrome test opening: http://www.workroom-productions.com/papers/puzzle_03.swf</td><td></td><td></td></tr>
<tr class="Test" style="background-attachment: initial; background-clip: initial; background-color: lightyellow; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:15:03 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Test</td><td>Open in Chrome results in blank screen? odd</td><td></td><td></td></tr>
<tr class="Test" style="background-attachment: initial; background-clip: initial; background-color: lightyellow; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:15:48 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Test</td><td>closed Chrome; try again... now instead of about:blank its spinning.</td><td></td><td></td></tr>
<tr class="Test" style="background-attachment: initial; background-clip: initial; background-color: lightyellow; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:19:04 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Test</td><td>Try Firefox; but it wants to download (may not have plugin installed</td><td></td><td></td></tr>
<tr class="Test" style="background-attachment: initial; background-clip: initial; background-color: lightyellow; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:19:08 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Test</td><td>Try IE 9... and it loads</td><td></td><td></td></tr>
<tr class="Note" style="background-attachment: initial; background-clip: initial; background-color: cyan; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:19:50 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Note</td><td>See 8 buttons 2x4 half blue half gold 2 read out lights left red; right green; and a meter. Label at bottom says puzzle 3</td><td></td><td></td></tr>
<tr class="Note" style="background-attachment: initial; background-clip: initial; background-color: cyan; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:20:20 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Note</td><td>Since only IE 9 will load for me (will explore why later) I'll use IE 9 for testing</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:20:50 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>hover over flash to see what is clickable; or where mouse over changes.</td><td></td><td></td></tr>
<tr class="Note" style="background-attachment: initial; background-clip: initial; background-color: cyan; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial;"><td>2/4/2012 1:21:40 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Note</td><td>Hover over image in lower right reveals ''Work room productions; Test Strategy Quality Assurance text</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:21:53 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>what happens when you click on the icon in lower right; now being called the spool</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:22:52 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Click reveals some text; partiall transparent. Seems to be a message from the author.</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:22:59 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Can't read entire text and type at t same time.</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:23:27 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>text covers most of the app; but it bleeds through making reading a bit tricky.</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:23:51 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Click the ? above the spool; reveals another message.</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:25:03 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>nothing in the blue area appears clickable.</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:25:26 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>So now trying the blue and Gold Buttons; will do a zig zag starting in upper left.</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:26:05 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Seems that four buttons pressed causes green light to come up</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:26:21 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Try all blue: (if any yellow are pressed beyond four it has both lit up</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:26:48 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Green light lights up with B1; Y1; B2.</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:26:56 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Stays on with Y2</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:27:02 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Stays on with B3</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:27:07 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>stays on with Y3</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:27:13 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>stays on with b4</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:27:20 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Goes red with Y4</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:27:23 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>un click all</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:28:05 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Seen combinations of 7 and 6 buttons that will stay green. Y4; B1; Y2; B2; Y2; B3..</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:29:10 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Now check if any single button causes the red</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:30:04 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>S+T Exposes Timer</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:30:49 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>If I click in the target that shows up with the target/timer thing I can type into it</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:30:59 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Only seems to except numerics</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:32:00 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Trying finger drag across keys to see if any other special commands.. noticed that a box can go around the buttons; how did I do that?</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:32:24 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>U puts a box around the buttons</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:33:37 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>the lower right yellow button now seems to cause a red condition when certain pairs of the blue are pressed along with it.</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:33:48 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B1 B2 -> Green</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:33:56 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B1B3 -> Green</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:34:03 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B1-B4 -> Red</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:34:17 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B2;3 -> No indicator</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:34:33 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B3;4 with Y4 -> both red and green</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:34:54 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>If both Y3;Y4 are prssed; either B3;4 causes red</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:35:23 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Now checking pairs of blue with Y3</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:35:35 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B1;2->Green</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:35:47 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B1;3 -> none</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:35:54 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B1; $ -> none</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:36:13 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B2;3 -> Nothing</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:36:22 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B3;4 -> Green</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:36:26 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>Interesting is that the only one?</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:36:36 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>All but B3 -> Green</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:36:43 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>All but B4 -> Green</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:36:55 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>If I had more time I'd setup a truth table for this chart and see what happens.</td><td></td><td></td></tr>
<tr class="Next Time"><td>2/4/2012 1:37:25 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Next Time</td><td>Set up a truth table on a sheet of paper and note what the combinations are. I'm betting there are some that are DOn't Care some that are alway on. could be wrong.</td><td></td><td></td></tr>
<tr class="Next Time"><td>2/4/2012 1:38:20 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Next Time</td><td>Aha; i finally got the dial to move by rapid clicking</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:38:36 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B1; B2 Y2; B4; Y 4 un check B2 and it rolls</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:38:47 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>unchecking B4 causes the dial to go down</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:39:21 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>now what happens when it gets in the red of the dial?</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:39:41 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>dial can be pegged</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:39:44 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>doesn't move</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:39:49 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>now trying to unclick B4</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:40:03 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>red light went off still pegged in red</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:40:13 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B2 and dial is going down again</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:40:35 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>hit combinatoin b1 B2 Y3 B4 Y 4 to stop at 12 oclock position.</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:45:24 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>each button... do they do the same thing each time clicked click each button on and off five times to see</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:46:02 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>not with individual buttons</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:46:12 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>however; I've noticed that sometimes the needle goes up when its not green.</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:46:39 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>B2;3;4 Y2;3 causes needle go down and green</td><td></td><td></td></tr>
<tr class="Check"><td>2/4/2012 1:47:04 PM</td><td>Tim Western</td><td class="notetype" style="font-weight: bold;">Check</td><td>also if you press B2;3; Y2;3 -> Y1 goes down.</td><td></td><td></td></tr>
<tr class="Session End. Duration" style="background-attachment: initial; background-clip: initial; background-color: #ffff99; background-image: initial; background-origin: initial; background-position: initial initial; background-repeat: initial initial; font-weight: bold;"><td>2/4/2012 1:51:23 PM</td><td>Tim Western</td><td class="notetype">Session End. Duration</td><td>00:50:10</td><td></td><td><br />
<br />
<br />
</td></tr>
</tbody></table></div><br />
<br />
What follos is a very a brief conclusion after I looked over the report from RapidReporter before I began to compile my notes to share with the other session participants:<br />
<br />
<br />
Conclusion: There seems to be some sequence or groupings that determine if any, one, or all lights go on, and in sequence one button toggled may affect the dials movement up or down, or freezing. I didn't feel I got accurate information as I was not taking as concise a notes in this regard as I would have liked too. I could see doing a combinatoric check on this thing for combinations up to 7 or 8 buttons before having a full idea of how its logic would flow. <br />
<br />
<br />
<br />
<br />
Lastly, as far as the Weekend Testing session went, I think it went well. I'm glad I attended and participated. I have felt stuck in a rut when I have been testing of late, or focusing on logic and business rules rather than what interfaces may actually be doing. My testing environment at home was actually not a distraction at all, but I'm wishing now I'd taken even five minutes to get a note pad to do a truth table. It would have saved me some effort I think, and perhaps given me more confidence in the session over all. In truth, at present my tasks have been more coding than testing centered for the last few months, so exercises like this were important to keep some of my testing knowledge fresh. If you've never had the chance to attend a Weekend Testing session, or if you are interested in participating in our next Weekend Testing Americas Session, it will likely fall on the first Saturday of March (March 3rd, 2012), and typically starts at 2 PM Eastern time for me.<br />
<br />
<br />
Acknowledgements: The exercise was produced by James Lyndsay who has a number of such exercises for practicing exploratory testing. Thanks to James for allowing us to use this in our session today. His site can be found at: <a href="http://www.workroom-productions.com/">http://www.workroom-productions.com/</a> and this exercise is one of the public Black Box Testing Exercises at: <a href="http://www.workroom-productions.com/black_box_machines.html">http://www.workroom-productions.com/black_box_machines.html</a> I give major kudos to James for having a practice applet like this on his site and would recommend anyone interested in learning more, to please contact him to find out how you can learn more about Exploratory Testing.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-47393611935151958522011-12-30T04:45:00.000-05:002011-12-30T04:45:40.474-05:00Reflections on 2011, a year of trial, growth, and questionsIt has been a while since I've had time to pick up my bloggers pen. October is traditionally a hellish month for me, even when work isn't trying, the Cub Scouts have kept me busy all but a single weekend that I recall, and then on Sundays our son plays soccer. However, things quickly went off the rails this year. The whole family came down sick over the span of a month, and one week, we were all sick at the same time. Yikes! Praise God, we are better now, but that isn't the only change.<br />
<br />
My duties on my project have shifted back to more coder oriented tasks, and less focused on testing. While I enjoy both programming and testing pursuits, I'll admit, I miss the testing aspects of what I was doing before. It is funny in a way, when I first got approached about a 'Automation testing' position in 2009, I was worried about being pigeonholed as a tester and excluded from some kind of elite club of programmers. Yet that was a bit a naive thing to worry about in hindsight. Testing brought back my love of learning in a way I had not felt since college. It returned to me part of who I always was, but had kept silent in order to make ends meet. I've learned a lot as I ran that course, and I wouldn't trade the decision for the world.<br />
<br />
My current role on project has me pondering though. I've heard it debated around twitter, about whether you can be both a programmer and a tester. I know I can do either, but at some point do you not need to decide which to specialize in? The reality is there are only so many hours in the day for study and growth, and the opportunity cost of each new learning investment, in effect is at a loss for learning something else. This is a reality that I've now come face to face with in the last two months. I still have the knowledge from what I learned as a tester, but it has been hard to try and keep up on my learning where testing is concerned, especially when my current responsibilities require me to act in a more code-centric role.<br />
<br />
This feeling has left me feeling a bit lost internally because I know I can succeed at anything I choose to focus my efforts upon, it wouldn't matter if it was testing and programming, or some other group of tasks from which I must choose. I have the drive to do what is necessary to succeed. Still, I find myself at this cross roads because I enjoy doing things that provide value to the teams that i work with, no matter how small or great the achievement may be. Up till now it hasn't mattered whether I was a tester, programmer, or performing some other service to the project team. As long as it was value that I added, I was happy, content and felt fulfilled inside, Yet I find myself feeling as though I am stuck at a fork in the road. <br />
<br />
I feel as if I have paused at a great fork where two rivers meet. One is a possible focused career on testing, the other, a continued focus on programming, its methodology, and potential as a generalist programmer. Either fork in the river looks potentially enjoyable from a learning stand point, with its opportunity to pause to fish, relax, or just skip a rock to the other side. Like most rivers though, I realize that I can only paddle up one stream at a time. Although the left fork might be easier, the right might be the more fulfilling, or the converse could be true. <br />
<br />
For nine years, I have worked professionally to develop, test, and support various software efforts. I have learned something from every experience that I have been fortunate to endure. I wouldn't trade those experiences away as they define a bit of who I am personally and professionally. As I enter my tenth year of service in software development, I find myself looking back over the peaks and valleys behind, and ahead up the forks in the river, yet seeing behind the first bend of either is impossible. So I am presently anchored, where I am at this fork, pausing to consider and reflect upon what my dreams are for the next ten years. Where do I want to be? What roads will I need to travel to get there? These are questions that I have no answer for currently. So given that the new year is around the corner, I can see myself at least initially focusing greatly upon what exactly it is that I most want to do, and the realities of that choice which may require not just myself, but my whole family to adapt as well.<br />
<br />
It may take some time for me to come to some answers, and the pot has clearly become foggy and hard to see how its contents will turn out when I finally reach that conclusion, but I want to consider things more closely, set a plan and then rush after it to attain it. Perhaps it is the nature of how I 'fell into' my current assignment that is at the heart of this muddled mind of mine. At least I know it is something I can do for now, while I sort through my feelings and make what could possibly be the biggest personal, and professional decision I have made in my life thus far.<br />
<br />
But enough about me, as you read this and other blogs, I imagine you may be reflecting on recent events, just as I have been. Where do you stand? What's your dream? How will you decide what to focus upon this year, and as a result, what areas that get left behind will you perhaps miss when we reach this point a year from now?DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com5tag:blogger.com,1999:blog-2726520768982943835.post-5105514372130213952011-10-01T11:56:00.000-04:002011-10-01T11:56:14.771-04:00Diary of a Soccer Coach: Week 4You've been there before a well worn meeting room with your team gathered around a table going over a list of action items related to the project you've been working. Sometimes they are new requirements, perhaps they are refinements of existing functionality, or tweaks of the deployment procedures taking into account lessons learned in that first deployment of the software. The first practice after that first game, is very much like this. Often there isn't enough time to discuss defensive or offensive tactics with the players before the first game. <br />
<br />
It is sometimes a matter of perspective, a coach after the first game of the season, or a manager or lead on a team reviewing the steps they took on that first ever critical deployment, the first game, the first actions of substance as far as the customer might see. So in hindsight, that first practice, that first meeting, is often a discussion of the aftermath. For our kindergartners, we discuss the issues I noticed during that first game. There are almost always a few areas to correct, and they aren't always the same from Game 1 in one year versus any of the others.<br />
<br />
Typically a reminder of the rules is necessary. A reminder about which goal we are attacking, which one we defend, a reminder not to use hands except for the Throw-ins, and an encouragement to stop play when the whistle blows and quickly bring the ball to the referee when there is a stoppage of play for going out of bounds. While errors can occur in any game, I try to point out the mistake, and correct the behavior without singling out any particular team member. The point after all is not that someone erred, but that we play the game correctly to reduce stoppages of play. <br />
<br />
With the instructional league sometimes this is difficult. Some players have an over arching desire for the ball, and they may indulge this by diving at the ball. This is a behavior we try to discourage. For one thing, falling to the ground is as bad as waiting flat footed for the ball. They aren't upright able to move with the ball, and if they are down where the ball is, there's a higher possibility of injury as other players go for the ball around them. Sometimes they fall down, and stay on the ground, and again even if the ball isn't near them, this isn't behavior we want to encourage. Sometimes its a sign that the kid is tired, but they rarely will get into shape if they sit down when they are supposed to be in the game, and the ball could come at any time too.<br />
<br />
Ever been on a team where similar behavior happens? Where they get tunnel vision, seeing only the one task before them at the expense of what is going on around them on the field? How can we avoid this behavior? What's worse is what if a team mate ends up blocked or stuck in some area and doesn't realize it? As professionals we can try to encourage, give a second set of eyes to these issues, but in the end its really up to the individual to get themselves back on track. We can encourage that team mate to come back on board, but honestly, if that person cannot take the initiative, there may be little we can do to really fix an issue that is internal to them.<br />
<br />
Just like with my soccer players, I take them back to basics, breaking down the basics of the pass, of corner kicks and goal kicks, of throw-ins and kick-offs. Only so much time can be spent on correcting the past, as new challenges and new games await. So before the second game we spend a little time talking about defense. Of reminding the kids that in our league, there are no goalies and therefor noone should go into the goal arc even to go after the ball, but more importantly we teach them what to do to protect their own goal.<br />
<br />
First involves positioning, if a player is following an opponent bringing the ball up, we show them how they can move their feet with out crossing them, using the balls of their feet to have better response time in their jockeying back and forth. We show them how to encourage the ball handler to dribble a particular direction, to funnel them away from a straight shot on goal, or to where we hope additional team mates can cut off their lane of advance. We also try to show them that having everyone covering one person leaves open lanes of passing to the opposition, it leaves area of the field uncovered, and opens up easy attacks on their teams goal.<br />
<br />
In software development, testers play a part of defense, not from bugs scoring on them, but from preventing threats to the value of the product. If the goal as a team is to release a product with value that's usable by the client, then anything that allows the product to be misused, leaves features less than fully implemented, or just plain not covered is a threat that we as testers try to find. The one difference here is that unlike in soccer where we can see the ball coming many times before it arrives near our zone of defense, in testing we don't have the ability to look at the software and say a bug is coming from here or there. We have to instead visualize it with our mind.<br />
<br />
How can we visualize where bugs might be? One way is to be involved early in the process, be in with the conversations with the customer or client and helping to determine how the software may be used. We also must consider the negative, the view of what invalid data, or improper operations might do to the software. What if a file consumed is missing settings, does the software resort to a default and store that in the configuration for next time? If you start typing before the software can fully load, will it cause an unexpected behavior? We can brain storm a horde of test ideas to try to cover the entire areas of the application, but the reality is just like in soccer, we are just one tester, we can only cover so much ground in eight hours of work time. <br />
<br />
What about opposition tendencies? It may be possible in soccer to see that certain players tend to favor an attack on goal from the right or left side. Some players may prefer passing the ball forwards, or looping back rather than continuing forward at a bad angle. As testers, we can evaluate the software for tendencies, are there certain areas that seem more bug prone, are there areas that are more critical, or more likely to be highly used and thus could cause more risk? Is there a particular feature set which sets your software apart from another, then that is an area I'd be sure to test. <br />
<br />
Then a foul may be called. Maybe one player pushed or tripped another, maybe it was a hand ball. Maybe there's an area of your software that is of particular risk to the customer. They need that feature to work, quickly, to solve a time critical problem. Whatever the case may be, we try as hard as we can to find every single bug there may be, but the reality is we can't cover the whole of a software that's anything but trivial. The nature of software and the myriad of systems it may be installed upon create such a large volume of possibilities that we cannot test it all, so we use techniques to break the software down into areas that we can cover. We find ways to distill problems to a range of possible outcomes, and we try to think of new ways to test old functionality, because you just never know when a new feature may impact an old one.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-37843417885325455352011-09-27T21:36:00.002-04:002011-09-27T21:49:29.750-04:00If its not random, how to decipher the pattern?Earlier this week, I <a href="http://discoveredtester.blogspot.com/2011/09/pay-freeze-slightly-melted-random-bug.html">wrote about the software fault in the Staunton, Virginia, teacher payroll system</a>. I talked at length about the concept of 'random', and the importance of distinguishing between something that which is truly random, from something that is better described as unexpected, unpredictable, or just 'having no discernible pattern to me as far as my sense go'. Using precise language when describing defects in software benefits everyone on the team, including the customer. <br />
<br />
Unfortunately, the fault in Staunton, Virgnia's payroll system wasn't found by a tester, instead it was discovered by someone researching the finances of the county's school system. Now we may not know exactly how this fault was first brought to the attention of this school district. That doesn't preclude speculation how an investigation of a similar fault on a hypothetically similar system could be conducted.<br />
<br />
So imagine a hypothetical payroll system for an organization with multiple locations, accounting for user entities of diverse pay grades and positions similar to the school systems. The more layers you add to the structure of the system, the greater its complexity. Now let's suppose the vendor of this software received word about an apparent bug. This bug affects certain persons within the system who would receive an unexpected, and here to fore unnoticed pay increase. So if you as a software tester for this software vendor, receive this notice where would you start?<br />
<br />
Reporting and analyzing a defect that a tester stumbled upon through his or her own investigation of the software is one thing. Trying to track down a flaw someone else found and reports is quite another. If we follow the example of the of the system we discussed earlier we can imagine the reports taking the form of output, potentially pay stubs, ledger logs, bank statements, etc. In short, we possess a log or evidence that the problem occurred, but this evidence may be far enough from the system itself to not be able to produce the same conditions without a bit more digging. <br />
<br />
So how can we reproduce these conditions and figure out where the real defect resides? More information is required, and like a software Sherlock Holmes we must examine the evidence, and piece together the story of what happened. In the case of the pay roll system it is likely important to know how many individuals were impacted. Might a search for more information related to the individuals effected, reveal each user to be part of particular entities or organizations with in the system? Did they work at particular locations, or have their data maintained at a particular data center? An exhaustive analysis of whatever data can be culled from the system could help establish a definitive relationship between the affected users.<br />
<br />
From the headlines, it sounds the School district performed an analysis just like this. The result seemed to have something to do with individuals who went to a particular school. Now the age of the defect in the system may not be clear. If a lot of time has gone by, it may be possible that the connection is more subtle, and won't track to any particular organization, or be so obvious; however, in this case we strike pay dirt. One piece of the puzzle is in place. <br />
<br />
Given that all those results might track to a particular organization within the software, this may lead to our first hunch. Were all the people assigned to this organization, also receiving the same bug? This might be where the first bump in the investigation may be encountered. Maybe they aren't all affected. That idea may lead to a belief that our initial hunch was wrong, but it could be that there's a reason why they turned out to be the exception. <br />
<br />
It's at this point in the defect analysis where a history of debugging similar enterprise applications could prove beneficial. From reviewing some of the articles around the defect, a number of ideas come to mind, all of them based on similar behavior I've encountered in other projects I have worked. If these employees all worked at a facility that was shuttered, what happens to their accounts when the facility is shut down? Are they transferred to a new facility? Are they suspended out right? Are they removed from the system?<br />
<br />
I recall once with a customer relation management system that we encountered a bug when a user account was removed from the system. All the records linked to it, would cascade and delete, or disappear and not show up in the system when searched. Could a data integrity issue regarding the integrity of the data for these closed locations be responsible for this behavior? <br />
<br />
Another possibility that occurs to me, is that a system that freezes pay for all employees may apply to a group of employees by group. Might a group that these employees belonged to be used to freeze all of their pay for some time period? Might failing to belong to a group due to the original group being inactivated cause the issue of applying this freeze to miss these accounts?<br />
<br />
It may be difficult to see the cause from just reading the few reports you receive from the user, but a simple logical, and step by step examination of the system could help reveal how the issue happened, and if it was a case of the system being used in a manner that was unplanned by the software vendor, it may indicate a fault in the business rules, or lack of training for the users of the system. Whatever the case, the team is now on its way to finding where this issue occurred. Where would you test next?DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-68904013857821968822011-09-21T19:57:00.004-04:002011-09-27T21:00:31.834-04:00Pay Freeze, slightly melted, a random bug? Maybe.Every now and then I read about a problem in a software system that makes the news. I look at the article and read what is described as the problem, and I often wonder, how this supposed flaw got into the system. In my experience it can be easy to fault the software for an error. There have certainly been enough cases of odd failures for the general public to believe them, but is it really the software? <br />
<br />
This week I heard about the story from <a href="http://www.newsleader.com/article/20110902/NEWS01/109020312/Glitch-thaws-Staunton-pay-freeze">Staunton, Virginia</a>. Apparently the school board had frozen pay for all of its employees for some period of time, and as the article stated, the glitch went uncaught by a number of employees who spot checked this up until a news station requested records for salaries under the freedom of information act. This is when the discrepancy was apparently noticed. Now this glitch appears like something of a scandal. The political black eye alone could be enough to make anyone nervous about the 'quality' of the application in question.<br />
<br />
What concerns me though is that this glitch is being described as completely random. First, do we really understand what it means when something is truly random? According to <a href="http://dictionary.com/">Dictionary.com</a>, <a href="http://dictionary.reference.com/browse/random">random has four customary definitions</a>. The first means 'proceeding, made, or occurring without definite aim, reason, or pattern. The second is its use in statistics, specifically the concept of a process of selection whereby each item of a set has an equal likelihood of being selected. The third definition applies to physical trades, where a part, parcel, or piece of land or item may appear non uniformly shaped. The last one is an informal use implying that it was an occurrence that was completely unexpected, or unpredictable.<br />
<span class="Apple-style-span" style="background-color: white; color: #333333; font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 16px;"></span><br />
<h2 class="me" style="color: black; display: inline; font-family: 'Arial Unicode MS', Arial, Helvetica, sans-serif; font-size: 18px; font-weight: bold; line-height: 1.25em; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><br />
</h2>Let's take a moment and consider the story for a moment. The first definition implies that there is no rhyme or reason, no discernible pattern to something which may make it random. Is that the case here? Reading further I notice the following:<br />
<blockquote><span class="Apple-style-span" style="color: #2c2c2c; font-family: Helvetica, Arial, sans-serif; font-size: 16px; line-height: 24px;">" the pay increase malfunction was random and included three teachers at Bessie Weller Elementary School, four at McSwain Elementary, four at Ware Elementary and a speech teacher and a secondary special education teacher."</span></blockquote>Several of these teachers had one thing in common, they attended one of three elementary schools nearby. Wait does that mean what I think it means, could this be the beginning of an actual pattern emerging, enough to discount the perceived randomness? It could be, but as testers in this situation, our job is to determine the nature of the fault, not just give our 'best guesses'. We know a fault happened, therefore we must find a way to duplicate it. If we continued on this analysis, we'd likely have a couple of test ideas to begin testing, we'd look at the data for all of the affected persons and see just what is it that happened. Is the over payment of salaries here the problem, or is it a side effect of some other hidden flaw that just became visible due to some quality of the instance that we are examining? Fortunately, I did a bit more research and found another article on this on <a href="http://www.msnbc.msn.com/id/44552224/ns/us_news-weird_news/#.TnT4Key-bl6">MSNBC's site</a>. Now I will note that MSNBC's article is dated the sixteenth of September, and the other article earlier on the Second day of September, however, as I read I find another nugget that seems to confirm my suspicion.<br />
<br />
<blockquote><span class="Apple-style-span" style="background-color: white; color: #333333; font-family: Georgia, Times, serif; font-size: 15px; line-height: 24px;">"All the affected teachers had previously worked at Dixon Elementary School and were reassigned to other schools after Dixon closed two years ago."</span></blockquote><br />
So it appears that this bug affected teachers that had all been assigned to a school, that closed two years ago. (No doubt around the time of the glitch actually occurring.) Would you call this random? No I see a pattern, so it doesn't hold on the first definition. The Second definition doesn't hold up to the story at this point either, as given a sampling so large, would you really expect to find just a handful of salaries that are wrong? I don't buy that either. The third definition doesn't apply in this context, which leaves us with the remaining informal definition: simply that it was odd or unpredictable <br />
<br />
This fact I do not doubt, no one predicted this to happen. Now I'm not writing this to criticize the vendor or the county in question where this happened. That's not the point of this article. Instead, my hope is to make you think. As a tester, developer, user, consumer of computing appliances, how often do we encounter behavior that surprises us? How often do we not only get surprised but feel the event to be unpredictable, with no reason it should be happening?<br />
<br />
I imagine this happens more than we might like to admit. How many times do we sit at our computers, doing something normal. We're checking our email in our client of choice, we have had no problems with our service and expect to get a no messages found if the service has none waiting for us. We hit the send/receive button, and wait gleefully hoping to find/not find email. Then we get a message that it was unable to connect to the server. That catches us by surprise, maybe we think its an aberration, so we click the button again. <br />
<br />
That second click does what? It allows us to check to see if it was a hiccup, a momentary failure, or perhaps a sign of a long term issue. I've had this happen from time to time on web pages I may visit frequently. A forum for a football team may load very fast during the week, but on game day as people are checking up on their team, it slows to a crawl, and a dependency like a style sheet, or images fails to download due to the sudden hit to bandwidth serving the multitude of requests at one time. It might even take minutes before you get that white page with some structure, and no formatting. Do we immediately think, wow that's random, this forum is really bugged? But I know from experience, this isn't a fault of the software itself, at least as far as I can tell, but instead it is a function of a high load on a system that may not be able to keep up with a sudden increase in demand.<br />
<br />
As testers, simply finding and reporting bugs is wholly insufficient to communicate to the developer the nature and scope of the fault we've encountered. In the case of the forum software, a subsequent refresh might fix the page, and it may load fine for several hours thereafter, unable to have the issue reproduced. Whatever the issue is, we must dig, and see if we can prune down the steps that we followed. We can try to see if the bug happens if we hit another location, try a different path through the software, or perhaps try a different role or persona. The point here is it is our job as testers to imagine how this bug could have occurred. What would your tester instincts tell you to go to prove and find this error so it could be fixed? Do you have the answer?<br />
<br />
Hold that thought, because I am going to revisit this question later in the week. For now, just remember that just because we can't see the pattern for a bug, doesn't mean there isn't one, and as testers in particular, our use of language should be careful so as to not mislead the public, our developers, our clients, or our managers.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-1404277397803196632011-09-20T22:37:00.000-04:002011-09-20T22:37:20.376-04:00Diary of a Soccer Coach: Week 3 and First Game!I'm a bit behind on blogging due to duties last week, but I'll catch up by throwing the third week of practice alongside the first game. In our league the third week of practice heralds two things, first the last practice before our first game, and the arrival of our team rosters and uniforms. On this particular day, the 'head coach' of the league, who has been helping out with our Kindergartners had to distribute the uniforms to all the different divisions. This left me alone to do a lot of work with the kids on my own.<br />
<br />
This worked out fine, and as I always try to keep the kids moving it worked out great. The Third practice is where we honed in on basic shooting skills. For most introductory soccer players, the more advanced steps are not always easy to pass on. At this practice I focused in on keeping their eye on the ball and following through as they shot. Also, as with most practices I got and kept them moving as much as possible.<br />
<br />
I started by having them stand next to the ball and shoot it stationary. After each player had tried this a few times, I had them try shooting the ball by first running up on the ball and then kicking it into the goal. Afterwards, I made it more difficult by having the players dribble the ball and then kick it into the goal.<br />
<br />
On many development projects I've seen a similar step by step building up to completion. A feature might start out very simplistic, or it may seem that way so we start by taking our first shot at it, just as my players might in their third practice. Sometimes we may not understand some of the nuance to a requirement. It may appear simple, just like striking that ball, but there are intricacies and un revealed flavoring that needs added for the code to really pull off what is intended. So as a team maybe you work up to these features, adding a little more speed, a bit more control, and higher accuracy in its calculations.<br />
<br />
I've found similar patterns in testing. The first time through, you may just be poking around in an exploration of the application under test. You may not have a full grasp of the features, how to activate or use them, or the intent, but you build a bit of confidence and then take another test run at the software. Then you might discover that this type of software is documented to have a particular susceptibility to one kind of fault, and begin tailoring your exploratory testing to hit those weaknesses.<br />
<br />
<br />
The first game of a soccer season is always exciting. Its the first time the kids are in their new uniforms, and you just never know how much the kids have absorbed from the limited practices you've had thus far. Each year is a little bit different. One year, one team may have a very good grasp of the game and create a lot of goals in that first game. Others might find it difficult to juggle defending the approaching ball, redirecting it to the goal they are attacking, or they may even get a little winded as they aren't used to moving so much at one time.<br />
<br />
The first year I coached, an older coach told me, "You'll see the most improvement between the Second and Third games." I wasn't so sure how to take that, but later I realized what he meant. Suffice it to say, many kids may not listen early in the practice. Until they see how they can apply it in a game situation, they just may not realize the advice you are giving them. I've seen this happen on development teams too. A tester might make a suggestion about how to improve a process or function within the application, and might be ignored, because its simply not their job, or because the developer is too much ' in the zone' to stop and see what is being said. There might even be, as is common in our first game of soccer, a lot of stops and starts as you build to a sustainable pace for development.<br />
<br />
Bottom line though, remember it's just the first game. A lot can change over the course of time on a project. Change is inevitable in many projects, and how we handle and respond to it sets a strong light on our teams and how we cope with that change.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-49081505636988089952011-09-12T21:59:00.000-04:002011-09-12T21:59:17.737-04:00Nature Vs Nurture: Do we train the tester out of our Kids?While working through a serious of tests on our automation framework today, a thought came to my mind. Do we train our kids to lose the very attributes that could make them a great tester? Do we risk killing their curiosity, or train them to accept what is told, because that's how our schools are run? Psychologists and scientists have argued nature versus nurture for a long time now, but I never really framed it in this way before.<br />
<br />
We have two young children, and I've had the privilege of watching my first born grow into a smart young boy. Now our almost two year old daughter is starting to pass 'little milestones' hand over fist. I remarked to my wife today that she looked like she had grown three inches since breakfast. Then later this evening while putting her to bed, my eyes did another visual inspection of her height; this time against something I knew was constant: The height of the bed rail for her baby crib. <br />
<br />
Earlier this week, we had to take down the pack and play yard because she had discovered a way to easily climb out of it. Plus we knew she had grown to a size that had become too big for it anyways. So we knew she was getting bigger, growing as all kids inevitably do. Then tonight was the kicker, I saw her gymnast style on an uneven parallel bars nearly pick herself up to climb over her crib's bed rail. Then switching to a new tactic, she climbed up one of the spindles of the crib, one side of one foot on one side, and the other on the opposite. She was climbing as I've imagined or seen climbers on TV working an ascent on many a wall faces of a mountain, before slinging her leg over the rail, just as I reached out and caught her in shock and awe of all I have seen.<br />
<br />
I love my little girl, she's been a blessing even when she was born and admitted to the Neo-natal Intensive Care Unit (NICU), but she continues to amaze me. Born a couple of weeks early, you'd never know it to look at her now. She's a runner, a climber, a ball player, and a wrestler. Not to mention she sometimes likes to practice tackling her much bigger brother from behind. She's my little explorer, my future Venture Girl, and I wouldn't trade that for the world. Both of my children are special, and highly intelligent, but lately as I grow as a tester and parent to her and to him, my mind ponders how best to raise her.<br />
<br />
The natural inclination of a parent is to want to protect, and keep their little ones safe from as many dangers as possible. Yet we as parents know the perception of our provided security is is not complete, as much as we may want, we cannot protect them from every possible hurt or injury. Just as testers realize that many of the security features we test, provide only a facade of protection, in this current day and age. <br />
<br />
The process of rearing children has me pondering this fact. Parents set up rules for their kids behavior and activities in and around their home. Some of these rules may seem unnecessary or excessive at times, but they provide a structure, a framework around which they can begin their learning experience. Then later, if you follow the public model, they go off to school, or other extra curricular activities that provide additional rules, and layers of precepts that try to mold the child into a particular form.<br />
<br />
Sometimes I wonder, just what are we trying to achieve? Are we stifling creativity by requiring them to paint within the lines? Are we killing their spirit by requiring them to sit like mindless zombie automatons. As I've watched my children in recent days I'm amazed at how many times they look fresh at some toy or item in our house, and find another unique way to interact or play with it. Many times this is fine and worth encouraging, other times it could be something they are doing is unsafe. Our urge is to jump, rescue, and shield them from this dangerous situation, but are we doing more harm than good?<br />
<br />
I wonder. More with our younger child than our oldest, I hope to hone and focus that curiosity. I'm very cautious about how I deal with her when in the course of exploring she is doing something that could bring harm to her. It's like walking a tight rope though. I want to encourage the curiosity, embrace the questions, and the goofy ideas that may come. I want to give her the freedom to learn without constraining her to the factory school of thought. <br />
<br />
We opted to keep our son home for his first year. Everything we'd read about child psychology suggested that boys may do better if not thrown into the structured environment of elementary school. Three years later we are still home schooling. What started off as an experiment to provide him room to grow paid high dividends. He has grown as he has learned, and it amazes me how much he can learn in a short span of time. Seeing his progress makes me sad at times, because I know that we may cover as much if not more than what a single day of school might cover, and yet he absorbs ever more. Heck this kid in second grade was upset that we hadn't taught him multiplication tables yet.<br />
<br />
Our youngest isn't yet of schooling age, but I already see her reaching out and testing the boundaries the environment provides. Some of them are provided by us her parents, and some of them are a structure of nature and design of the furniture, or artifacts in our home. Yet I'm more cognizant of the decisions we make to correct, or alert her to dangers in her environment. If you're reading this entry and pondering the same things, I'm curious as to your perspective. Does our rule, and school structure result in breeding out the curiosity, the intellectual spark that may draw a child to be a creator or investigator of the world around them?<br />
<br />
If like me, you have thought about this, and come to the conclusion that these factors do affect the development of the mind of a potential tester. Do they affect them for the worse, or the better? How can we improve them to harness that curiosity and prepare people to test the applications and services of tomorrow? I don't have the answer to these questions, but I may continue to ponder them for some time.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com4tag:blogger.com,1999:blog-2726520768982943835.post-23681473047497192152011-09-09T21:58:00.000-04:002011-09-09T21:58:08.756-04:00Off on the trail of testing, but wait, I forgot this one thingHow many times in life, do we surrender to the habitual nature of our human psyche? Do we capitulate and allow what seems to be an established routine, repetitive task, and in our minds we've got it down to an art, so why just flip a button, and cruise on through each step of the process, without giving much time to pause between steps to evaluate where we are going?<br />
<br />
News flash, tester or not, we all do this! In fact, I did it twice today, without even realizing it. Oh I wasn't 'testing' software at the time, but I allowed my inferred conviction of my own understanding of the early day's chores to not only lull me into a state of numbness, but introduced my own sort of performance speed bump that wasted hours of my time. <br />
<br />
What started out today, as an early morning jaunt to accomplish to simple chore, turned into an exercise to remind me why falling into the assumption and autopilot trap are so dangerous. It started simply enough, just two chores and then I'd be home for the rest of the day, became an afternoon of back tracking, acknowledging a flaw in my own understanding, correcting it, and then executing essentially the same process, but in a slightly varied way. It started with a trip to the court house. <br />
<br />
I was rather pleased with myself, because I thought I could complete two tasks in one visit. I might have even boasted internally, that I was a genius to take care of these two tasks at the same time. The first task, was to renew my license registration, at the Sherrif's office to bring my license plate current, and get the sticker to indicate to any law enforcement officer that might be checking my progress, that yes, I had paid my taxes and fees, and was not violating the registration laws for operating a vehicle in our state.<br />
<br />
The second task, is one that maybe no one else will have experienced, but it required a trip to the County Clerk's office. You see, as a member of our community at larger, I stepped up back in 2006 to serve the community as a Poll worker for a number of different election cycles within our county. I do this as a service, because to be honest, the rate they pay for thirteen hours of open poll service, almost two additional hours of setup and tear down, is not a pay rate I'd accept for any of the professional work I do. However, as a concerned member of my community, an Eagle Scout, and a person of faith, I value the integrity of elections, and believe that doing so is an important part in ensuring our elections are fair. In order to participate after being selected, I have to fill out, sign, and return a form to the clerk's office. It was this task that I desired to complete today, because I had delayed sending it in earlier, due to uncertainty with my current job situation, and did not want to commit if I felt in good conscience that I would not be able to serve. <br />
<br />
Those were the first two tasks of the day. I stepped out the door with the letter in hand, and my registration as well, and drove to the court house. I was pleased because I found a parking meter open within an easy walk, exited my car, added some time to the meter, and off I went, and that's when the first error dawned on me. I had my insurance statement, the registration, and the signed form for the Clerk, but a nagging thing in the back of mind then came into focus. 'Does the Sheriff's office take check cards, for payment?' Why I didn't ask this question before I left for the court house is unknown, but it proved to be the pivotal question, because they in fact did not take it. I asked the Sheriff's Clerks whether they accepted it as a form of payment, knowing already in my mind that the answer was probably no, and received the confirmation that, no, they only accepted, cash, check, or potentially a money order. I didn't have any of those options on my person, and truthfully I don't usually carry a check book with me unless I know I need it.<br />
<br />
Now after reflecting I realize I could have potentially, walked over to the post office, paid the fee for a money order and then did it that way, but I'd have no record, no good one in my register to verify when it was paid. So the first task of the day, I struck out. I went ahead down to the County Clerk's office, handed the form to one of the workers, asked if it was yes, (it was), and was happy that I had at least one task done. I would have to return home, find our only check book and return to complete the initial first part of the plan.<br />
<br />
Have you as a tester ever started working through a problem, you jumped to some assumptions, maybe you feel they are good ideas about how it in theory should work in your own mind's eye, and proceed to work through the process of massaging through the interface. Ever stop at some point to realize, you know, I wonder what would happen if I had done something differently that previous step, only to find that hitting the back button on your browser really isn't a good way to check this new test idea? It happens to the best of us. Sometimes a test that would make more sense to apply first, isn't, and has to be run again at the start of another iteration through the process. That can seem frustrating, but it is part of the learning experience we go through as testers.<br />
<br />
<br />
Well, this did not just happen to me once, it happened twice. See I was also looking for information about a repair on my car. I traveled to the mechanic's garage I have grown to trust, and proceeded to inquire about an estimate on how they might do it, how long it would take, and at what cost. Surprisingly, they told me they couldn't do this kind of repair. They had an idea of how the repair could be done, but there was something particular about my engine that required something that they lacked, something that did not give them confidence they could complete the repair in a timely fashion. <br />
<br />
What a bummer that was. However, I countered with a question. "Okay, if you can't perform the repair, as I understand it, then is there another shop who you might recommend to perform this fix? I know its not a critical issue on my car, but I would like to get this fixed just as soon as humanly possible." They gave me a name of another shop, and I then asked, if they thought simply calling them would be enough to get an estimate. They didn't really know, but suggested that maybe that would work, and it would save myself some money on gas driving out there unnecessarily. I liked that idea, and returned home to eat lunch with my family.<br />
<br />
After lunch, I began looking for the phone number of the shop, first in a few paper phone books, then turned to google, and superpages, but could not find it. I found other shops, but not this one. So later this afternoon, I hopped in my car, and drove out to where the shop was (having received directions earlier). In hindsight, I wish I had pressed for a contact number before I left the first shop, but I honestly didn't believe that finding it would be that much of a hassle. That was my mistake, and yet another lesson learned. <br />
<br />
Ever start testing a piece of software, and then at some point just stop because a question comes to the forefront, that you almost feel, man I wish I had asked that before I started? Sure, it happens, maybe more often than we like. We are creatures that learn and grow, and as Testers, we are many times going to develop ruts and habits. Try to break those habits from time to time, maybe you'll discover a new way to flex the software, to bend and contort it to find a brand new class of defects. <br />
<br />
Ultimately, we must try our best to avoid jumping to assumptions. Never assume you already know the answer, if you've never even asked the question. Never assume that the developer obviously must have done something a particular way if you've never had a conversation about it, and never assume that you've brought and used the right tool for a type of test. Now this does not mean that we can never make assumptions, if we do, we need to realize that our testing is based on certain assumptions, maybe its a platform they are running on, or a particular style of device, that may be an educated enough of a guess to allow us to proceed, but we should remember that they are fallible assumptions, and present that as part of our test story when the time comes.<br />
<br />
However, today's experience reminds me of something from earlier in my life. It's funny in a way, since college, my first rule of life has always been to avoid making assumptions. This was even before I considered, or even had a clue at all what it meant to test anything. Western's Rule #1: "Assume nothing, for when you assume, you are usually wrong!" At least that's how I wrote it as a freshman in college. Today I'd transform that rule to read more like this. "Make no assumption absent evidence, for assumptions are often based on illusions, that when that illusion is removed or proven false this can result in a great embarrassment to you in life." Honestly, it isn't that our assumptions are wrong, or that they might be based on inaccurate intelligence of the situation in our projects, it's the false confidence, it can breed, and the blindness it can bestow that limits our ability to test accurately and effectively that are the risk. It's the shock and awe that an illusion the team may have held as true, once removed can cloud judgment on the value in the product. <br />
<br />
To conclude, monitor yourself as you test from day to day. Check to see if you feel yourself itching to turn on the autopilot. Recognize it, take a step back, and find another way forward. Use the <a href="http://www.shino.de/2011/07/02/pomodoro-testing/">pomodoro technique</a> (Thank you Markus Gärtner), or some other focusing/de-focusing heuristic. Be skeptical of even your own best ideas relating to testing, and always strive for one more thing to learn as you flex your testing muscles. Try testing with a partner (Pair testing), so that you can keep each other from falling into a rut, or team with a developer and show him what you are doing to test the software. Whatever it is you use to affect how you keep from falling into an zombie like coma while testing, do it as often as necessary, you'll thank yourself later.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-24673878857191236932011-09-08T00:58:00.000-04:002011-09-07T22:30:20.768-04:00Oh that Bug? Yeah it happens all the time, don't worry about it.Tell me if you've heard this one before. A User calls into a help desk saying, hey when I go to do X with my software, instead of doing X, something else totally unexpected happens? And at some time in the past a root cause analysis is done on this, and they discover what has happened. The user has done something to the software, perhaps they've configured some optional setting that isn't a part of the normal settings, or maybe an incompatibility with another piece of hardware or software results in it being unable to perform the problem. <br />
<br />
Normally you would expect the team to find, and smash this bug, and fix the defect right? Well what if that wasn't what they wanted? Or what if it was a feature they wanted to leave as it was. Maybe its a link to some documentation that moved on a website. It might be easy to fix, but getting a patch might be more expensive than simply telling the user another way to get that data? This situation comes to mind as I viewed today's Wizard of Id Comic:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="http://www.johnhartstudios.com/wizardofid/2011/09/wednesday-september-7-2011.php"><img border="0" height="150" src="http://www.johnhartstudios.com/wizardofid/strips/2011/september/wiz110907dcc.jpg" width="480" /></a></div><br />
If you've been on any software team long enough, odds are you'll eventually come across a defect or bug that you see as a potential loss of value in the product. After discussion with the team, that bug may be marked as deferred, or left as designed by the developer, and not handled as it is slowly forgotten in the code base. There are times when cosmetic changes, a font size, a color, may not make much difference to the overall user experience, but what if this deferred bug turns out to be something more, more insidious? What if it could be the bug that begins to build to a buffer overflow vulnerability that could result in your system being compromised and hacked? <br />
<br />
As testers, its important that we maintain objectivity as we are testing. Sometimes, the development team may not all see eye to eye on what is of value to change for the customer, but we must be every cautious when a somewhat mundane bug is deferred. Deferred bugs may never get fixed, and as they get left in their unfixed state. Sometimes this may be fine, and something we have to accept as we strive to produce the most value for our clients, but we must always be careful that the thing we are putting off could be something serious that could put our customer, our client data, or even our own companies at serious risk.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com2tag:blogger.com,1999:blog-2726520768982943835.post-56467962125882468902011-09-07T20:22:00.000-04:002011-09-07T20:22:03.606-04:00Diary of a Soccer Coach: Week 2 - Inter-team Communication, Noise, and Dealing with the UnexpectedEven before I woke up the day of our first Soccer practice I found myself glued to the weather channel. After seeing the massive thunderstorm and rainfall affecting so many College Football games during the past weekend, and knowing that the remnants of Tropical Storm Lee were predicted to stall out and take time to clear out I was growing in concern. Some of the weather maps predicted as many as five inches or more of rain locally. Flash Flood Watches had been issued by the National Weather Service, and there appeared a high probability of rain in the forecast for our game.<br />
<br />
Now normally, a little rainfall would not have introduced a bit of concern about whether to have our Soccer practice or not. Typically the only thing that has ever affected that was thunder storms, or extreme bitter wind chill conditions. In truth only once in the three previous years I have coached has any of these conditions even been approached. So naturally I was a little concerned here. The place where we practice is in a low lying area, that lies in a flood plain area that has flooded in recent memory. Because of this it was important to know what the weather conditions might be for the weeks practice.<br />
<br />
As it happened though, by mid day the rain had mostly moved on, and though still overcast, the rain was gone, and it was modestly cooler, although humidity was still high for our practice. In software teams, how often do we plan for contingencies like these that could disrupt development, or delay deployment? Sometimes the unexpected may happen. A freak ice storm could knock out power to your data center. A Nor'easter could barrel up the coast and cause localized flooding around the facility you were supposed to test remotely against. What if the shipment carrying a key part for your data center crashes forcing you to wait an additional two weeks for a customized component to be fabricated? <br />
<br />
These are natural impediments that could affect your development process. As a Tester, a network issue could deprive you of the ability to test on your virtual laboratory. It could result in a mistaken deployment that results in a dirty configuration unlike the clean environment you are expecting and then it can cause problems as you start encountering bugs, and half to track them down. There are perhaps a hundred or more situations that could result in what I call a noise condition within the team. Some of them are internal, within the mind of the tester or team member, some of them are virtual, on the box or serve where testing is to occur, and some could be physical or natural noise that can interfere with your ability to test effectively.<br />
<br />
Some of these impediments can be considered in your deployment process, and perhaps avoided or at least minimize the effect it has upon your testing. However even the most rigorously documented process is only as good as the people implementing it. As we are human beings, and prone to make errors, you can never avoid all of these. A missed deployment to a test environment could indicate a whole or missed script in the deployment process. Why did this script get missed? The manager may ask this question, but I find the same thing can happen in the soccer field as well.<br />
<br />
For the second week of Soccer, I like to hone in on two key skills. The first is communication. Whether the players realize it or not, learning to talk to their team mates on the field make a big difference in how they will play down the line. For Soccer the simplest form of conversation is the pass. A simple plant of the off foot toward the targeted team mate, and then following through with the passing foot, ankle locked to connect with the center of the ball right about the inside of the ball of the foot. If done right, the ball will travel straight and follow the same line of travel that your foot was pointing. I demonstrate this once or twice to our players, who I've saved the trouble of having to find a team mate to pass to, by pairing them up, and have them start with this basic pass.<br />
<br />
So I watch the players as they work on their first few passes. Some kids pick this up very quickly, some quickly get frustrated. Younger, smaller kids may not be able to kick the ball as far or hard, or might be more focused on kicking the ball, than the technique of the inside pass. I watch for moments like these, and let the player try a few times before stepping into correct them. Sometimes they figure it out by trial and error, but sometimes they keep doing it the same incorrect way, and I can see it could cause a bad habit to form.<br />
<br />
At this point as the coach, I step in. I remind the kids to plant their foot toe pointing towards their team mate, to lock the ankle with their toe slightly raised towards their shin and connect with a straight swing of their foot connecting just above the mid point of the ball with the ball of their foot. A couple more passes, and a little more encouragement may be required. Keep your eye on the ball (once they get the skill down this may not be as important, but early in the drill process a it may help if the player sees as the perform the task.) After a few more times if one or another player are having difficulty, then I may step in and demonstrate again, showing what to do, and then emphasizing the difference in how I performed the pass versus how they are doing it. <br />
<br />
One of the common early problems I notice is a player trying to kick the ball with the toe of their shoe. Kids seem to think they can get more power passing this way, but it really leads to an unpredictable movement of the ball, especially for the younger inexperienced player. This isn't something you want the kids to do early in their development. The toe is a very small area on the foot, and many shoes are 'V' or 'U' shaped meaning that if you miss the exact center of the ball and shoe you may hit it more to the right or left and the ball will go out in the corresponding direction. I may even have to demonstrate how wrong this is, so the kids can see the difference, but after doing so I get them doing the passing correctly, back and forth, and may float between pairs, repeating this process as need be. <br />
<br />
As more of the players seem to get a hang of this simple pass, I will then offer them the option to try the same style of pass with their normal off foot (typically the left), and then give them a demonstration of a more advanced pass. This time using the outside of the foot just behind the joint of the littlest toe and driving the foot to the side, you can actually pass to the side. All the while I continue stressing, getting the team mates attention, pointing the foot in the proper direction and following through on the kick.<br />
<br />
This may seem like a very repetitive and boring process, and for some of the older kids it might. It doesn't take but a few minutes before I begin to see the first side effects of noise on the practice field. There are other things to get the kids attention. Someone brought their dog with them, a butterfly might fly onto the field drawing attention from the drill. Kids on another field might be doing a slightly different drill and that catches the kids attention. We have the same kinds of noise in our software teams as we communicate.<br />
<br />
A HVAC unit could be louder than normal, a team member may be mulling over some problem they've encountered as we're describing a test we just ran, and the flaw we think it uncovered. Whatever the noise may be, that noise can impede our ability to communicate effectively the point we are trying to make. So how can we avoid noise? Sometimes it may involve asking another team, that is goofing off in the cube next to you, to keep it down as their voices are starting to carry. Maybe it involves interrupting another conversation that has your team mates attention, when what you need to say is more vital. Sometimes we have to wait for the noise to pass, such as when a train goes by blaring its horn and drowning out almost everything else you might hear. Assuring we can communicate our message is key in any context.<br />
<br />
Now these example are good if it is an audible noise, what if it is an internal noise? This is where noticing nonverbal cues is important. If your team mate is listening, but focused on reading something on a wall, or their computer screen, it may indicate their attention or focus is elsewhere, that could be internal noise. Another example, is if someone has a habit of doing something with their hands. It could be something as simple as scratching the back of their hand, playing with a toy of some kind, or twirling of a pen in their fingers. All of these are nonverbal cues that your team mate try as they might, may not be committed to the conversation. <br />
<br />
So how can we avoid these things? In soccer, when passing I encourage my player to start the passing conversation by calling their team mates name. Then as the ability to pass becomes second nature I instruct them to keep their eyes ahead of them towards where they are passing the ball. The other player, I instruct to keep their eye looking back towards the ball as often as they can while moving around the field, so they are prepared to receive and complete that transmission of the ball across the grass of the field to their feet. Ever wonder why eye contact is often stressed in verbal communication situations? If our eyes are turned away from the team mate who is trying to communicate with us, then also our ears may be turned away and reduce the optimal ability to hear what they are saying. That is not to say that we should stare a hole into the head of the team mate we are trying to communicate with, but we should make enough eye contact to show that we value what they have to say.<br />
<br />
What do you do when your comrade's attention is wandering, or they are busy multitasking and can't seem to keep up with the conversation? Ever been in a meeting, in person or virtual were someone is being told something and then the speaker follows with what should be a typical yes response question? "Does that make sense, John?" The initial reaction may be for the person to say Yes, but what if their attention had drifted, they might realize they didn't fully absorb the importance of what was being translated to them, and the cue, is John's chance to say, "No, I was having trouble following what you are saying, can you please repeat that?" During any conversation we can show our continued attention, not just by eye contact by other nonverbal and verbal cues. Nodding of our head, a quiet yeah, or aha can indicate we are following the chain of the conversation well.<br />
<br />
<br />
There is one more nonverbal cue I look for when talking with a team mate. That's when their hands come up to their mouth. You've probably seen someone at some point do this. You mention something, and they may begin to cover their mouth with one or more fingers, indicating subconsciously that they are trying to parse together a question or response to what is said, but those fingers indicate that a lot of thinking is going on. This is a telltale stop sign. If you see a team mate do this, then it is highly likely that they have a different point of view, or something to contribute to the conversation. There are other mannerisms that can indicate this desire to contribute back to a conversation. Someone looks like they are trying to reach out and give you a subtle stop sign, is another.<br />
<br />
There are so many things we may communicate through nonverbal cues. How often do we ignore these cues and keep on rambling through our point, wanting to reach its conclusion without allowing our colleagues to collaborate and fully commit to the conversation? Regardless of your place on the team, be it a tester, developer, manager, or team player. Communication is critical. Without it, the noise may increase, and the ball we are trying to pass to our team mate may end up in the wrong cue, or intercepted by the competition and then we are back tracking trying to recover, and catch back up to what we've lost. <br />
<br />
So as you go back to your work spaces, consider these thoughts: Where in your environment does audible noise interfere with communication? What can you do to work around it? What can you do to react better to the nonverbal cues of your colleagues? How can you make sure they are able to contribute to the conversation, and thus collaborate towards a better end?DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-48887047873693377282011-08-30T23:44:00.001-04:002011-08-30T23:58:35.280-04:00Diary of a Soccer Coach: Week 1 - Getting the Players AttentionAs I enter my fourth season as a Soccer coach, I can't help but reflect back on the previous three years. There have been a lot of kids I've had the opportunity to work with. The prior three years I coached the 'instructional division' as it was first called to me. In essence I've had the honor of coaching up kids ranging from Pre-K through Kindergarten age. I've coached the same level for three years now, and again this year I am stepping up to do the same. <br />
<div><br />
</div><div>In some ways its hard to be a coach to the youngest in the league. Some of them are already quite athletic, but lacking in coordination, some are squeamish about falling down, or running into someone else. Some don't quite know what to expect from this their first athletic endeavor. It isn't always easy on the coach either. Sometimes we just barely get a season to get to know the kids before they begin to bloom and are ready to play up to the next level. Some only get the one year, then are moved up with the other kids their age. In essence each season is like starting over fresh. There are always a few faces you remember from the previous years, and you may recognize the growing boys and girls practicing on a field not too far from your own teams, but your focus is now on the next group.</div><div><br />
</div><div>For companies like my current employer, where professionals might work on multiple projects, or bounce between roles and wear different hats within or between teams as the needs become evident, to a team leader, it can seem quite a challenge to start from scratch with a team. It can also be a challenge to get the new people involved and fully engaged and invested in the project. </div><div><br />
</div><div>For the first session of the Soccer season as with any team, it is imperative to set the ground right. The first practice always starts with a quick introduction and brief overview of the most basic rules of soccer. Then we quickly transition into a couple of warm up exercises. They aren't typically long exercises, but are designed to get the players moving, loosened up, and in the mind of being at practice. Warm-up exercises could be great for team building, or for helping to transition a new team member into being part of the esprit de corps. Having transitioned to several projects in my career, I feel that many of these warm up exercises did a lot to help smooth my transition into the team.</div><div><br />
</div><div>Of course, practice does not end with these warm-ups, nor should it be the end of the team building process either. As <a href="http://mkl-testhead.blogspot.com/">Michael Larsen</a> (@MKLTesthead) so eloquently put in describing the stages of team development in his emerging topics talk on teams and the EDGE method employed by the Boy Scouts of America - The team is still forming, here, it hasn't even begun to play soccer, nor has the new team or hire really progressed to a state of being fully in tune with the team's software development process.</div><div><br />
</div><div>So the next step of practice, is the first of many drills. For soccer in the introductory division, the most basic of drills typically involves dribbling, around a pair or square of cones, emphasizing technique for controlling the motion, the tempo of the dribble. For software teams, tempo, and pace are something that teams struggle to achieve, and hold once they have it. The same is true for Soccer, and our instructional division is no different. The secret you see to these kids, and I'm betting many other youth organizations, is to keep the meeting or practice as active as possible. </div><div><br />
</div><div>So we run them through a drill for five, maybe ten minutes, and then may start another. This gets the players used to moving around, and focusing on one type of task or another. However, at some point the kids need a break, so we give them a bit of time to run to the restroom, and get a drink before having our mid practice huddle. More on these huddles in a later entry, but suffice it to say, the kids almost without fail come back refreshed, and ready to do more, but it is still the First practice, and i its important to temper our expectations with that in mind. </div><div><br />
</div><div>With software teams, the same is true. Each member needs down time to assimilate lessons learned, to rest from exertion of the mind, and for testers, to de-focus or refocus on whatever the next task may be. Without this crucial down time, tasking begins to run right into the other so quickly that one may begin to lose sight of where one testing objective ends and the next begins. It also presents a danger, because as creatures of habit establishing routines that are unhealthy will inadvertently result in inattention blindness, and as tempo and feel for how the project flows is learned, the risk increases that things move at a certain pace simply because they've always moved at that pace, whether the process or project are really at a achieving and sustainin velocity or not.</div><div><br />
</div><div>For my players in their first practice of the season, I see them in the scrimmage we typically run at the end of each practice. They chase the ball without any rhyme or reason, with out any strategy or objectivity. They believe the goal is to get the ball, and that to do that they must run to the ball. There's just one problem. someone else has that ball, and is accelerating in a different direction. so each player changes their angle to try and chase the dribbler down often from behind. </div><div><br />
</div><div>At times it can look like a swarm of honey bees, buzzing to one corner of the field, then changing course and quickly buzzing to another all around and trying to catch the ball each time, and inevitably these kids will trip, fall, or bump into another. We haven't yet taught these kids how to move laterally, or even backwards when necessary, nor have we shown them the strategy of picking a place between where the ball might go to cut it off, rather than trying to chase it as if chained to the player with the ball like a rail car. Inevitably the first practice always looks like this. The older kids may not run into this scenario, as many have years of soccer under them by the time the season has started again, but for these new boots to the field of play, or to their team in the software world, they're a blank slate practically and in need of mentoring and guidance. </div><div><br />
</div><div>The question you have to ask, is how can I influence them for the better, and help them see their role as more than just chasing the ball, as chasing this bug or that bug, when they may be leaving square feet of the field uncovered that could give them an advantage? For me, it is important to remember that this is the beginning of a long journey we take together, and for better or worse we are here as part of a team striving to create that which our client needs. However, even us experienced software professionals can occasionally trip over someone else's toes and fall flat if we aren't careful. So trod carefully in those first days on a team, until you have a good picture of the terrain you must climb.</div>DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com2tag:blogger.com,1999:blog-2726520768982943835.post-19547375026022714882011-08-29T22:27:00.001-04:002011-08-30T06:50:14.388-04:00Today, I Choose YellowSometimes the smallest things can be inspiration for test ideas. Sometimes they make you sit back and pause and consider. After enjoying a nice lunch with my family, my wife presented me with a colored page of Dora the Explorer, with my daughter's name and the date on it. I took it to work and proudly displayed it on part of my cube wall. I examined the picture and considered how she had gone to work coloring it. Now she's not even two years old yet, so staying within the lines is not something I'd expect from a soon to be two year old. <br />
<br />
However, two things jumped out at me. For this page she had chosen a single color: Yellow. It wasn't any particularly stark or bright shade of yellow, just a plain almost mustard color. So then I began to think, why Yellow? Dora isn't blonde, she has brown hair. Her shirt is usually pink, and she wears orange pants. Her backpack is a shade of magenta or a light lavender. So there isn't a lot of Yellow there to work with as inspiration as far as I could tell. <br />
<br />
So when I ponder why she selected those colors, as a Tester, the initial thought was that she had a reason for choosing that color. I wasn't present when the picture was colored, but when I think back to the beginning, of other pictures she has colored, then the reason becomes clear. My little one you see, isn't logical yet, her mind not quite fully developed. She's bright and smart, charming even in her own way, but she's not even two yet. More than likely she chose the color yellow because that was the first crayon her hands came around, and then, instead of changing colors for different parts of the picture, she continued until her mind felt she had colored as much as she dared too, and then moved onto something else.<br />
<br />
How many of us as testers and software professionals do that? How easily do we grab a hold of an emotion or an idea in any particular day and then view and work the entire day as if through that color whatever hue it may be? Does that emotion and idea affect everything we do that day? Do we let it bleed into our code, our testing, or our conversations with others? When they look at you that morning, will they see the color yellow?<br />
<br />
I learned from a book "<a href="http://www.amazon.com/Anger-Choice-Dr-Tim-LaHaye/dp/0310242835/ref=sr_1_1?ie=UTF8&qid=1314664595&sr=8-1">Anger is a choice</a>", that many emotions, anger in particular, are neither negative or positive. It is how we react when those emotions come up that determines how we are viewed. How many of us as Test or Software Professionals have encountered something, maybe it was minor at some point and we let it go, but it nagged at us. As much as we tried to ignore, or deny this one thing, there it was the color of red, that our mind didn't want to ignore. At some point, do we then find our focus so deeply on the color that we begin to see everything as if it was tinted that shade?<br />
<br />
There's a danger here for Testers. When our reaction and emotional make up begins to bleed through into our work, into our analysis and observation of systems under test, there is the possibility that it could lead us astray. It could cause us to miss something that we'd see if our attention were not partially focused somewhere else. It could cause us to react reflexively to something someone said that was intended in all good fun, or perhaps to help us with something with which we are grappling.<br />
<br />
It's important I think to remember that Testers, like the clients for which we test software, are emotional beings as well. Maybe our clients will see red when this one really annoying malformed feature crops up again and again. Maybe they've ignored it thirty times through the application, then one day, when their emotional system is already spent from something that happened on the way to the office, or at home, and bang that one annoyance becomes the lit fuse that sets them off. Ever know any people like that? Ever had a client like that?<br />
<br />
Now let's turn this on its head. Maybe there's some issue, some flaw, that the team doesn't really classify as a fault in the software, some piece by which something is not as correct as it could be, but in your mind you, or the team in general have decided that's not important. It's a minor, non-critical issue, and why would a user care if the time and date are displayed in YYYY:MM:DD form vs DD:MM:YYYY form? Maybe on a normal day they won't, but do we ever consider how emotion on the part of the client, may play to what bugs they see as critical or important?<br />
<br />
Furthermore, do we as Testers always try to enter into our testing sessions with as blank an emotional slate as possible? Do we try to be like the Vulcan's of Star Trek utterly devoid of emotion, and creatures of pure logic for the duration of the test run? If so, why do we do this? Do we think a user will view the software with such a dispassionate view of how it works? <br />
<br />
If, like me, you have ever had the chance to work the support lines for a company, then you know what I mean when I say that the customer has a personal way of dealing with his or her own emotions. They may be struggling to get the software to perform, maybe they've had training and have used it successfully in the past, but one little nook, one little detail escapes them as they are partially distracted by some emotional atrocity they have endured this week. <br />
<br />
<a href="http://www.developsense.com/blog/">Michael Bolton</a>, no not the singer, or guy from office space, gave a keynote talk at the 2011 Conference for Association of Software Testing. Michael talked at length about the history of testing, and at one point described how decisions on quality are "both political and emotional." Another tester there, <a href="http://www.thebraidytester.com/">Michael Hunter</a> (@humbugreality)<span class="Apple-style-span" style="color: #444444; font-family: 'Helvetica Neue', Arial, Helvetica, 'Liberation Sans', FreeSans, sans-serif; font-size: medium;"><span class="Apple-style-span" style="line-height: 22px;"><b> </b></span></span>in one of the emerging topics or lightning talks also talked about the 'emotional tester'. I was able to follow some of these talks online, and I can't give enough kudos to the organizers of CAST and the hard working volunteers that worked to have those talks and the keynotes available online for those of us who were unable to attend.<br />
<br />
How much role does emotion really play in our craft? Do we allow it to simply be, to drive our assessment, to harness, or control it? Do we inject it intentionally to see how it may color our opinion of an interface, or to see how our perception changes when our mood has changed? Those are questions I'll probably ponder for a while. It's funny how something as simple as a colored picture of a cartoon character by my youngest child could bring me back to contemplate these ideas in testing, but then maybe these ideas have been buzzing in my head since I first heard them, just like the color yellow.<br />
<br />
<br />
<br />
(Edit: Thanks to Justin Hunter (@hexawise) for remembering it was Michael Hunter who gave the emerging talk on "Emotional Testing""DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com1tag:blogger.com,1999:blog-2726520768982943835.post-28785498971430031292011-06-22T23:11:00.001-04:002011-06-23T08:41:52.380-04:00So what's your priority?In the fast paced world of software development, sooner or later you or someone on your team will find a situation where they are faced with multiple tasks, and competing priorities. So what does a person do when an urgent list of changes to a software under development is received? Then what do you do when the expected time to deliver these features appears to be something rather aggressive? <br />
<br />
Let's take a step back for a second to discuss the requirement process. Often requirements are not as detailed as necessary to begin to design or codify. Sometimes requirements may be in a list form, they may be rather obvious, but more likely they are obscure, cryptic, or down right confusing. Because of this, very often there is a necessary feedback loop between the team and the client just trying to make sure they understand the change, and the impact it may have on the system. <br />
<br />
This back and forth, haggling over specifics, and gaining enough understanding to model the process necessary to be implemented can be very time consuming, and often the hardest part of the software process. Customers do not always have the computer knowledge to give sufficient feedback about what they write, and as a developer it is our jobs to not just verify that we are building the software correctly, but even more critical, we must validate that which are building actually meets the needs expressed in the requirements. <br />
<br />
Often the understanding of a requirement will morph over time, but for simplicity's sake let's assume for a moment that the haggling over the requirements has already taken place. So your team is called into a rushed meeting to discuss a list of changes. The manager describes the ten items that need to be implemented, and expresses the hope that it can be done in a short amount of time. For sake of argument lets say its a week. <br />
<br />
Immediately the issue of time and priority may come to the forefront. If there are ten changes to a system, ten changes that do not necessarily 'interact' with another or depend on another, then how do you prioritize them? Depending upon what model your process is based upon the answer could vary. It may be the Project Manager who sets these priorities, or in a more agile environment it could be the customer, or the person designated as the 'product owner' within the team. <br />
<br />
So the team after a little debate comes to the decision, much as they might like to pick and choose which pieces to do in what order, they need one more layer of feedback about the requirements, what is the order of priority for these changes. Given that the team is taking some measure of risk trying to rush these ten features into the system in a week's time, they believe that this risk would be easier to stomach if they could impose some logical order to implement the features, so that if the time should prove insufficient, they could at least be sure to have the most critical features done and deployed.<br />
<br />
So what do you think the response would be? Hopefully you'll get a list of points itemized from one to ten giving a clear order of development and importance to the customer. That would be the ideal, but what if the list comes back with six of the ten items listed as priority number one, and the other four as priority number two? I posed this question on Twitter.<br />
<br />
I received a few different responses two of the more interesting ones were from Stephan (@S_2K)<a href="http://twitter.com/#!/S_2K/status/83621254963277825"> "Everything is most important. <sigh>"</sigh></a> <span class="Apple-style-span" style="font-family: inherit;">Morgan <span class="Apple-style-span" style="line-height: 17px;"><span class="tweet-full-name" style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">Ahlström (@Morgsterious) replied with the understanding words <a href="http://twitter.com/#!/S_2K/status/83621254963277825">"That noting is REALLY important."</a> These were very much my same sentiments when I pondered this question. If someone cannot distinguish a hierarchy between two things, let alone ten things, how then can you know which items are of most importance and tackle those ares of the software first? Even if you have multiple team members, it is only natural that some components may still have higher precedence. </span></span></span><br />
<span class="Apple-style-span" style="font-family: inherit;"><span class="Apple-style-span" style="line-height: 17px;"><span class="tweet-full-name" style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><br />
</span></span></span><br />
<span class="Apple-style-span" style="font-family: inherit;"><span class="Apple-style-span" style="line-height: 17px;"><span class="tweet-full-name" style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">So what do you do when you have this sort of situation? My hope would be that enough time would exist to push back for further definition of the structure of the requirements. However, given that the customer might not be as eager to be asked the question a second time, or might not be as readily available, the team is then left to its own deductive skills to try and figure out the proper order to do them in. This is where knowing a bit about potential dependencies within the requirements could lend aid into determining an order of construction. </span></span></span><br />
<span class="Apple-style-span" style="font-family: inherit;"><span class="Apple-style-span" style="line-height: 17px;"><span class="tweet-full-name" style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><br />
</span></span></span><br />
<span class="Apple-style-span" style="line-height: 17px;">Nevertheless, it is still highly probable that there can be no clear prioritization gleamed from this answer. With time being ever more tight then the team is left to its wits, and knowledge of prior conversations to try and 'best guess' the order they should tackle the changes. In the end the team might complete all ten, or they might completely only a handful of the tasks. In the later situation, the customer might then push back and ask why they completed one task but not another. The team then can remember that it didn't get the accurate feedback it requested on prioritization, and could provide this as reasoning for the order they had chosen.</span><br />
<span class="Apple-style-span" style="line-height: 17px;"><br />
</span><br />
<span class="Apple-style-span" style="line-height: 17px;">The customer might not like that sort of response, but sincerely in the absence of information, it becomes better to complete at least some of the work, even if you do not know how much, or what order it should be done in. This may not be an ideal solution, but it should be evident from this how important communication with the customer must be to ensure accurate building of the software and the timeliness and sequence of delivery.</span>DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-39694202288850601242011-06-21T21:55:00.000-04:002011-06-21T21:55:43.019-04:00Some thoughts about Version ControlIn my time as a software professional I have seen many systems for maintaining and tracking changes to code in progress. Whether this is the archaic and I hope no one ever really uses this on a real project practice of passing along zip files of changed code with comment mark-up to indicate where changes were made, or to full featured source control solutions, like subversion, team foundation server, or git, every project at some point or another will have to make a decision on how to handle these files.<br />
<br />
I could spend time dealing with the importance of choosing any source control solution, but that's not what this post is about. Instead I'd like to take a moment to describe some of my experience with source control solutions. <br />
<br />
In general there are two sorts of source control mechanisms, one is like that used with Microsoft's Team Foundation system (and before it Visual Source safe an incremental check out/check in system, and the other is a check out the code base, and commit/update to get/add the latest changes to the current working version. There are pluses and minuses to both schemes. The incremental check out/check in system really feels like a pull system. While you can download the latest tree from a base directory, to make changes, and have them be saved with out the IDE screaming about read only or protected files, you have to make a point to check out in essence pulling on TFS to make the file ready for editing. Team Foundation Server (TFS) allows you to do this in an exclusive mode, essentially locking out any other attempts to check out and change the file by other users, or a less prohibitive more shared lock. <br />
<br />
I've only had the privilege of working on a single project with TFS thus far, and that project relies on the exclusive kind of check out system most of the time. This means that if a developer needs to make a small change to a file that you are working on, he will have to communicate that need to you, so that you can then do a hand shake of releasing the locked edit, so that the change can be checked in. It can be a real pain at times when you want to try a quick fix and the IDE prevents you at times from trying something simply because it's marked as 'read-only'. It provides a benefit in that it limits the number of files you might accidentally touch which can be a blessing at times. It also means that anyone trying to check in another fix may not be able to do so while it is held checked out, and as has happened to me a few times, sometimes files in a project may get checked out by the IDE automatically even if you did not intend it so.<br />
<br />
The other form of version control management is the style for which Subversion has become known. It is in essence a more push style of architecture, where by, you can check out an entire source control tree (much like getting latest for any particular folder), and then have significant freedom to make changes, erase a file and refresh/revert it from the last commit, or even make a series of changes and check them in one after another. The draw back of this system, is that because anyone can edit, there is the possibility of files getting more and more out of sync from the base or trunk as it is called in subversion. This can present problems when during a commit a major conflict arises trying to reconcile differences between the two files. In my experience though, if used judiciously and kept up to date and committed as often as necessary, these conflicts can be minimized fairly easily. Subversion also has the bonus in that it is portable to other operating system types, and is not dependent on the windows/.Net stack as much to function.<br />
<br />
There may be other tools that fit your needs better, Wiki's can be edited inline from a browser and keep good version tracking of each meta page, content management systems like Share Point can allow you to track updates to your requirements, contacts, or serve as a portal for storing key documentation for a production. Whichever tool your team chooses for source control, it is critical that your entire team is on board with its function, and regular use to help avoid major headaches between builds. The value of source control is manifest in how it allows teams to track branches of in progress development code, tag certain releases for easy comparison later, and giving a history with comments about the update and logs of the files that have changed. This logging function can be of great value when trying to track down how a particular strain of bad code may have made its way into your product.<br />
<br />
Then some source control systems allow you to track the progress of development from requirement, to deployment. Team Foundation Server has these features out of the box, while open source tools like Subversion require other components, for example TRAC, to help track the progress of development. I hope that whatever your team chooses to use, that you will find the source control to be a great asset, that helps facilitate your development needs.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-7883662902124049862011-03-13T15:40:00.001-04:002011-03-13T15:45:28.175-04:00Some thoughts on the Programmer vs Tester Argument<a href="http://shrinik.blogspot.com/"><span class="post-author vcard"><span class="fn">Shrini Kulkarni</span></span></a> wrote an interesting blog about arguments related to the programmer vs tester who is better at testing and why (<a href="http://shrinik.blogspot.com/2011/03/programmers-make-excellent-testers.html">Programmers make Excellent Testers - Arguments and Counter Arguments</a>). The idea comes up now and then in development circles, and whether you are a programmer, a developer, a manager, or a student of Software Development, sooner or later a question like this may enter the picture.<br />
<br />
Shrini does a pretty good job of illustrating some of the reasons why at any given snap shot moment, why a programmer or tester may be better for a particular task of testing. I've hard some of these arguments before, and I find that the context of the situation, and sometimes the budget of the team often dictate just how much a programmer or tester is asked to do. One thing that I find interesting, is why things end up being the way they are. Each of us if you will may start off as a blank slate at birth, and we learn many, many different tasks. When we enter the Software field, we even at that point have different ideas, philosophies and hopes and dreams of what a life and career in software might be. Whether that is to be an Engineer, Programmer, Designer, Architect, Manager, Lead, Tester, each of us started off with an idea of what we might get from this rewarding field. How many of us though stay on the path we initially pointed our ships bow of learning and experiences towards? I'm not sure a survey could even give a good idea for why people choose one career path from another. People are as varied and individual as grains of sand or rocks on any given beach. <br />
<br />
So when I reflect back on my career in the software profession. I can't help feel the irony to have basically come full circle, and reminded of at least one of Shrini's points. When I Started, all I wanted to do was write Code. I had aspirations of being a "Software Engineer" and thought I had a pretty good idea of what that entailed. Though I lacked experience I had not thought beyond that little view point. So when it came time to start my first professional job, it didn't take long before the ideal, and reality clashed.<br />
<br />
Learning is often seen as climbing a pyramid between four points. Along one Axis is technology and the closer to it the more experienced with technology, particularly specific technological contexts represents your learning and experience. At the other end of that Axis is the Second point of our Pyramid, that of working with people. The closer you get to it, the better you work with people, but perhaps you end up spending more time with people then the technology. Then off on the opposing axes would lie the other two points, in one direction is that of being more of a specialist. The closer to it, the better and more specialized you are in your context. You might call someone at that point an 'expert' of sorts, while at the other end is the generalist. They may be better at one thing than another, but their broad base of experience helps give them a larger context to evaluate their problem set. <br />
<br />
As you move from the Technologist point of extreme towards the specialist you are really moving around the base of the Pyramid. Or if you move to generalize you may be moving back the other way along that face. However, since Computer and technological problems are inevitably people problems, noone stays on the base very long, they begin to grow upward, and toward being able to deal with people. <br />
<br />
I recall being taught taught this 'myth' that programmers code, and testers test, and neither the twain shall meet. Thus fresh out of college there was this propensity among some of us, myself included to see testing as beneath them and not worth their time. This may have been vocalized by professors, or it may simply have been a matter that Testing was not a focal point of CS or Engineering coursework in college. The absence of a tester point of view from my Academic experience, clearly colored my early thinking about software.<br />
<br />
I had a desire to crank out quality code, usable code, and I saw Software Engineering as a tool to get there. Just one problem. In the first shop I worked, Software Engineering principles were not necessarily being practiced. Even if they had been, I'm not convinced it would have solved the host of problems I encountered early in my time there. I came to realize very quickly that programming alone didn't insure the code was ready and met the customer expectations. Heck sometimes even other programmers don't meet other programmers expectations. Now almost 8 years later, I'm a full time tester, and now I see the other end of the equation, where some developers seem to see the majority of testing related tasks as 'not their job' or beneath their position or station.<br />
<br />
I'm sure, I am not alone in seeing this, and I find it both sad, but in some ways understandable. After all in the technical realm you have generalists and you have specialists, to be good at one thing requires many hours of study and honing of that skill, often at the expense of others. Thinking back to that Pyramid, and admittedly, this is a very rough model or metaphor in need of tweaking and polishing, We may walk around a level of that Pyramid closer to one skill or another, We may move up it at an angle as we learn multiple skills practically at the same time, and sometimes we may move straight up.<br />
<br />
I see the technical Specialists close on the face between the top of the Pyramid (the metaphorical pinnacle of learning success), and the Specialist and Technology points. The more specialized the developer, the closer to the specialist point they may be, and likely further up the pyramid to its zenith. For generalists, the same could be said between the Technology and generalist point to the peak. The more a person pours into one technology or into technology in particular the less time they've spent gaining other experiences. So it should come as little surprise when a twenty or thirty year experienced developer who has poured his life into coding has gotten so far away from knowledge about testing as to see it as a foreign country. <br />
<br />
<br />
Even among programmers there are generalists and specialists in particular ideas or platforms, and each type of developer brings their own unique viewpoint to the problem at hand. What does that have to do with testing? Well we have something a bit similar do we not? We have people who focus on particular schools of thought, we have people who pour everything into automation, or pour everything into exploratory style testing. We even have some who are trying to learn as much as they can from every category, not knowing what tool they'll need next.<br />
<br />
<br />
<br />
<br />
Now Imagine that there were two more points, a high and a low along the Z Axis the top would be Programmer and the bottom Tester. I can see from adding those points, and removing the metaphorical pinnacle and replacing it with a point for programming and testing. A person who is a good programmer and continues on that path, may learn at the expense of learning things related to testing. Just like a programmer who chooses to learn more about testing, will have less time to focus on the craft of programming. Each of us then may move upward or downward toward being more programmer or tester. Forward or backward at being more involved in the technology vs the people, or more a specialist in one area vs a Generalist in others.<br />
<br />
So when I look at the tree of development with Development as the grand daddy of all the other nodes, and programming, and testing each on separate branches, I can't help see that there are so many varieties of testers and developers out there. Making each of us unique, or different as our experienced build upon one another. The real question I think we should be asking, is where do we fit in these areas. Are we more technical or more people oriented? Are we more specialists or generalists? More programmer or more tester?<br />
<br />
So who is better? I'm not sure the question matters. I think the context of program budgets, team size, and backgrounds often weigh into who does what, and as a team on a project we must each do what we feel we can do that will help the project effectively. Maybe that Senior level developer with twenty years experience would be wasting his time testing, its not his strength, or area of study, so a specialist tester would be better. However, I would hesitate to ever rule out a particular individual from testing or other development tasks simply because of lack of experience. The software profession, like so many is one where dedication to life long learning is paramount. So we should bear that in mind as we forge ahead, no matter which direction we point our bow of learning to on the next course we may set.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-36316891899551064382011-02-05T19:37:00.000-05:002011-02-05T19:37:23.983-05:00One question, many responses.I have been going through something of a transition phase in my life. The project I was on, has effectively run its course, and so the uncertainty of where I would end up next began to pop up. It has been a stressful time, and I know that there may be more stress in the future. Change seems to require quick study, a measure of patience, and the eagerness to take it by the horns and honk till the cows start moving. <br />
<br />
As I have begun preparing for this next assignment, anticipating that Testing will once again be a primary role that I will fill, I began thinking of how I could be better prepared to meet my new project team, and how I can be ready to give an answer to some of the things that may come up as a direct result of statements I might make, statements that are greatly influenced by a large measure of reading, and collaborating with other Test Pros online, through the blog o sphere, or twitter. I find I am on edge, not in a bad way, but the Eagle Scout in me likes to be prepared.<br />
<br />
Up till now, I've been something a lone gun tester, going where ever the current test ideas, or recent releases seem to point as areas of risk. Sure, the previous project I worked with a pair of testers, but so much of the work I was doing, was different than theirs, that in essence, I brought something unique to the table something that provided value for those teams. I want to continue to provide value on my next task. There will be a lot of new faces, new ideas, and new challenges to overcome, but I'm ready, and eager to mount up and go chasing over and after what lies at that horizon.<br />
<br />
As I have been preparing myself mentally for this scenario, a thought occurred to me late last night. It was a thought that gave me pause. The thought was this:<br />
<br />
<blockquote><blockquote><a href="http://twitter.com/#%21/Veretax/status/33690236454174720">If you as a tester on an existing project, were allowed only one question on your first day about the project, what question would you ask?</a></blockquote></blockquote> <br />
I regret that I had to hit the hay before I could see some of the fantastic responses from the twitterverse. Some of the responses were similar to the ideas ruminating in my own head. Reid Sheppard asked: "<a href="http://twitter.com/#%21/ReidSheppard/status/33695890841542656">What's the URL to the SUT?</a>" Lanette Creamer asked: "<a href="http://twitter.com/#%21/lanettecream/status/33716057063555073">1 question to who? Very important to know.</a>"<br />
<br />
Truthfully, Lanette's question was really asking for clarity to the context of the above question I asked. Since it was framed with out any context for where, or who was available to ask, it was very much an open ended question. I intended it to be a little open ended, but as I slept on it I realized something. Not only was it lacking in context, but it begged the question. For any given project X, how much might you as a tester know before officially being on the Team? <br />
<br />
Every test effort begins a bit differently, but I imagine many of them where a new person is brought into an existing team (the specific context of which I was mulling), would begin by some kind of interview, or screening. Unless the tester and hiring authority had some familiarity with the particular tester they are hiring, The tester could be just as much an unknown to the hiring manager, as the hiring manager, the test team, and product under test is at the start for the newly hired tester.<br />
<br />
I find myself coming into a team that to my understanding has already been through several bug/fix and release cycles. So to me, the context in this case actually includes a few things that are not clear in my statement. The Customer is an entity with which many people might have some experience, or at least preconceived notions regarding how this entity helps them leverage what their service provides. <br />
<br />
Stepping back from the specific instance in my mind, to a more general case, which is really where this question began. How much does the average tester know about a project before going in? It might depend upon how their interview and contact with the team began. Maybe you know who the Customer is, but maybe that information is sensitive information that is only given once a person is a part of the team and agreed to sign on. Maybe, you do not even have an idea of the scope or mission of the subject under test before agreeing to act as a tester. Sure there may be cases were that information is available in that screening process, but if it isn't, that's context most testers would hope to grasp early in their testing role. <br />
<br />
Others responded with questions about why the limit of only one question? <br />
<br />
<blockquote>Ajay Balamurugadas asked: "<a href="http://twitter.com/#%21/ajay184f/status/33695643457294336">How can I get information without getting restricted by number of questions? #there is a better question :)</a>"</blockquote><br />
In fact, the limit of one was not meant in my formulation as a hard limit. My thought was what is the first, question, the first one you'd ask so you can begin pondering all the who's and whats, whens, wheres, whys and hows of testing the application. Perhaps I can give more idea why my thinking was 'limited' in this context.<br />
<br />
When I first came on board here, I remember days of reading that I had to go through. Much of it wasn't even related to the tasks I would be doing day to day. time sheet training, ethics rules, benefit packages, taxes, I-9, so much paperwork. So for someone who is coming in brand new, to be a full part of a team as a tester, the thought occurs to me that they might spend most of their first hours doing a lot of untester things. So what if at the end of the day, you have an hour left, your tired from all of the tedious paperwork you've had to file, you've signed your life away with a non disclosure agreement, or perhaps signed a contract for term to serve. Whatever the case is, the thought was this: <br />
<br />
What if in the span of that one remaining hour, in the process of being greeted by the team after you emerged from where HR had you sequestered, what if one of those team members, not a customer, probably not even the manager, it could be another tester, a developer, a business analyst, whoever it may be, what if they turned to you and asked that question, "Hey, we are glad to have you with us. Just ask us any question and we'll try to answer it." However, the new colleague, then adds that they only have a few minutes to spare before they will be leaving for the end of the day. It's possible you might only have enough time that first day for a single question to ask. There are so many questions you could ask, and maybe there would be time for more than one question, but depending on the answer you receive it could take up the rest of the time to complete. This was the frame of mind I was in when pondering this question.<br />
<br />
In reality, that first question could be very important, or it could be just the first of a series of questions. That first question might be forgotten, and left in the dust bin never to be remembered. The thought would be not so much what is the one question you might ask, but rather what is the first question you would ask? Would it be the same question in a different context? Perhaps it would be different.<br />
<br />
Truthfully this does not really matter, there are many questions that will need to be asked to successfully test any product. Some of these will be obvious, and some will have to be unearthed after peeling back a few layers of the onion. So I come back to Lanette's question, who is there to ask? <br />
<br />
If it is a tester, perhaps you might ask what areas they've had difficulty testing? Or maybe you could ask them how the team documents the defects encountered. If it was a programmer, you might ask them where they think the code is weak, or as Lanette suggested: '<a href="http://twitter.com/#%21/lanettecream/status/33918722942959616">I like to ask developers, "What was the hardest part of X to design? What was the hardest to code so far? What worries you?</a>' Sometimes the developer will know where their code is weak, or what areas were harder to code, or what areas they've had particular difficulty with. These kinds of questions would help to identify areas of risk.<br />
<br />
In actuality, it may not matter what our first question might be, in fact the thing that sticks out to me is that the issue really shouldn't be what the first question is. The first question could have a myriad of answers, and in turn a number of counter, or clarifying questions could come up in the course of understanding that answer. I found this an interesting puzzle to consider. Just like I do not believe there is a such thing as 'best practices' in testing, there is testing that is good, and techniques, or approaches that are good in some contexts, but perhaps dangerous in others. So perhaps the best response would be to ask the developer or tester that has opened this door, to simply explain their experience while testing the product. As such an open ended question, the ideas that might be unearthed could be true gems that lead to effective testing down the road.DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0tag:blogger.com,1999:blog-2726520768982943835.post-60885960202637571262010-12-03T11:15:00.005-05:002010-12-03T13:14:22.609-05:00From the dark ages, to test automationSome days inspiration comes so thick that all your ideas begin to stick together in one melding like pancakes with butter, maple syrup, and strawberries. What follows is a second post, that begin inside my earlier posting: <a href="http://bit.ly/e33Ou0">Where in the world are all the software testers?</a> So if some ideas from that post are also found mixed in here, that is why, but I did my best to refactor my initially 2,500 some odd words from the initial post to create what I feel are two very distinct posts. While the first one is my thinking about<a href="http://angryweasel.com/blog/?p=257"> Alan Pages blog on forward thinking</a>, this one is a bit of an experience report, all be it a bit of a history about my first experience in software, and a bit about some of my more recent challenges. I describe the trial of working with out a source code repository, and even describe some of my frustrations at having to build using clearly out dated technology. Sometimes it is hard to realize how valuable any given tool is, unless you have ever had to try to work on a project without that tool. This is where I began, and to forget how I first tread foot into the industry, would be to forget how far I've come, how much I've learned, and how much more there is to learn and do...<br />
<br />
<br />
When I first started in Software, testing was seen as something I did when the time and necessity called for it. Though initially I was hired as an integration developer, focusing on piecing module code from one place into another in the stable product line, testing was something that seemed as if it was just part of the job, but not necessarily the most important. Maybe its the way software engineering is glamorized in the courses I took at WVU, I'm really not sure; nevertheless, as I began to grow understanding around the product, its structure, and purpose, It was not long before I found myself in a "Test First, Test Often" mindset. <br />
<br />
The reality of my first professional Job seems almost barbaric compared to recent projects I worked on. There was no source code repository, no Visual Source Safe, Subversion, or CVS in general use, so this meant that when new modules were being developed they were always in what was deemed as the last stable development build. Many times the build that the features were added to comprised a number of new features and bug fixes, so the porting of that code had to be done laboriously by hand. If I had not been so green as a Developer at the time, and had scene the value of source code repositories before my professional time, maybe we could have changed how things worked.<br />
<br />
In any case, it was my job to kick the tires on these features by porting them into the latest and greatest build, and then seeing what the new additions actually did. Suffice it to say, I highly recommend a source code repository for projects, especially when you have people working remotely or on different sites. However, this was the situation I found myself in as a young software practitioner. Finding all the right bits of code, initially was not easy. Sometimes I would get complete source, and have to scour every class for changes (And really, with little background on the project before, It is quite possible some of those changes had been changed not by that module, but in previous bug fixes in the new build.) So I began pushing back to have some way of seeing and marking what areas they had changed, so I could focus there, and not on other parts of the code. The wrap around comment with tags for each module became the first thing I searched for when integrating.<br />
<br />
That's how the process was initially described to me, so that's how I went about my tasks early in that assignment. This turned out to be a harder more time consuming method though. With developers deployed overseas, sometimes on different operating system setups, even the code they wrote that worked on their systems did not always work correctly on hours. Were these Regression errors that crept in as features kept being added? Or were these simply the result of the different platforms we were using? To this day I am still not certain, but the obvious solution came that we began requiring a demonstration build of each feature. First it would allow us to see if the build actually worked on our production systems which were perhaps closer to what our customers actually used, and secondly it would also allow us to see whether the feature worked as expected before putting the time consuming 'monkey work' of copying code A to class B. So I found myself doing about 50% more testing than actual clerical/coding work on our project, and found a lot of bugs in the process. Suffice it to say, I look back on those first steps as very raw, hard ones in my development professionally, and am thankful that I have learned and moved beyond such outdated techniques. <br />
<br />
So it has been just over five years that I have been on this latest assignment, and one area that I have found myself thrust into has been testing. What others on our team realized, though I couldn't see it from my eagerness to help out the team in any way, was that I was actually a pretty good tester. I'm not sure where this skill, or perceived confidence in me as a tester came from, but as a Colleague said to me just this past week, "He's a testing Wiz." I smiled and accepted the compliment graciously, but deep down, while others think I have it all figured out, I know that I have a long way to go before I consider myself to be any kind of expert, at Testing or Software Development. Truthfully I wonder if it is even possible to be an expert in anything software related given how fast the state of the art moves in some areas. <br />
<br />
<div class="MsoNormal">So what does this have to do about forward thinking? Before the last year I had not seen myself as a tester, just a software developer, eager of doing whatever the team required to succeed on our projects. I stepped up to an offering of a different role as a developer of test automation, and also testing in general, and I embraced the chance to learn and do something new. I did so not enter this assignment knowing how long I would be kept on the project, and I did so, having done perhaps more coding work than actual testing on recent assignments, but I didn't let that stop me. As I had recently learned from the classic book from Spencer Johnson, M.D. "<i><a href="http://www.amazon.com/Who-Moved-My-Cheese-Amazing/dp/0399144463">Who Moved My Cheese?</a>"</i> to not only anticipate change, but to embrace it, and relish it, I plunged into what initially seemed like a rather dull engagement, and not to mention highly time consuming, but I could see why they needed someone to focus on test automation, it just frankly took too much time to have the regular testers doing it. </div><br />
So I embraced the change, challenge, and at times boredom of test automation, and learned how to string together tests in a version of the test tool we used. While I looked for and anticipated change, I had set my mind to get 'comfortable' in performing these tasks the way they were demonstrated to me. I didn't know at that time that anyone else had tried and given up on the effort in the past, nor did I have any prior experience or research to back me up in finding a better way, but it was a start.<br />
<br />
As I pushed forward through these challenges, I took time when I could to research the certain unique problems I encountered along the way, and at some point I stumbled onto a blog. I don't recall which blog it was that I first read, but I began reading this blog and using the challenges to learn more about testing in general. Then one blog linked to another, Google Reader began suggest some other similar blogs, and suddenly a new world of testing was open to me, and as I learned, read, and absorbed new ideas, and techniques, and it transformed my understanding and view of the field. <br />
<br />
Now I am engaged on a new project, one where I am not constrained to a particular tool set, rather constrained to doing the best job I can in testing the application to find bugs, to learn, and to apply as never before new found technology. Some of my tasks have afforded me the chance and need to develop a test harness, utilizing both C# and Selenium. Perhaps later I shall try to bring some examples of the generic process I went through to create and build a test harness for some of the automation tasks that make sense for this project. For now, it feels good to no longer be back in the dark ages, but to be in the present ever living moment, applying new ideas as I encounter them, learning and building both my knowledge, as well as wisdom on when and where to use certain testing ideas.<br />
<div class="MsoNormal"><br />
</div><div class="MsoNormal"><br />
</div>DiscoveredTesterhttp://www.blogger.com/profile/03855355676637483294noreply@blogger.com0