Observations......Part 1

Note: This post is rather old… has been gathering dust in the pile of ideas, but I think is still relevant in terms of its observations of back then.

I wanted to share some of my observations on my ET/SBT experiment.
The first observation is around People... the way it affects me as a manager and I believe my team members....

The last project that I led (an ATM rewrite), was a really large and complex one and I was a lot younger and more stupid then :). I led the test strategy and effort based on what I thought would work best for the context. We ended up with around 4000 test cases and initially we had 12 very junior testers (brand new to testing) to execute them. We also had 5 more senior testers, checking the results, and updating them on our management tool. The reason we had to split these tasks was to be more “productive”. We were testing on physical ATM machines and therefore the tests were run from paper “test packs” and then updated after the fact. The defects were also logged on paper and then updated into our management software by the more senior testers. The problem with this is that we ended up with very bored junior testers. Why? Well, firstly they didn't log their own defects formally into our defect management tool (and therefore didn’t get to practice this properly). They also didn't update their own results. This meant that they didn’t have to take direct responsibility for their work – someone else would be deciding whether or not to mark the test passed or failed. We also had very bored senior testers – their job basically became administration. Even though this wasn't the desired result – I had set it up that way.

In “Lessons learned in software testing”, the guys have a really important lesson that they share - “if you want your staff to act like executives, treat them like executives”. I was trying really hard in every other way to do this, but the responsibility for testing had to be shared. This created opportunities aplenty for conflict and not much for learning. And I guess, maybe there were also other project factors that were to blame for this. The thing is that even though there was every intention that good mentoring and coaching and skills development would happen by splitting the team, it often didn’t.

Let me contrast it with the situation on the SBT project. Firstly, my experimental project was a LOT smaller. My team was made up of junior testers that all had some experience, but were not super experienced. I met with them every single day. This is the first difference – I would meet with the more senior team members in my last project every day, but only in emergencies (and mentoring sessions) with the rest. I was quickly able to assist and guide the session based team members in their day to day testing activities mostly using through the Debrief sessions. They did a great job of logging defects, following up on them, reporting back to me and recording their sessions and updates for the few scripted tests that were run. Basically, I feel that these testers didn’t suffer from boredom at all and they all really relished the opportunity to be responsible for a function or a section of testing. In terms of job satisfaction as well as opportunities to learn and grow, I definitely think that my SBT project was far more successful.

Some thoughts on estimation...

As some of you may know, I am currently working in a test services department that is in the process of outsourcing all testing to an Indian company. During our handover, we were asked if we used any tools to estimate for projects. Hmmm we thought.. tools? Well, if you mean our experience and our brains... then yes, I guess we use a tool. In fact, most of what we do when estimating, we do almost subconsciously. So this gave us an opportunity to really reflect about all the factors we take into account when estimating.

The result is this mindmap (please note these ideas are not just my own - all the TMs at SS and of course all the people I read have contributed). The idea of splitting it into 2 sections came from something Jerry Weinberg said at CAST which really resonated with me. People are always counting fixing time as testing time.....this is something that I really want to start communicating more thoughtfully about. I'm sure that this map doesn't have everything, and can certainly be improved on.. so if you have any thoughts - please share. There also may be some context specific things that are a little unclear so if there are questions - please ask.


p.s. the interesting (and somewhat scary for me) part is that the outsource company have included some of the factors that we consider into their estimation tool (an excel spreadsheet with formulas for all kinds of things) where one would allocate some kind of percentage weighting for each factor that then increases the number of person hours... it gives the impression that there is science in estimation... which takes me to another thing Jerry said in his new book, "Garbage arranged in a spreadsheet is still garbage"