Friday, March 27, 2009

Mentoring fieldstone 1 - Requirements analysis

Much of my testing experience has been in regulated financial institutions (yes, there are such things - we have rather tightly regulated banks here in SA) and so we spend quite a lot of our testing time analysing and raising issues against, specifications. I have evolved a Mindmap to help me when mentoring people in this process and would like to put it out there for comment or maybe it helps you in some way. I realise that since this is a Mindmap, much of it may be rather cryptic, so please also feel free to ask me questions.

Please note that the reference items for this Mindmap are:

Tuesday, March 10, 2009


James has just written a thought provoking blog outlining a hypothesis that Quality is dead. Having seen the things going on around me lately I tend to both agree and slightly disagree… and then end up firmly on the agree side.

At our previously major client (a large financial institution) they are now firmly in the trenches of their shiny new outsource deal with large outsource company.

A couple of reasons why I think that on the surface it may appear to have been a fantastic idea (and supportive of quality):

Millions have been spent on an end to end test environment – they have made a significant investment in testing

There has been a concentrated effort to get an end to end environment in place that can be accessed from Chennai. The outsource company will in the end have full control over the environment. We were testing most of our mainframe code along with the developers making their changes as and when they liked and with absolutely zero control over what they were doing. Sometimes if they were nice, they would let you know. But I guess if you are going to be spending that much on the total testing effort, then another few million is worth it to make sure that your testing is happening in a proper environment.

People are having to pay attention to their project plans and project planning – and so in theory, planning will improve and more time can be allowed for testing

The deal is fixed price based on a budgeted number of hours that need to be decided on, a quarter in advance. The ways things worked (when we were still around) is that when projects ran late, testing always “made a plan”. There was no fixed number of hours so we reallocated resources, cut our scope, worked silly amounts of overtime, analysed incomplete specifications and basically did everything possible to be “part of the team”. These guys don’t have to. I suppose that neither did we. Maybe the difference goes back to a perception of value. Since the Financial institution is now paying significant amounts more than they were previously paying per hour, people seem to listen more. Everyone has to sit up and take notice because if they budget for hours and the rest of the project is late…. well it’s tough. They need to repay for the new testing hours that will now be required in the following quarter OR pay a premium to have an amendment to the contracted hours. So should we have played the same game? Does it result in better quality? I don’t really think so… but I don’t have any proof yet. I do know that the perception when projects run late is no longer that it is a “testing problem”. The blame is now sitting firmly back in the Project Manager’s court.

Reason why I think it sucks

“The skills argument”

One of the primary reasons given for the outsource deal (or the way that it was sold to the trade union) was that the skill required to test the types of applications key to the future strategy was not available in South Africa.

I was copied on an email the other day where the new team were required to test a change control and so instead of analysing the change to determine what needed testing, they simply asked the developers, “which of the list of abc do you suggest we test?”. And then that was what they went with. Is that skill? Is that evidence of skill? I suppose they can always blame someone else if there are bugs in the part that they didn’t test. I guess they have a different view of quality.

How was our skill judged I wonder? Was it the fact that we didn’t put our foot down hard enough? Would we have been adjudged to have more skill if we were more expensive? Perhaps, if we had big fancy certifications…?

Even IF it was the case that we had no skill or not enough skill, and this is the part that kills me, why not invest all those chunks of money in upskilling local people? In a country where there is over 30% unemployment! Why not work with people who are trying their utmost to develop the skill within the sector that needs it the most?

Instead they have chosen to spend millions in someone else’s economy. Our industry is now over 400 jobs short with around 300 of those people searching the market – just from this company’s decision - with less and less jobs available as more and more financial institutions head in the same direction. And this from a company that claims it is committed to black empowerment. I just don’t get it.

It all makes me tired and depressed and feeling unappreciated. Tomorrow I’ll be inspired again to change that… for now I had to rant.

p.s. See also Ben’s thoughts after reading James’ blog.