…(jmeter-server.log on server machine) I could see that the connection back failed. I must add, that I am using Ubuntu in a headless mode to do these things and it looks like a Linux/Debian only issue.

The log output was:

ERROR - jmeter.JMeter: Uncaught exception: org.apache.jorphan.util.JMeterError: Could not return sample
 at org.apache.jmeter.samplers.AsynchSampleSender$Worker.run(AsynchSampleSender.java:148)
Caused by: java.rmi.ConnectException: Connection refused to host: 127.0.1.1; nested exception is:
 java.net.ConnectException: Connection refused

After searching the web for quite a bit there was little to go on. Eventually I figured out that the issue is actually on the client side and not the server-side as you would anticipate by the error log. Google didn’t quite help as much as I would have liked it to so I thought I’d just do a blog post o that others can find a clearer answer.

The solution is to go into the /etc/hosts file and find the entry that looks something like

127.0.1.1     <server-name1> <server-name2>

Comment this line out with a # or delete it completely.

Please note though that I don’t know if this has any other implications. The line was added to deal with issues in Debian when using Gnome.

I hope this helps someone else that runs into the same issue and sorry to the non technical people out there for this post.

The tester’s task was to formally document their testing effort using a test case repository tool. To do this they used their memory and experience of using the system to write out in more detail the test steps already performed. The tests were already executed, and bug reported. This was just a documentation exercise to capture the tests, for future use.

The testing performed was done using charters in an exploratory manner. The tester was left to learn about the product feature and build up a group of tests. They came up with many tests, including the following. They knew what each meant, but there was too little detail for them to be reused.

Test Idea: Logon
Tests:
1. Known bad password
2. Invalid password
3. No password

And each of the above were supposed to be split out so anybody could pick up the tests for next time. Ambiguity was to be cleared up, and greater levels of detail added. The intended detail was something like below.

Title: Log in to app with known bad password
Preconditions and Setup
1. User is general user, with general roles and permissions
2. Site is \\testserver\mainbranch\Application3.0\Login.htm
3. OS is Win64b Pro
4. Browser is FF9
Test Steps
1. Go to login page
2. Enter known bad password eg notmypassword12
3. Press Enter or click on Log On button
4. Wait for response
Expected Result
Error message saying “bad password” will appear

This would have to be repeated for each of the three tests they outlined (bad password, invalid password, no password).

But the tester did this:
Title: Log on using various passwords
Preconditions
1. User is general user, with general roles and permissions
2. Site is \\testserver\mainbranch\Application3.0\Login.htm
3. OS is Win64b Pro
4. Browser is FF9
Test Steps
1. Go to login page
2. Enter known bad password eg notmypassword12
3. Press Enter or click on Log On button
4. Wait for response
5. Enter invalid password eg !@#$%^&*
6. Press Enter or click on Log On button
7. Wait for response
8. Enter no password
9. Press Enter or click on Log On button
10. Wait for response
Expected Result
[4] Error message saying “bad password” will appear
[7] Error message saying “invalid password” will appear
[10] Error message saying “please enter password” will appear

The tester made some interesting comments.
Too much admin
The tester decided to mix up multiple tests in the one test case record. The problem in the testers mind was that they felt that there was too much description being replicated. They would have had to copy/paste the preconditions across all tests. They thought that if they combine tests, then there is less administration work.

The system will change anyway
The tester mentioned that there was little point in spending much time on the exercise as the system would change over time, and the tests would need to be checked each time for accuracy and relevance.

I wouldn’t follow this script myself
The tester exclaimed that they would not actually follow the test script. They find test scripts very boring and monotonous to follow. They find that they just read over the script, get some general ideas, and go and perform testing in their own way.

Documenting wastes a lot of my testing time
The tester estimated that they spent more time on documenting previously completed tests, than actually performing the tests themselves.

Illusion of structure
The tester confessed that they knew what they were doing was not adding much value. But they did comment that they thought management like to see tests all in one place, and in a table of some sort. The table gave the test script the illusion of structure.

Is this a disobedient tester or one trying to use their brain?

Author: Richard Robinson

And that got us thinking… what other testing animals make up the testing profession zoo?

• Possum testing – Playing dead in the headlights of process.  aka pathetic compliance
• Leopard testing – “Leopards are agile and stealthy predators”  – Wikipedia.  The stealthy tester is well camouflaged and appears possum-like to the untrained eye.  But quietly, the leopard is an agile hunter and will do valuable testing unseen and below the radar.
• Peacock testing – A deliberate attempt to wow via a magnificent display of extravagant test cases and scripts
• Meerkat testing: Always sticking your head up and being receptive to new information.  First to stand when overhearing a BA talk to a developer and go “wait, what was that?”
• Ostrich testing: Testing with your head in the sand.  If it ain’t in the requirements document, I don’t want to know about it.  It doesn’t exist until it’s written down.
• Penguin Testing: Testing in a tuxedo.  Black Tie testing.  Or Formal Testing.
• Blue Whale Tester: Big ‘fish’ but all blubber.  Has to stay near the surface lest it gets out of its depth.
• Manatee Tester: Noone really knows what they do or what they are.  They just seem to sit there
  

  Noone knows what they actually do

Have you spotted any other testing animals in the wild?

The question in my mind was “Why do we keep on producing such cr*p software that people just don’t like to use?“. I need to caveat that a little as I solely focus on bespoke development for larger organisations here.

There’s lots of rational reasons why this happens. Examples are:

• Overly aggressive timelines
• No well-defined quality criteria
• Poorly understood requirements/no understanding of solution
• Bad SDLC
• Politics
  
• People
• Budget
• …

But there’s one that I actually never thought of listing….. FUN.

I can see how you go FUN?!?!?! Yes, fun. Let me explain…

IT people are not the drones they usually appear like (what managers like calling a resource). All those IT geeks & nerds are actually fiercely creative and clever. If you look at the hobbies and little things they do outside work you’ll realize that they are massively diverse and engaging. I’m talking about the guy that developed a new open source phone interface stack that outperforms anything on the market or the guy that did a full automated regression suite for some game because it was the cool thing to do. All of that with no pay at all.

What motivates them? Two things: challenge and fun

At work we totally ignore fun (not the challenges though). Projects are dull, rigid, predictable; everything but fun. Fun is difficult to plan for. It’s revenue neutral (is it really?). It’s something that the company has just shied away from banning in their work contracts (for legal reasons).

Fun you have usually working with all the diverse and interesting people. Project content wise….

Imagine working at Ferrari and the only important requirement is to get a soul from A-to-B.  The end product certainly wouldn’t be a Ferrari!

I am sure everyone working for Ferrari is really engaged in the product and will have at least a modicum of fun building one of the coolest machines on the planet and watching their team win the F1. Getting a soul from A-to-B is probably quite far down the list of important things (if not further qualified).

Why wouldn’t we try and harness that engagement, the fun and coolness factor? I think agile, scrum & co is a big step in that direction and my personal experience actually reflects that. Does that mean though that these things are out-of-scope for Waterfall/V-Model? I definitely don’t think so. Although I would put my money on the fact that one of the first things that would happen is that the project would go agile & scrum or something along those lines.

Maybe it has to do with trust. Freedom -and therefore fun-is bought by trusting your employees and peers. Missing trust is the cause of so much of the grief and issues on a project level. It is one of the reasons for RFPs, SOWs, contracts, reports, metrics,….

One way to increase fun, coolness and get a great product is to let the SMEs, geeks, nerds and creative heads have more say about how things are done. Even to the extent of redefining requirements and challenging business decisions. It takes a lot of trust to do that and it could go wrong. With the abysmal project success rates on large projects I doubt a case against can really be made.

But look at what you stand to gain!! It could even equate to a new gold rush in IT. Success rates might improve and $$$ wouldn’t become the only motivational tool you have to gain & keep staff (not that $$$ ever are).

So why would I start mulling over such things in a testing blog? The reason is that I look at quality of the produced product.
Let’s stay in the motoring world and imagine you’re a worker at Toyota producing Priuses. Now we all know they produce great cars with a high quality standard. Imagine that worker goes home, where he has a Ferrari 250 GTO in his garage that he is painstakingly restoring. When the restoration is finished which car do you think will have the better end quality? Which one will be “better”?

Now imagine you’d let that employee contribute to designing/engineering the new Prius. Wouldn’t you think it looked more like below? Or any other/better design than the no-design that Prius currently has.

Toyota Fun (my case in point)

The example here is a bit flawed but I think I can make my point. People do good work but they do they excel if they are personally engaged, have fun and pride in the product. That includes quality!

Or take the famous 20% employee-time thingy that Google does (did?). They say all new products come from that 20%. Probably because people are having fun with developing cool stuff. So yes, there are easy ways to harness all this creative energy. Question is then, why it’s only 20% and even more pertinent, why is not everyone following such a successful model?

From a testing perspective I can actually tell (over and above factors like complexity), when a developer/architect/BA/DBA didn’t have fun doing something. It will be riddled with defects and the product “feel” will reflect the work-for-$$$$-attitude. So I/we have a vested interest in seeing developers have fun.

On top of that I think testing something, where a developer had fun and could be creative, is way more fun and satisfying to the tester too. There is nothing as boring as testing a product that has been un-lovingly designed and assembled.

As testers, we can apply this to our own work, too.  Are we having fun while testing?  One of the issues I have with the factory school of software testing is that they are trying to eliminate the fun from testing.  Where is the fun in creating spreadsheets of scripts, and ticking them off?  Where is the personal challenge in declaring you have achieved “100% test coverage” because you have written a single test against each written requirement.  Compare that with the child-like glee in being given a new build, and the personal challenge of trying to find all the ways this program may fail in the field.  Which approach do you think would ultimately be most beneficial for the end user?

So next time you look at your process, product or whatever is in front of you in your work life, ask yourself: “Where is the fun in this?”. If you can’t find any please don’t despair or do something rash! Think about what you need to install fun 2.0 into whatever you are doing. Then try -with baby steps- to move there. You might be surprised about the returns.

Author: Oliver Erlewein

It takes time to build a community - and initially I was doing the *heavy lifting*. I so realised this is natural as it takes time for the community to *come together* and to share the load

Once the community gathers momentum, new voices/leaders emerge – The more members, the more potential leaders. The more leaders, the more momentum generated.

Enthusiasm can become a group multiplier – (paraphrased from General Colin Powell) – When discussions abound and there is energy within the group, the group becomes larger and more involved.

Like ideas/minds attract – Since the group started, I have met approximately a dozen liked minded testers with at least four or five with whom I share a similar philosophy.

Robust discussions take time to generate – it’s a safety thing for most people. They want to feel safe before they contribute

Discussion topics – What is interesting here is that there have been a number of robust conversations (certification, use of tools, performance testing and quality to name a few) that seem to ignite tester’s thoughts, ideas, opinions (or in other words passion). On one discussion thread, what was evident were the voices/leaders in this group not taking comments at face value. As testers, we have the right to question other tester’s comments/experience/statements. This is for a number of reasons:

• First – we want to understand and then possibly learn from you
• Second – we will challenge your assertions. We hope that you can defend your position and by doing so help us question our own models. It does not mean that we will agree with you but such challenges are necessary to build the craft.
• Third – it most cases – it’s not a personal attack
• Fourth – If your assertion is misleading or damaging in anyway to the craft then expect to be challenged (the end result if we don’t is a soulless, pathetic craft devoid of any creativity or thought)

Leaders emerge at unexpected times – often they have something interesting to say – at the very least, its fodder for discussion. I have been surprised at the activity of the quieter members of the group and often I have seen them raise valid and valuable points

There ARE very controversial topics that will get testers attention – It seems to me that certification is very controversial. Lets not shy away from it, lets discuss it

Most members of the group lurk – (which is not in itself a bad thing) but imagine if they actually contributed to the group?

As the groups reputation grows, so does the number of members that join – which leads to more leaders to the group

There is a ton more activity in the middle stages of the year – and a lot less at the bookends

The more vibrant the community, the more robust the conversation – which has been true this year. The group has regularly engaged in some really robust discussions such as performance testing, test tool management and certification.

The more robust the discussions, the more opportunity for *passive* learning by those members who prefer to lurk – There has been the odd comment by a *lurker* saying how much they have gleaned from the online discussions

 In summary, we have a small (184 members) vibrant community. The strongest voices tend to be those testers who align themselves with the context driven school of testing. However, there are a number of testers who don’t align themselves with any school of thought (or haven’t declared themselves as such) who contribute well.

This leads me to the opinion that we are building a better craft OUTSIDE of what I call, the template school of thought (those test managers who see testing as an artifact, standardise driven way to test). If we don’t engage in communities like software testers new zealand, then we will have less leaders participating in worthwhile forums such as KWST.

In my opinion the solution is *simple* – be involved in the craft, participate, engage, debate, discuss and learn!

Author: Brian Osman

*duh* because we want faster response times…. oh and we want to know how to scale our virtual machines…. oh and we want to tune our systems… oh and XXXXX….  there are tons of reasons. Performance testing has it’s testing rigor and we go and “hammer” the system to get at those answers.

One thing I like to do (because it’s fast and cheap) is use a calculator/spreadsheet for performance testing. I take architecture diagrams of present and future systems, infrastructure diagrams, requirements, human oracles and more and put all the numbers together. Then I check if they stack up. Like where the product tries to get 1GB of data across a 10Mbit network link in under a second. I don’t need a test to be able to tell you, that there’s a problem there.

But then it struck me today. There is something similarly simple that I am not doing (and am guessing not many performance testers do)….

Ask yourself, what is the web page that has a response time of 0.000 milliseconds and has a infinitesimally small  throughput footprint?

It’s the page that doesn’t get loaded!

Think of purchasing something online. You run through a dozen screens entering passwords, addresses, delivery types…. on and on it goes. Usually one shop worse than the next. Just as you start thinking that it is actually simpler to drive to the shop and buy the damn thing there, someone comes along and invents the 1-Click purchase. Never mind, what that did to sales of goods but think of what the advantages from the performance perspective are.

1. Fewer web pages, resources and redirects to serve up
2. Less transactions in flight at one time
3. Less database interactions
4. Less infrastructure handshaking and latency

These are just the obvious ones, there’s probably a dozen more. The example here might not even be a good one either but I think you get where I am going.

Is it not time for performance testing to look at a bit more than just response times?  This kind of analysis gets us looking beyond response times of single web pages but looks at complete flows. How much interaction does a whole flow create and can the process/business flow be optimized?

It seems to me that architects and business analysts ignore performance related issues in their designs. It can be false assumptions on what appropriate design is or just bad customer requirements that proliferate this behaviour. Too many unnecessary steps, interactive-polling overload, re-entry or confirmation of trivial data and many more that can be really annoying. Normal performance testing might highlight these issues too but the cost and effort involved might be much higher. This method is quick, easy and cheap.

So next time you front up to a performance testing gig maybe start right at the project beginning and have a look at what can be cut out of process and business flows and screen design/functionality. See if you can’t just go 1-Click. It could even make your pending performance test a lot easier by having less of and a simpler application to test. I must admit I have not yet done this but I can’t see how a simple check like this could not be worthwhile.

…and who knows, there might be a patent hidden in there too

Author: Oliver Erlewein
& thanks Aaron for the good tips for improving this post!

I’ve spent the last couple of years helping projects with their application performance in NZ (mainly Wellington). I thought it’s about time I wrote something on the experiences I’ve had during that time and the lessons learned.

NZ is comparatively a smallish place. 4.5m people live here. A large bank for example has about 0.5-0.75m customers. One of the biggest online applications running in NZ is probably TradeMe. They have 2.8m customers and about 75k-200k active customers at any point in time. On average they have less than 1m logins a day. If I contrast that to large international systems this is laughable. Ebay for instance has 83m users and  670 million page views a day (I don’t know from when these figures are though). Facebook has 750m users,…. So big international companies talk about building another datacenter, where we might start clustering.

We do things a bit smaller. That has its advantages – if we do our homework correctly. Most products used nowadays are designed to be massively scalable to the requirements of large international companies. So we should have no issues with performance….EVER!

But as you probably know from your own surfing experience this is not always the case. It gets even worse when we use web applications that are in-house. All of this should actually be a no-brainer. So what’s going wrong?

I’ll try and list the thoughts and experiences that I see are common in projects here (no particular order).

1. No performance testing/analysis
   Performance testing is still a rarity. Luckily awareness is rising and due diligence is being done more often. There are still long ways to go though.
2. Unrealistic & bad requirements
   In most projects I have encountered the requirements are about 10-100 times higher than needed. This results in tuning for the wrong sweet spot. You might have an app that works excellently under high load but performs badly under low load. Or you might agree to test something the user actually doesn’t use that much. Describing performance is not easy by any stretch.
3. Misunderstanding of terms
   Concurrent, load, stress, performance, latency, response time,… they mean different things in different projects.These need to be clarified first before discussing anything. Do not assume that a technically correct interpretation equates to what people understand what the term is.
4. Bad architecture
   My experience is that most solution architects treat performance as a minor worry, if at all. It presents one of the biggest risks to any project. I have seen re-architectures very close to go-live. Performance specialists (and I don’t mean performance testers only!) should be part of the early architecture & design phases.
5. Developers unaware of performance
   Not only architects but – in the heat of meeting the next milestone – even developers will not  think of how their code will perform. Only when confronted directly do they start tuning their code. It is less an active ignorance as a result of too narrowly focussed expectations. Non-functional requirements are often just an afterthought in development. There is the fallacy that they are minor things that are just by-products that any developer does anyway.
6. No implementation of product tuning guidelines
   Most products have tuning guidelines. They tell you things like threads per core, buffer sizes at different configurations or how to set up thread pools. Or they tell you what optimal hardware/virtualization configurations are for running the product. There might be hints on when and how to correctly cluster your solution. These are straightforward things that should be done/defined by the architects and developers before releasing their product. I have not yet seen a project do this without being told to do so by the performance tester. Oh and the excuse “there are no guidelines” doesn’t count. In that case the guideline is revealed by Google.
7. Political & bad choices for products
   IT is not as easy as the glossy brochures make it out to be. Products don’t work together, hardware won’t perform in certain set-ups and things are just plain buggy. As any good performance tester will tell you, everything will fail (that is actually always true. The question here is, does it matter). These are the things that should guide proper product selection but often products are selected by company default (we use Microsoft only/only HP servers/Cisco networking/…) or because the CIO went to dinner with XYZ product sales guy. Often products get chosen by people who have never even seen the product themselves or have any clue what it does. These people also are the same, that ignore advice from SMEs that know what they are talking about.
8. Lack of monitoring or useful monitoring
   Monitoring is one of the most important parts of running something in production. It tells you when things break. So why on earth would you not focus on it? Monitoring is also the brother of performance testing. Without it how can you see what load does to your system? But also the converse is true. Monitoring only can never replace performance testing.
9. Lack of qualified people
   There are only so many people in this country with specialist know how. We cannot keep up with the volume of new information that is being disseminated internationally. But amazingly it’s not only the complex stuff that goes wrong. It’s the basics. This to me just shows that we are not thorough enough. I think employers and customers should be asking for more and taking the lead with giving people the correct/full list of expectations and requirements.
10. Blind belief in delivery dates
    Ship and then performance test. Or “can we cut 2 weeks off the performance test?”.  Need I say more?
11. Lack of  performance testers
    My rule of thumb is the 10/20/70 rule. In performance testing 10% is the tool, 20% the training of how to performance test and 70% is just plain experience. It’s the 70% that make a performance tester. In Wellington I’d say there are probably only two-dozen performance testers that have that 70%. The market could do with a multiple of that but because this is learning from experience it’s a viscous circle. We don’t test enough because we can’t get the specialists and we can’t get specialists because there is too little performance testing.
12. Project timelines
    Ok, so when do we performance test? 99.9% of PMs will put a PT phase just before go-live. If you look at some of the issues above you might see why this can be a really bad idea. If you get architectural or product issues you might have a huge challenge on your hands and an immediate and huge delay in delivery. So performance testing should be pervasive throughout the SDLC to minimise the big risks.
13. Staff turnover
    I have worked on projects that have been running for years. Developers “get the feel” for performance once you have done a few performance testing cycles. They get better at developing performing code from the onset. As is normal for a project there is turnover. This means though that performance will drop when new team members come on board. They do not know the product well enough and will run into issues when the next PT comes. The process starts over. On projects that are highly performance critical I would suggest leaving team members in place for as long as possible and to never exchange a whole team big-bang style.
14. Testing vs. Investigation
    Last but not least it helps to understand the difference between these words. Testing means I have a clearly defined outcome that I am expecting. This is rarely the case. There are rough ideas, some data and guesses but certainly nothing that is clear and defined (see 2). So what performance testing – in all it’s diversity – is, is an investigation and tuning exercise. Think of this when creating/reading/critiquing the performance test plan or when asking for documentation. Also remember that investigation is an agile process and will form as time progresses so don’t be too rigid when prescribing what tests should look like. The investigation might lead you to unanticipated tests.
15. Complex Tools
    Performance Testing from a tool perspective is actually quite straightforward. Complex tools might make performance testing projects more intricate than they need to be. Worst case they get dumped into the “too hard” basket and don’t get done at all. Keep things simple and increase complexity when needed. In my experience the issues encountered in NZ are mostly not that intricate.
16. Following advice
    On nearly every project I have the situation, where a vendor/developer/PM/architect/… doesn’t believe that there is a problem or that the cause of the problem is something other than what I tell him/her. I have spent weeks waiting and trying to convince people to look at XYZ without success. Time and effort wasted, just to come back to XYZ and find it’s exactly what is causing it. I am not saying that I or performance testers are infallible (far from it actually). But it would help immensely if whoever could start by looking at XYZ first, as chances are that the performance tester’s experience in performance exceeds whoever’s.
17. Averages are worthless
    People love to talk about averages. They are easy to understand and from school we know they are important. Well, in performance testing they tell you very little. They actually tell you nothing without other numbers that form a context. I think it is safe to stipulate that you shouldn’t use averages in performance testing (at least when reporting). The number to go for is 90th percentile (or any other percentile between 90% and 99%). If you need details go to Wikipedia, they do a good job of explaining these. And when using these numbers always relate them back to throughput. Without this relation they reveal nothing.

There is a sector of the software development community that believes, no, accepts unquestionably as a truth, that testing is writing test scripts then executing them. This leads to a vicious cycle of managers and clients asking for test scripts, and testers delivering test scripts because they were asked for them, thus reinforcing the requests and so on ad infinitum.

I’m not going to describe how wasteful scripting before testing is, or even the oxymoron of putting scripted and testing together in a sentence. If you want to read further, Michael Bolton does a great job here and here. I make an attempt here.

Instead I’m going to put forward some propositions:
1) At the beginning of a project we know least about the project
2) Good testing requires sapience
3) Skilled exploratory testers perform hundreds of what could be considered discrete tests in a session. Few are worth repeating.
4) Most tests performed are informed by the results of the previous test.
5) The usefulness of a test is often not known until it has been performed. You don’t know if a rock is worth lifting until you’ve lifted it to see what’s underneath.
6) Testers, when following a script, will deviate from the script.
7) Different testers interpret scripted instructions differently resulting in differences between testers even when following the same script.

Therefore:
8 ) Test Scripts don’t tell you what you may think they are telling you.

Instead, let’s delve a little deeper into what’s being asked of us when others request test scripts. Could it be that that’s the only artifact they know of that will deliver what it is they want? Why do people want test scripts?

* They want to know what you did
* They want to be able to verify coverage
* They want some kind of repeatability

If you want evidence of what happened, for whatever reason, then how about thinking beyond scripts, and into transcripts. Like scripts, but written after the fact.

What’s written down in a script and what a tester actually does is often two separate activities anyway, so scripting, even when done with the best of intentions, is already behind. But if we transcript our tests, we can write down what we actually did.

And I’m not talking about recording EVERYthing we do. But when we perform some testing activity that we determine the particulars of to be very important, perhaps for some future formal testing, perhaps to show as evidence of some claim we make, we transcript those details. It could look like a traditional script, or it could take some other form, but it’s a win win for everyone:

* The tester is free to investigate the software as they see fit at the time of testing using their judgement
* Clients and managers get evidence of what was performed
* Tests activities that are judged to be worth repeating in a particular way in future are recorded
* Time isn’t wasted writing down testing activities in minute details at exactly the point in the project we know the least about it.

In conclusion, I’m asking for a subtle shift in the language we use to bring about a more fundamental shift in the activities and artifacts that are judged to be useful by others.

Let’s move from Test Scripts to Test Transcripts as an alternative. Perhaps if we start offering transcripts as an alternative, and provide good value from them, then people may begin asking for transcripts and we can break the cycle.

Alternative Test Artifacts may be the topic of a future post.

Author: Aaron Hodder

I’ve wanted an Apple ever since the original Apple II. My first Mac I ever saw was actually an Apple Lisa at my fathers design department. They were using it for CAD with a whopping 5MB Winchester drive. But the world turned out a bit differently. I never got to having an Apple II or a Mac.

Only in 2004, when we immigrated to NZ did we shell out for a MacMini and enter Steve’s world. Today we own several Macs, have had many more, have iPhones, iPods and are 101% Apple followers. We’ve never looked back.

But what has that got to do with a testing?

As it turns out there is/was someone at Apple which had a relentless drive for quality and usability. Now as you can easily guess that person is/was Steve Jobs (still struggling with the was here!). This drive is pervasive in all Apple products.

Again! What has that to do with my testing?

Ever since switching I have been confronted daily with something that works and gives me the power to do what I want. I have been using Windows PC’s for decades and have used Linux for nearly a decade and on no system have I felt this kind of empowerment. Empowerment, usability, friendliness, quality, ease of use, performance, craft, art, predictability and many more adjectives are what I use to describe Apple’s soft & hardware. And nobody can dispute that that is Steve’s footprint right there delivering all these things.

That is -fellow testers- what I took away, the ability to expect more. To be able to discern when something is badly made, unpractical, riddled with defects and plain unusable and to call it out. Steve has given me a new base reference I measure against. I now do care how many clicks something takes or if menus make sense or if screens and messages are understandable or not. The IT world has been badly served by most of it’s masters in that we are complacent and accept the bad quality and lack of proper thought when delivering IT to our customers. I can see it in the deliveries I get from developers. I can see it in the lack of criticism by testers. I can see it in the products that exist. No, the requirements are not the only source of truth. You are.

That means we should call out if something gets developed and is absolutely unusable or just plain wrong, even if requirements are met or the customer asked for it. Raise it as a defect! Raise a defect for things that annoy you, that you don’t want to live with. Or put yourself in the business users shoes and ask yourself if he/she would want that. Take a step back and look at the creation. Could it be better? Should it be better? Dare to ask the fundamental questions and act upon them. Steve Jobs has shown us all that thought and a relentless drive for quality can make you immensely successful. I’m not saying that all testers should now rebel against the establishment but I am pledging for common sense and reason to be used more readily and don’t be dissuaded by “it’s not in the specs/requirements/whatever”. That just means you have to find a logical, reasonable and good for the defect to be fixed (btw, one of those reasons should include why it makes $ sense!).

So that is what Steve did for me. He made me a royal -testing- pain in the bu**.

….and that is a very good thing, that I will always be thankful for.

RIP Steve Jobs 1955-2011

Author: Oliver Erlewein