Sikuli – for all those hard to reach places!

Ever had the scenario in automated testing, that you had something to automate that really didn’t fit any of your tools? Something that was as bristly as an Echidnea?

Normally I try and steer away from anything that doesn’t use standard protocols and/or interfaces. Things like Flash, Silverlight and others. Not that there aren’t test tools that handle these things but it’s just that it’s messy to say the least. For open standards like HTML or SOAP there are gazillions of ways to automate.

So I got surprised by having to test an application on -or should I rather say through- Citrix.

Continue reading

Web Page Analysis Basics for Testers

I usually move in the performance testing realm and one of the things I regularly do, is check for obvious omissions in website design before I get into the low down with testing.

What do I mean by that?

There is such a thing as (and I am having difficulty writing this) Best Practice, when it comes to web page development. These are technological imperatives that can be easily checked by using simple tools. You don’t need to be an HTML guru to use these or to gain more knowledge about your website under test.

Continue reading

Performance Ideas from 1-Click-Buy

Why do we performance test?

*duh* because we want faster response times…. oh and we want to know how to scale our virtual machines…. oh and we want to tune our systems… oh and XXXXX….  there are tons of reasons. Performance testing has it’s testing rigor and we go and “hammer” the system to get at those answers.

One thing I like to do (because it’s fast and cheap) is use a calculator/spreadsheet for performance testing. I take architecture diagrams of present and future systems, infrastructure diagrams, requirements, human oracles and more and put all the numbers together. Then I check if they stack up. Like where the product tries to get 1GB of data across a 10Mbit network link in under a second. I don’t need a test to be able to tell you, that there’s a problem there.

But then it struck me today. There is something similarly simple that I am not doing (and am guessing not many performance testers do)….

Ask yourself, what is the web page that has a response time of 0.000 milliseconds and has a infinitesimally small  throughput footprint?

Continue reading

Thoughts on Performance Testing in NZ

I’ve spent the last couple of years helping projects with their application performance in NZ (mainly Wellington). I thought it’s about time I wrote something on the experiences I’ve had during that time and the lessons learned.

NZ is comparatively a smallish place. 4.5m people live here. A large bank for example has about 0.5-0.75m customers. One of the biggest online applications running in NZ is probably TradeMe. They have 2.8m customers and about 75k-200k active customers at any point in time. On average they have less than 1m logins a day. If I contrast that to large international systems this is laughable. Ebay for instance has 83m users and  670 million page views a day (I don’t know from when these figures are though). Facebook has 750m users,…. So big international companies talk about building another datacenter, where we might start clustering.

We do things a bit smaller. That has its advantages – if we do our homework correctly. Most products used nowadays are designed to be massively scalable to the requirements of large international companies. So we should have no issues with performance….EVER!

But as you probably know from your own surfing experience this is not always the case. It gets even worse when we use web applications that are in-house. All of this should actually be a no-brainer. So what’s going wrong?

I’ll try and list the thoughts and experiences that I see are common in projects here (no particular order).

Continue reading