JMeter is a wonderful product but in some aspects it has it’s kinks. So when you do testing on several remote clients and have CSV data that fills variables you start to hit some ugly issues. In my example here I am reading login data from a CSV file. The thing is, if the same user logs in twice (or more times) simultaneously it’s  FIFO. All other users end up throwing an error.

The usual way to tackle CSV files in distributed JMeter environments is to copy the CSV to every client. But that would mean all of them kick off with the same line, thereby causing the problem. You can prevent that by cutting up your CSV into pieces and have one for each remote/client machine. This works but is tedious if the number of clients varies or the CSV changes often. You’d ideally want something more versatile and automagic.

So my solution is to have a webserver (or the JMeter master machine) dish out JSON responses with the data I need and each remote/client just queries that. And that’d look something like this in PHP:

[UPDATE] I changed the script. The last one I had was a bit unreliable if the CSV contained only one item per line. Here is the new one. Much shorter and does 99% the same.

<?php
function random_line($filename) {
 $f_contents = file($filename);
 $line = $f_contents[array_rand($f_contents)];
 // fclose($handle);
 $line=str_replace("\r","",$line);
 $line=str_replace("\n","",$line);
 return $line;
}

 $env = $_GET['env'];
 $csvFile = $_GET['csv'];
$line = random_line('data//' . $env . '//' . $csvFile);
 echo json_encode(explode(',',$line)) . PHP_EOL;
?>

The output will look something like this:

["Testuser01","34passw0rd"]

This can now be easily interpreted by JMeter with some regex. With files of thousands of lines the 90th percentile response time of this under some load did not go above 10ms so it would not interfere too much with scripts.

The solution above spews out a random line from the CSV but that could easily be changed. It should also be noted that it doesn’t prevent the same line to be dished out twice! It is unlikely if you have large CSV files but for small files it might definitely be an issue. In order to get around that you’d need to have some sort of thread-safe repository but I doubt most projects would require this.

I’ve also tried to do this in Python and stopped half way through but here is the key line for doing it:

print json.dumps((random.choice(open("testfile.csv").readlines()).splitlines())[0].split(","))

Definitely is more elegant but missing some of the error catching from the PHP bit.

So I hope this might help some people that are facing the same issues out there.

Author: Oliver Erlewein