jmeter

JMeter is a load / stress tool built in Java which allows you to simulate multiple user connections to your system and monitor how the application & hardware response to heavy load.

In STARLIMS, I find it is a very good tool for performance optimization. One can detect redundant calls, chatty pieces of code and identify bottlenecks, even when running with a single user.

As a bonus, Microsoft has a preview version of load tests based on JMeter, which can be integrated to your CI/CD process!

So, in this article, my goal is to help you get started – once setup, it’s very easy to do.

I will proceed with the following assumptions:

• You know your way around STARLIMS
• You have some scripting knowledge
• Your STARLIMS version is 12.1 or + (I leverage the REST API introduced with 12.1. It is possible to do differently, but that will be out of scope)
• Xfd is the most difficult technology for this. Therefore, that’s what I will tackle. If you are running on HTML, it will be just easier, good for you!

Environment Setup

On your local PC

• Install Java Runtime – you might have to reboot. Don’t worry, I’m not going anywhere!
• Download JMeter and extract it somewhere (remember where!)
• Make sure you have access to setting up a Manual Proxy. This can be tricky and may require your administrators to enable this for you. What you’ll want is to be able to toggle it like this (don’t enable it just yet! Just verify you can):

On your STARLIMS Server

• Make it available through HTTP. Yes, you have read properly, HTTP. Not HTTPS. I think it can work HTTPS, but I ran into too much problems and found out HTTP is easiest. This is to simplify traffic recording when recording a scenario for re-processing.
• Create your load users. If you expect to run 100 simultaneous users, then let’s create 100! What I did is create users named LOADUSER001 to LOADUSER250 (so I would have 250 users) and have their password to something silly like #LoadUser001 to #LoadUser250. Like I said – don’t do this if there’s any sensitive data in your system.
• To help you, here’s a script to generate the users:

:RETURN SubmitToBatch("LoadTestPrep.UserCreator.ASync", { 100 });

:PROCEDURE Async;
:PARAMETERS nNumberOfUsers;
:DEFAULT nNumberOfUsers, 1;

:DECLARE sUserName, sOldPassword, sNewPassword, i, nOrigrec, oNewUser, aUserDetails, pwEncOld, pwEncNew;
resp := "nothing yet";
:FOR i := 1 :TO nNumberOfUsers;	
	oNewUser := CreateUdObject();
	oNewUser:USRNAM := "LOADUSER" + StrZero(i, 4,0);
	oNewUser:FULLNAME := "Load User " + StrZero(i, 4,0);
	oNewUser:JOBDESCRIPTION := "Load Test";
	oNewUser:EMAIL := "user" + StrZero(i,4,0) + "@dummy.com";
	oNewUser:LANGID := "ENG";
	oNewUser:POWERUSER := "Y";
	oNewUser:TREEAUTH := { "L" };
	oNewUser:RASCLIENTID := "Internal";
	oNewUser:DEPTLIST := "Changzhou";
	oNewUser:QUESTION_ID := 1;
    oNewUser:ANSWER := "1234";
    oNewUser:CONFIRMANSWER := "1234";
    oNewUser:PIN := "1234";	
	oNewUser:Id := "UserManagement.newUserModel-" + LimsString(i);
	
	UsrMes("Processing " + oNewUser:USRNAM);
	resp := ExecFunction("UserManagement.createNewUser", { oNewUser });
	
	resp := "User " + oNewUser:USRNAM + " does not exist";
	nOrigrec := LSearch("select ORIGREC from USERS where USRNAM = ?", 0, "DATABASE", { oNewUser:USRNAM });
	:IF nOrigrec > 0;
		aUserDetails := {
			{
				"TREEAUTH",
				{
					"L"
				},
				"S",
				{"L"}
			},
			{
				"SHOWERRORDETAILS",
				"Y",
				"S",
				"N"
			},
			{
				"STATUS",
				"Active",
				"S",
				"Pending"
			}
		};
		
		pwEncOld := "#LoadUsr" + StrZero(i, 4, 0);
		pwEncNew := "#LoadUser" + StrZero(i, 4, 0);
		ExecFunction("UserManagement.saveUserDetails", { NIL, "USERS", aUserDetails, nOrigrec });
		ExecFunction("Security_Module.ChangePassword", { oNewUser:USRNAM, "NEW", pwEncOld });
		ExecFunction("Security_Module.ChangePassword", { oNewUser:USRNAM, "", pwEncNew });
		resp := ExecFunction("UserManagement.updateHTMLUserSecurityInformation", {NIL,"USERS",{{"PWEXPD",Now():AddYears(100),"D",Now()}},nOrigrec,{}});
	:ENDIF;
:NEXT;

UsrMes( "Done" );
:ENDPROC;

You will need to test the above, on my system it worked fine (haha!) but setting password and security is not always working as expected in STARLIMS; so do not despair – just be patient.

• Edit the web.config file. I will presume you know which one and how to achieve that. You need to change / add the following appSetting to false: <appSetting name="TamperProofCommunication" value="false" />

• Add Endpoint to Encrypt function. That’s really the tricky part. In both XFD and HTML, STARLIMS “masks” the username and password when putting it in the payload for authentication, to prevent sending in clear text. But this encryption is significant; it is part of .NET and not easily integrated to JMeter… Unless it becomes a REST API endpoint!.
• So, in a nutshell, the trick is to create a new API Endpoint that receives a string and a key, and call the EncryptData(text, key) function, and return the encrypted string. I will not stress it enough: do – not – enable – this – on – a -system – with – sensitive – data. And make sure you will only use load testing users. If you do so, you’re fine.

This is the code of the REST API method to expose from STARLIMS:

:PROCEDURE GET;
:PARAMETERS payload;
:DECLARE response;

response := CreateUdObject();
response:StatusCode := Me:HTTP_SUCCESS;
response:Response := CreateUdObject();
:IF payload:IsProperty("text") .and. payload:IsProperty("pw");
    :DECLARE t, p, secret;
    t := limsString(payload:text);
    p := limsString(payload:pw);
    secret := EncryptData(t, p);
    response:Response:message := secret;
:ELSE;
    response:Response:message := "Missing data";
    response:StatusCode := 500;
:ENDIF;

:RETURN response;
:ENDPROC;

Since it gets exposed as a REST API, the concept is that at the beginning of the load test, for every user, we call this with the username and the password to get the encrypted version of each, which allows us to tap into STARLIMS cookie / session mechanism. Magic!

Now, we are kind of ready – assuming you’ve followed along and got everything setup properly and were able to test your API with POSTMAN or something like that. Before moving on, let’s take a look at a typical load test plan in JMeter:

The idea is we want each user (thread) to run in its own “session”. And we want each session to be for a different user. My scenarios always involve a user login into STARLIMS once (to create a session) and the to loop on running the scenario (for example, one scenario could be aboout creating folders, another scenario about entering results, etc.) . I will leave to you the details of the test plans, but the idea is you first need to login the system, then do something.

At the level of the test plan, let’s add user-defined variables – in my case, this is only so I can switch STARLIMS instances later on (I strongly recommend you do that!):

Always at the level of the test plan, add a counter:

This will be the magic for multiple users. Note the number format – this has to match your user naming convention, otherwise, good luck!

Now, let’s have our user login STARLIMS.

1. Add a Transaction Controller to the Thread Group. I renamed this one “System Login” – call it what you want.
2. On your new transaction controller, add a Sampler > HTTP Request, which will be our call to the REST API

As you can see, I did a few more things than just call the API. If we break it down, I have a pre-processor “Initialize User Variables”, a HTTP Header Manager, and a JSON Extractor. Let’s look at each of these.

Pre-processor – Initialize User Variables (Beanshell preprocessor)

This will run before this call is made – every time this call is made! This is where we initialize more variables we can use in the thread.

currentUser = "LOADUSER" + "${un}";
s = "000" + "${un}";
v = s.split("");
s = s.substring(v.length - 4);
currentPw = "#LoadUser" + s;
vars.put("currentUser", currentUser);
vars.put("currentPW", currentPw);
vars.put("startFolderNo", "LT22-000" + "${un}");
log.info("Current User: " + currentUser);

This will initialize the currentUser and currentPW variables we can reuse later on. Since this is a pre-processor, it means the request can reference them:

Now, let’s look at the HTTP Header Manager:

Pretty simple – if you have STARLIMS 12.1 or +, you just need to get yourself an API key in the RestApi application. Otherwise, this whole part might have to be adjusted according to your prefered way of calling STARLIMS. But, long story short, SL-API-Auth is the header you want, and the value should be your STARLIMS secret API key.

Finally, this API will return something (the encoded string). So we need to store it in yet another variable! Simple enough, we use a post-processor JSON extractor:

What did we just do? Here’s a breakdown:

1. Initialized a user name and password in variables
2. Constructed a HTTP request with these 2 variables
3. Called the REST API with our secret STARLIMS key using this request
4. Parsed the JSON response into another variable

If you have set the thread group to simulate 10 users, then you’ll have LOADUSER001 to LOADUSER010 initialized. This is the pattern to learn. This is what we’ll be doing all along.

Wait. How did you know what to call afterward?

Great question! That’s where the proxy gets into play. Now, we don’t want to go around and guess all the calls, and, although I like Fiddler, I think it would be very complicated to use.

In a nutshell, this is what we’ll do:

1. We’ll add a Recording Controller to our Thread Group
   1. Right-click on your Thread Group > Add > Logic Controller > Recording Controller
2. We’ll add a Test Script Recorder to our Test Plan
   1. Right-click on your Test Plan > Add > Non-Test Elements > HTTP(S) Test Script Recorder
   2. Change the Target Controller to your recording Controller above, so you know where the calls will go
3. We’ll activate the proxy (bye bye internet!)
   1. Open Windows Settings
   2. Look for Proxy
   3. Change Manual Proxy > Use a proxy server to on.
   4. Local Address = http://localhost
   5. Port = 8888
   6. Click Save! I didn’t realize at first there was a save button for this…
4. We’ll start the Test Script Recorder

5. We’ll peform our action in STARLIMS
   1. WARNING: A good practice is to change the value of Transaction name in the Recorder Transactions Control as you progress. What I typically do is put SYSTEM_LOGIN while I launch STARLIMS. Then SYSTEM_LOGIN/VALIDATE when I enter credentials, then SYSTEM_LOGIN/OK when I click OK, etc.
   2. If all works well, you should see items being added to your Transaction Recorder.
6. We’ll stop the Test Script Recorder – just click on the big red Stop
7. We’ll deactivate the proxy (yay!) – just toggle it off.

You should have something like this in your recorder:

If, like me, you let your Outlook opened, you will have all kind of unrelated HTTP calls. Just select these and delete them. You should be left with something like this:

Now, let’s understand what happened here. We recorded all the calls to STARLIMS. If you wish, you can remove the GetImageById lines – typically, this should not have any performance impact as these should be cached. But heh, that’s your call.

Let’s look at the 1st request:

Interestingly enough, we can see the Protocol is http, and the Server Name is our STARLIMS server. If you created user defined variables, then you can just clean these 2 fields up (make them empty). We can default them at the test plan level (later on). But if you do that, you must do it for all requests! So, let’s not do this (just yet). Let’s leave it as is.

Now, what we want, is to re-run this so we can have actual data to work with and to make our script dynamic. But we need to record all the requests sent and received.

Right-click on your Thread Group > Add > Listener > View Results Tree

I find this listener to be the best for this activity.

Now, let’s run this “as is” clicking the play button

The beauty here is you can get the data sent to STARLIMS as well as the responses, allowing us to understand how everything is connected. Let’s take a look at the Authentication.GetUserInfo – that’s our first challenge:

If you look at the Request Body, you’ll see your user name (which you used to login), as well as a 2nd very strange parameter that looks like the above highlighted string in kind of pink. Now, when we log into STARLIMS, we must send that string, which, essentially, is the password hash based on the user name (one-way encoding). So the question is: how do we get this? This is where our REST API, which we prepared earlier, comes into play!

With this, you can do everything now! Well, as far as load testing is concerned, it can at least get you started!

Earlier, I mentioned you shouldn’t leave your Server name / path / protocol in there. Indeed, in my screenshot above, you can see it’s all empty. This is because I added a HTTP Request Default to my test plan:

You’ll also want a HTTP Cookie Manager. This one doesn’t need configuration as far as I know; but it must exist so cookies are carried over.

CONCLUSION

What?? Conclusion already? But we were just getting started! Well, don’t worry. I have done a little bit more than just that, and I am including it with this post.

You can get a semi-working test plan here.

You will need to figure out a few things, like some of the APIs I use and some forms/scripts that you won’t have. But this should give you a very good overview of how it works and how it is all tied in together.

As a side note, the reason I got involved into this was caused by Microsoft adding JMeter as part of the tools in their load test preview!

Hope you find good use to this!