JMeter Response Assertions

For those of us who work with JMeter and performance tests, we know the importance of adding validations to all HTTP Requests. Validations are not just a way to confirm that the server is responding as expected, but they also serve to verify that the script we are developing is acting as it should, meaning, it is simulating the user behavior as it was designed.

For example, if running our script on the system implies that a record is created in the database, we have to ensure that after running the script, the record has indeed been created and that the script doesn’t say everything went smoothly if it hadn’t actually created it. That would be an example of a bad choice of an assertion or validation of our requests.

We want to avoid false negatives and false positives. Here’s a review of what those mean:


● 
False Positive: The test says there is an error when there really isn’t one and then we lose time looking for the cause of the error until we realize that the error was just in the test, or in the environment, or the data, etc.

False Negative: There are errors but they are not found. Since they were not found, we believe that these features have been fully covered and they are bug free although they are not.

If there is an error, it’s important that the script fails. Conversely, if it goes well, the script has to confirm that it went well and we have to be able to trust it.

How do we select good assertions?

The reality is that I have not found a guide on how to choose these, and it always stumps me when I am teaching JMeter to someone and they ask me how do I know what to validate in the response of an HTTP request?

As I mentioned in the post, “Checklist: 8 Ways to Improve JMeter Scripts,” Just validating a response code is not enough, because we can lose other errors that we need to capture in order to act accordingly. 

Therefore, today I am sharing with you the methods and criteria for selecting which JMeter response assertions to make when validating the responses of the server. 

1. Validate the login

Typically, after authenticating a system, a “Welcome @username” message appears and our name appears somewhere on the screen, indicating that the login was successful. In these cases, we have an excellent element to validate.

Let’s see this with an example:

1) I log in with the user’s credentialsuser login screenshot2) I successfully enter the site and it shows the user name on the screenlogged in user3) I search for how the element appears in the response HTMLHTTP response request4) I validate it in JMeterJMeter validation

Recall that in JMeter, we can specify a variable in the validations, then, in cases where the user name is parameterized (read from a csv file or defined as a user variable), one can verify the text ${UserName} and JMeter will replace the variable value.

2. Validate window content

Many other times requests return a page in which you can choose text/titles that should come in the HTML response. For example, if we access the JMeter site, (http://jmeter.apache.org/) we can see the following:JMeter website

We know that if this page loads correctly, the title “What can I do with it?”  should appear in the response to the request. If it doesn’t appear, then you know you have an error, so it’s a good assertion to add as a validation that we are getting the expected page.

The challenge in this type of validation is to verify content that we know that, if there is an error, then that text does not come in the response. We can rely on developers to provide us with this kind of information.

3. Consider the next steps in the script

If we know that the next action to take is click a button X, or access the link Z, or select in the combo Option B, a very good strategy is to validate that the element on which we operate in the next step, appears in the response of the request. For example, if after loading the JMeter site, I want to select the option “Download Releases” I can validate this text and I assure the flow continues.

For this, I capture the response from the server verifying that it includes the option I want to check: server response in JMeter

And then I create the validation in JMeter:

JMeter validation

4. Validate values that need to be extracted

There are times when we obtain variables from the response of an HTTP request through the use of regular expressions (Add -> Post Processors -> Regular Expression Extractor) with the intention to use that variable in a subsequent request.

We can then validate that the variable appears in the server response.

For this we can make use of validations that use regular expressions.

For example:

JMeter response assertion

Thus, if the regular expression doesn’t match in the response, we will get a validation error.

response assertion in JMeter example

5. Take special care with special characters

Be careful when writing the assertion in JMeter because the text is not always given in the same format that is displayed. For example, to validate the title Apache JMeter ™, I cannot write it verbatim because the server response comes as Apache JMeter & trade; and not the other way as seen on the screen. If you do not use the text as it appears in the HTML, the validation (and regular expressions) will not find it.

6. Use a proper amount of assertions

JMeter allows us to add as many assertions as the we want, so that each of the assertions that were named above may be accompanied by other assertions such as the titles of the HTML responses (<title> … </ title> ), in order to add strength to the validation. When adding more assertions, we have to be careful that the script is not too loaded, which is why it’s not advisable to add more than two assertions per request, so we must ensure that the ones we choose are determining factors in defining the correctness of the response.

We hope that this list will be useful for you when you add assertions. Can you share any other ideas you may have?

If you are looking for more help with JMeter and BlazeMeter, we are now offering a one month certification course for these tools! 


Check out these related posts

Introduction to Taurus
Designing Performance Tests with a Little Bit of Futurology


Tags: , , , , ,