As the user base grows, software developers must consider how to ensure that their services are scalable enough to handle a large number of concurrent users. It is critical to make sure that the application's performance and reliability do not deteriorate, especially when it is subjected to heavy usage.
To accomplish this, teams must design their applications with scalability in mind and conduct regular performance testing to identify and address potential bottlenecks or issues affecting the application's ability to scale. Many tools available today could be used for this purpose, but unfortunately, not all of them are easily scalable when a large-scale performance test is required. I highly recommend the K6 Cloud tool if you need to run such tests. Let me explain why.
But first, let's go over the various types of tests you might be asked to run.
What are performance tests?
The International Software Testing Qualifications Board (ISTQB) defines performance testing as an umbrella term that includes any kind of testing determining how well a system or component performs under a particular workload. Aspects evaluated during those tests are responsiveness, stability, and scalability.
As for use cases, different types of performance testing may be useful for specific types of projects depending on the goal. The final decision is usually heavily influenced by the expected traffic pattern on the service.
Load testing – focuses on the evaluation of system performance with an increased number of generated requests sent to the system. It is typically used to understand whether the system is going to handle expected traffic.
Stress testing – focuses on the system’s ability to handle peak loads that exceed the expected workloads. It often increases the traffic in steps to see whether the system will fail even before reaching a certain point. This test is designed to verify whether the system is stable and reliable under extreme usage.
Spike testing – focuses on the ability of a system to handle a sudden increase of requests and its return afterward to the previous state.
Performance testing with K6
There are many options on the market when it comes down to tools for performance testing. One tool that is commonly used for that purpose is K6, an open-source load-testing tool that can simulate real-world user traffic and measure the real-time performance of web applications.
To use K6, you need to install the K6 command-line interface (CLI) on your development machine. After that, it should be possible to write a script with a scenario definition that can be executed locally.
Here’s a simple script example that can be performed using K6:
import http from 'k6/http';
import { sleep } from 'k6';
export const options = {
vus: 10,
duration: '30s',
};
export default function () {
http.get('http://codete.com/blog');
}As you can see above, in our test scenario, I requested K6 to consistently open the Codete blog page over the course of 30 seconds using 10 virtual users.
Running that script should produce output similar to the one shown below, which includes the amount of data received/sent, request duration, failed request count, and iteration duration (The time to complete one full iteration, including time spent in setup and teardown). All of those metrics are meaningful and could be used to determine whether the system is performing as expected.
k6 run .\k6-codete.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: .\k6-codete.js
output: -
scenarios: (100.00%) 1 scenario, 10 max VUs, 1m0s max duration (incl. graceful stop):
* default: 10 looping VUs for 30s (gracefulStop: 30s)
running (0m30.4s), 00/10 VUs, 764 complete and 0 interrupted iterations
default ✓ [======================================] 10 VUs 30s
data_received..................: 464 MB 15 MB/s
data_sent......................: 952 kB 31 kB/s
http_req_blocked...............: avg=34.32ms min=0s med=39.33ms max=1.3s p(90)=54.2ms p(95)=58.71ms
http_req_connecting............: avg=32.07ms min=0s med=39.13ms max=1.06s p(90)=53.78ms p(95)=57.99ms
http_req_duration..............: avg=162.92ms min=66.48ms med=101.11ms max=1.34s p(90)=317.3ms p(95)=412.01ms
{ expected_response:true }...: avg=162.92ms min=66.48ms med=101.11ms max=1.34s p(90)=317.3ms p(95)=412.01ms
http_req_failed................: 0.00% ✓ 0 ✗ 1528
http_req_receiving.............: avg=71.56ms min=0s med=19.6ms max=642.38ms p(90)=195.24ms p(95)=255.27ms
http_req_sending...............: avg=55.17µs min=0s med=0s max=3.47ms p(90)=147.06µs p(95)=512.63µs
http_req_tls_handshaking.......: avg=541.77µs min=0s med=0s max=147.67ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=91.3ms min=57.19ms med=73.66ms max=1.11s p(90)=136.81ms p(95)=165.96ms
http_reqs......................: 1528 50.335437/s
iteration_duration.............: avg=394.75ms min=195.49ms med=332.94ms max=2.22s p(90)=538.25ms p(95)=680.68ms
iterations.....................: 764 25.167718/s
vus............................: 10 min=10 max=10
vus_max........................: 10 min=10 max=10We could also change the script so that the number of virtual users grows over time, as shown below.
import http from 'k6/http';
import { sleep } from 'k6';
export const options = {
stages: [
{ duration: '30s', target: 10 },
{ duration: '30s', target: 25 },
{ duration: '30s', target: 50 }
],
};
export default function () {
http.get('http://codete.com/blog');
}What if we want to run a complex scenario with hundreds of thousands of users? A single K6 instance is unlikely to process that volume of traffic. According to the tool’s documentation, a single instance can handle 30k (up to 40k) VUs, but this is heavily dependent on the test complexity and machine that the instance is located on computation power.
From that point, we have an option of either setting up multiple instances of k6 that need to be synchronized together or using a K6 cloud-based solution.
K6 Cloud
K6 Cloud is a commercial SaaS product that allows you to run k6 test definitions in the cloud. In other words, our previous test scenario definition can be easily copied into the k6 Cloud, making it simple to scale up the number of virtual users and test application performance on complex scenarios with high traffic.
It is relatively simple to use K6 Cloud for performance testing. To begin, you must create an account on https://k6.io/. Then, copy the above-mentioned test script and change the virtual user count for the test scenarios you want to simulate. Depending on the test size that will have to be performed, we can choose the pricing model that best satisfies the client’s needs.
The K6 Cloud tool provides a range of features to help you analyze and understand the performance of your application by visualizing the report with interactive graphs and charts. In addition, you can specify how traffic should be distributed across the desired load zones.
For example, by adding the following part to the options, we will now redirect our traffic to originate from the k6 Dublin load zone.
import http from 'k6/http';
import { sleep } from 'k6';
export const options = {
stages: [
{ duration: '30s', target: 10 },
{ duration: '30s', target: 25 },
{ duration: '30s', target: 50 }
],
ext: {
loadimpact: {
distribution: {
DublinDistributionZone: { loadZone: 'amazon:ie:dublin', percent: 100 }
}
},
},
};
export default function () {
http.get('http://codete.com/blog');
}Let's change the previous scenario and make it a bit more complex by opening all the blog posts available on the main blog page.
import http from 'k6/http';
import { parseHTML } from 'k6/html';
export const options = {
stages: [
{ duration: '30s', target: 10 },
{ duration: '30s', target: 25 },
{ duration: '30s', target: 50 }
],
ext: {
loadimpact: {
distribution: {
DublinDistributionZone: { loadZone: 'amazon:ie:dublin', percent: 100 }
}
},
},
};
export default function () {
var mainBlogPageResponse = http.get('http://codete.com/blog');
var htmlBlogPageResponse = parseHTML(mainBlogPageResponse.body);
var blogEntriesUrl = htmlBlogPageResponse
.find('a')
.toArray()
.filter(elem => elem.attr("href").includes("https://codete.com/blog/") && !elem.attr("href").includes("https://codete.com/blog/category/"));
var uniqueBlogEntriesUrl = [...new Set(blogEntriesUrl.map(item => item.attr("href")))];
uniqueBlogEntriesUrl.forEach(function (url) {
http.get(url);
});
}After running the above script, we should be met with a test report containing a default chart with a performance overview:
As well as a table containing metrics on all the endpoints requested during the test:
K6 Cloud markets themselves as being able to handle up to 1 million virtual users and it will likely be more than sufficient for most customers.
Can K6 Cloud be used for DDoS attacks?
You might wonder if testing tools could be used to launch DDoS attacks. When it comes to DDoS attacks, it is illegal as the K6 Cloud's terms of service state that you are not permitted to use the tool on sites you do not own or manage. However, you can still use it to test the efficacy of your own DDoS protection.
Summary
No one enjoys waiting for a website to load. That is why it is essential to ensure that the developed service will handle certain traffic without impacting the end-user experience. The best way to do so is to performance test your service.
K6 Cloud, in my opinion, is a great tool for that purpose, as it takes care of the infrastructure setup and allows the development effort to be focused more on the test script.
What would be the ideal virtual user count? The answer to that question will depend on your product-specific needs. If you’re unsure or would like our assistance with the test scenario, don’t hesitate to contact us for help – we would be happy to assist you.
