Skip to content

Load testing

Jump to bottom
a-a-hofmann edited this page Aug 28, 2019 · 2 revisions

Introduction and Motivation

To ensure our system will work when the semester starts with the first-semester students we need to test its functionality under load. A "Informatics I" course is about 400-500 students so we should plan for around 700-1k concurrent user to be safe and ensure the performance does not degrade when 500 students try to use the system at the same time, as it can happen shortly before a deadline or as soon that an exercise releases.

For load testing we should only concentrate on scenarios that are most likely to happen at large concurrent loads (e.g. code execution) or the frequency of occurrence of which is high (many users navigating exercises).

Components under test

The system is comprised of 4 major components:

  1. Identity Provider (Keycloak): this component is responsible of allowing users to login and changing
  2. Course service API
  3. Student API
  4. Evaluation API and in particular code execution

Possible bottlenecks

For each component we try to identify the possible bottlenecks performance-wise which might hurt availability and QoS

Identity Provider (Keycloak)

Keycloak provides the following functionalities:

  1. User login
  2. User account management: changing password, adding 2FA
  3. Initial user enrolment through the API from the course service

Of the three only 1 is likely to ever happen in bursts. Since we are using OpenID Connect (OIDC) for authentication using public cryptography, a large amount of JSON Web Token (JWT) signing operations might cause performance degradation. 2 and 3 are rarer occurences/one-offs so we do not need to cover this at first.

Course Service API

Any call to the Course Service API requires verifying the JWT signature. The users/courses/ endpoints simply echos information back to the user so we can use this as a baseline for comparison with the other endpoints.

The other functionalities are:

  1. Browsing courses, assignments and exercises
  2. Loading an exercise (includes file downloads)
  3. Updating a course based on a webhook event from CVS

Scenarios 1 and 2 are likely to happen with many concurrent users and should be tested. Loading exercises with resources such as images are effectively file downloads which are more resource intensive. Scenario 3 is a one-off event which should not impact performance.

Student API

  1. Exercise submission
  2. Fetching a previous submission/submission reloading
  3. Viewing results

both scenarios could happen with many users concurrently and require the DB which might bottleneck the API. However the queries are on the simpler side so it should not pose a problem. This should be tested just to be safe.

Evaluation API (Subcomponent of Student API at the moment)

The heavyweight component in our system. Of the 5 exercise-types present at the moment, code and codeSnippet exercises require spinning up a sandboxed system for evaluation which is a very heavy operation. This should definitely be tested under the load of many users concurrently. Testing many users submitting code exercises should give us an upper bound for the Evaluation API. Depending on time we can test the other exercise types as well.

Load Test Scenarios

Clone this wiki locally