Massive Virtual Assessments: Dealing with Dropped Exams and Frustrated Users
Sofia is 27 years old and has high hopes for her professional future. At the designated time (following the precautions outlined in an endless email), she sat in front of her computer. Nervously, she entered her credentials to access the online exam she had longed for.
Suddenly, the nightmare materialized: an eternal blank screen, an incomprehensible error message, time passing as if nothing had happened, anger and helplessness erupting.
“The website crashed,” “the system went down,” “technology failed,” and other euphemisms are used to refer to these situations, but reality may indicate that the problem was not in “the system” (the software and hardware used).
Sofia experienced this issue along with 5,000 other aspirants who had to take the virtual exam simultaneously. This simultaneous activity is what we call “concurrency”: students who, within a computer system, almost at the same moment, press the login button, choose their department, select and save an answer, or access instructional material.
“Concurrency” is a concept related but different from the “number of registered students” or “active students.” The latter do not necessarily experience the issue of simultaneous activity unless a specific start and end time is imposed (as in an exam or a Zoom-type video conference).
Think of a pedestrian bridge in a medium-sized city: during the day, 5,000 people cross it without any problems. However, the structure was designed to withstand the weight of a certain number of people together, and if that is exceeded, it will eventually collapse. If the 5,000 people who use it throughout the day decide to “converge” on the bridge, it’s easy to imagine the result. Therefore, it would be necessary to invest in a much larger and more expensive bridge or reduce concurrency.
Often, the limitation is not in “the system” itself but in the calculation used to estimate how many “concurrent” people it can support without collapsing.”
For a massive number of them (500, 5,000 or more), it is crucial to invest in a team of technical professionals with experience in large-scale implementations, who will decide (among other things) on:
-
- The platform to be used for delivering the exam and its recommended configurations.
- The type of software associated with this LMS, such as the database and webcam surveillance system.
- The type of servers it will be hosted on and their optimal configurations.
- The types of tests and simulations to be conducted, as well as data measurement and interpretation.
- Implementation, help desk planning, and alternative procedures.
- Clear, omnichannel, multimedia, and ongoing communication with users to guide them in using the software and provide reassurance.
- The total budget to allocate to the project.
As we can already guess, the expected number of concurrent users is one of the factors that strongly impact project costs. If investing in a larger bridge is not an option, the alternative is to reduce the simultaneity of users. In the case of exams:
-
- Abandon the idea of directly transferring in-person practices to the virtual environment.
- Implement a scheduling system throughout the day, with different questions randomly distributed.
- Prevent students from entering at the exact start time of the exam.
- Design the user’s path within the platform to minimize the number of clicks required to begin the test.
- Instruct the student to log out immediately upon completing the exam.
- Avoid single, lengthy exams.
Of course, not all issues will be due to high concurrency: it is possible that Sofia experienced a power outage or an internet service disruption at her home, her computer may be malfunctioning, or it could be infected with viruses. Perhaps the selected software has programming errors, the institution has been targeted by hackers, among many other causes.
However, it is crucial not to underestimate the importance of continuous communication with the user and, above all, the calculation of necessary hardware and software to provide satisfactory responses to users, without failures, without anxiety, and as quickly as possible.
Lic. Ricardo Acosta García
E-learning developer
More on e-learning
Updating Moodle is crucial for safeguarding the security and performance of your educational platform. Updates address bugs, shield against vulnerabilities, and ensure optimal functionality.
Moodle isn't just about installation. Successful elearning requires planning, expertise, and continuous effort, not a quick fix solution.
The widespread adoption of virtual courses introduces new instances of old problems, specifically, 'secure' or fraud-proof assessments.