Tag: online examinations

Going paperless: The revised portal page in Online Examinations at ETH Zurich introduced in Spring Semester 2020

In online examinations at ETH Zurich, the portal page is the website students see first when they face the exam computer and is thus the entry point for every online examination with Moodle. A revised portal page was introduced in September 2020, focusing on improvements in several key areas: 1) going paperless, 2) exam administration, 3) visual redesign and 4) technical infrastructure.

Figure of the revised portal page used in online examination with Moodle at ETH Zurich.
Figure 1. Left side: Portal page with filled out form and confirmation dialogue. Right side: Manual with general information about online examinations and Moodle, opened by clicking on the blue i-icon on the bottom left.

Going paperless

Before the revision, students writing an online examination always found a piece of paper at their workspace. The piece of paper contained general information about online examinations and Moodle, served to track the specific computers used by students (students noted their name and computer number on it) and had to be signed (to confirm knowledge that technical problems are to be reported immediately and that screens are recorded).

The preparation and distribution of the paper sheets was time-consuming both on part of the LET staff and examiners. Therefore, the goal  was to provide all required information to students digitally, at the same time ensuring availability not only during, but also before and after an examination, as it is the case with paper. Fortunately, SafeExamBrowser, which is used at ETH Zurich to provide a safe and intuitive exam environment in online examinations with Moodle, includes functionality to easily give students access to specific additional resources. We thus designed a “manual” webpage containing all relevant information that can be viewed by students at all times simply by clicking on the respective icon in the taskbar (see Figure 1, right side).

Instead of noting their name and used computers on the paper, the revised portal page allows students to enter their personal information directly on the webpage using a simple form. The information, including the used computer, is automatically and cleanly stored in a database. In addition, as a replacement for the signature on paper, the portal page was extended by a dialogue presenting information that must be confirmed to be able to proceed to the actual examination (see Figure 1, left side).

Exam Administration

Due to the growing number of online examinations at ETH Zurich, it was getting increasingly difficult to maintain the previous portal page and store the papers in a way to provide quick access to the required information when needed (i.e., association of computers and students). The revised portal page therefore includes a separate space for LET to administer the examinations that are available for students and search used computers on an exam-by-exam basis.

Visual redesign

Finally, while Moodle has undergone a visual overhaul in recent years, the design of the old portal page was lagging behind. In addition to updating the design in general, the theme of the revised portal page is now based on the Moodle theme to also provide a more consistent and streamlined exam environment.

Technical Infrastructure

Similarly, it was also time to update the technical infrastructure of the portal page. It was a plain HTML site that could only be updated by one person at a time to prevent that no changes were accidentally overwritten. In contrast, the revised portal page is a state-of-the-art application with a separate administration space that can be used by multiple users at once. Two load-balanced frontend servers ensure that all requests from the examination clients are reliably processed and all data are stored in a central database on a separate server.

The revised portal page was developed alongside the regular operations of the online examinations service and was planned to be tested in Spring Semester 2020 in a few select examinations. However, when it became apparent that on-site examinations in 2020 were to be conducted under special circumstances due to the Coronavirus pandemic, we decided early on to introduce it more broadly to facilitate the implementation of the hygienic measures by going paperless. Extensive security measures were taken and an organizational as well as a technical fallback was in place at all times, which – as expected – was not needed in the end. After the positive and smooth experience of the first examination session, slight improvements were added for Autumn Semester 2020 and analogous functionalities were developed for setups not using Moodle such as Linux examinations. In the future, the plan is to implement a functionality that allows easy pre-assignment of students to computers by displaying the corresponding name directly on the screen.

If you want to know more about online examinations at ETH Zurich, please do not hesitate to contact online-pruefungen@let.ethz.ch.  

Read more

Posted on ,

How to Multiple Choice with Moodle

Multiple Choice (MC) questions are a popular but challenging to master item format for formative assessments and examinations. They facilitate probing students’ knowledge of facts and understanding of simple concepts and their relationships, particularly in large student cohorts.
MC items consist of a stem followed by a number of options. The stem always ends with a short and concise question. It should be possible to answer this question meaningfully without looking at the options (e.g. ‘which of the following options is true’ cannot be answered meaningfully, whereas ‘which animal species are mammals’ can be answered meaningfully). The question may be preceded by a case-study, central idea, or more elaborate problem statement. The options are short and concise and avoid negations.
MC items can be split into two families: The one-best-answer family and the true-false family. In the one-best-answer family, students have to select a single best answer from a set of options. Options that are not the single best answer we call distractors. In the ETH Moodle, this functionality is available through the SC(ETH) question type. In the true-false family, students have to evaluate each option whether it is true or false. In the ETH Moodle, this functionality is available through the MTF(ETH) and the Kprime(ETH) question type. We discourage using the standard Moodle Multiple Choice and the True/False question types, which are no longer needed.

SC(ETH): Single Choice, One-Best-Answer

Single choice (SC), one-best-answer question types are the most appropriate MC format in most occasions. Students need to select a single best answer from a set of options, usually three to five. Because students need to select the best answer from the options, the options by themselves do not need to be strictly true or strictly false. Instead, the best answer needs to be clearly better than all other options. This facilitates the design of much more nuanced questions than is possible with the true-false paradigm. As an alternative to selecting the best answer, students can cross out options they consider to be distractors. Crossing out distractors may either be just a help for students to find the best answer (scoring method SC 1/0) or be relevant for scoring (scoring method Aprime).

Scoring method SC 1/0 (recommended): Students receive full points if the single best option is selected and no points otherwise. Crossed out distractors are disregarded.

Scoring method Aprime: Students receive full points if the single best option is selected, half points if all distractors save one (and save the single best option) are crossed out, and zero points otherwise.

MTF(ETH): Multiple True-False

When a question or problem requires the evaluation of multiple aspects simultaneously, Multiple True-False (MTF) question types may be more appropriate than the SC format. Students need to evaluate every option from a set of one to many options individually as true or false. This implies that each option by itself needs to be clearly and unambiguously true or false. This is a considerable restriction on the kinds of options one can formulate and makes formulating good options much more difficult. Instead of ‘true’ and ‘false’ one can also choose any other two categories, e.g. ‘blue’ and ‘red’, ‘mammal’ and ‘bird’, etc. The MTF(ETH) question type offers two alternative scoring methods.

Scoring method Subpoints (recommended): Students receive subpoints for each correctly marked option.

Scoring method MTF 1/0: Students receive full points if they marked all options correctly, zero points otherwise. This scoring method is rather unpopular with students – the Kprime question type offers a better alternative (see below).

Kprime(ETH): A Popular Compromise

The Kprime question type is a special variant of the MTF question type. It has been popularized through its use in human medicine. The Kprime question type includes exactly four options and the following scoring method: Students receive full points if they mark all four options correctly, half points if they make one mistake, and zero points otherwise. To use this scoring method, select the Kprime scoring method in the Kprime(ETH) question type.

‘Multiple Choice’: No Longer Needed, Soon to be Retired

Please refrain from using the standard Moodle Multiple Choice question type. The scoring mechanics implemented in the standard Moodle Multiple Choice question type are problematic. Please use the SC(ETH), MFT(ETH), and Kprime(ETH) question types instead. We are preparing the migration of existing Multiple Choice items to the new ETH question types. Until this migration is successfully finished, the standard Multiple Choice question type will remain available.

If you are interested in further information on MC items and their design, we recommend the Didactica Course on Multiple Choice. Please refer to the current program https://www.didactica.ethz.ch/

Read more

Posted on ,