Digital Examinations

The ‘digital semester’ has started and already had some changes for us concerning digital teaching. So it will probably come as no surprise to anyone that examinations will also have a ‘digital makeover’, as it is still not clear whether attendance examinations can take place or not. For this reason, last Wednesday, May 13th, the Senate passed an amendment to the General Examination and Study Regulations (APSO) which allows among other things “electronic distance examinations” for the next 2 semesters. This will allow some new examination formats to be used so that exams can also be taken from home. Last week, ProLehre published a handbook in which 4 of the new formats are explained in more detail. So that you can get a well-founded opinion of the different possibilities, we have written down the most important ones here.

Important factors

There are different aspects to be considered and to which different weightings can be given when choosing the right examination format. In our summary of the formats, we have tried to address each of these points so that you can decide which forms of examination you would be comfortable with and which you would not. A topic that was frequently raised at the FVV was comparability and the possibility of cheating. An exam at home certainly offers many incentives to cheat. There are also ways to at least make it more difficult to cheat, but these ways often include using a webcam to film the room or the examinee himself. This in turn raises new problems. For example, not every student has a webcam or a stable Internet connection to be able to take such an examination. In a survey conducted by the Faculty of Mathematics, about 8% of the respondents stated that they had an insufficient internet connection. Data protection is also an important keyword here, especially when third-party providers come into play. In addition, students who are stuck abroad and may not be able to come to Munich must also be considered, but of course the different faculties must also be taken into account, as they sometimes have very different requirements regarding capacity and possible task types. Last but not least, one of the most important points is to be mentioned: your health. At no time should you or your relatives be exposed to a more than necessary risk of infection.

Presence examinations

As things stand at present, if the situation does not deteriorate, we may be able to perform presence examinations in lecture halls with distance control. The physics faculty, for example, hopes to be able to make do with attendance examinations, which could be possible with about 450 students. The problem here, however, is that there are currently a large number of students abroad and it is not foreseeable whether they could come to Munich for a presence examination. In the survey conducted by the Computer Science Student Council a few weeks ago, approximately 17% of the 400 participants indicated that they were currently abroad. We currently have no information about the other two faculties, but it can be assumed that some students are also not in Munich. Another aspect of the attendance exams, which will be particularly important for the computer science lectures, is the new capacity of the lecture halls due to the distance regulations. Accordingly, the Audimax only has room for about 70 examiners. For lectures with around 1500 registrations, it is foreseeable that even with rented rooms, the capacities will not be sufficient for pure attendance events. A further factor that must be taken into account in the case of face-to-face events is the increased risk of infection for the examinees. Even if the distance regulations in the lecture halls are observed, there is still a residual risk. Moreover, many students would be forced to use public transport to get to their exams, thus exposing themselves to additional risk. A risk that, for example, students living with or belonging to at-risk groups might not be willing to take.

Oral distance examination

An oral distance examination is an oral examination, some of which were also held before the corona crisis, either as a separate form of examination or as a substitute, e.g. in the case of semesters abroad. The examination would be realised by a video call with the examiner, examinee and observer. Possible tools would be, for example, Zoom or BigBlueButton (a closer look at some of the currently used video conferencing tools can be found here). Advantages of this are that it is a format familiar to many and that one can react quite flexibly to the examination situation (e.g. if the Internet is down). Data protection depends on the tool you use, but is manageable, because the exam is not recorded and therefore only examiners and observers can see you. Cheating can be prevented by an initial 360° recording of your room. This can also only happen with your consent. Hardware such as a camera and a microphone is required (smartphone camera and microphone should be sufficient) as well as a stable internet connection. In individual cases an alternative with presence could take place (of course with the appropriate hygiene measures). However, the major disadvantage is that the workload increases linearly with the students, since examinations can only run parallel to each other to a limited extent. In addition, the exams of the first examinee could not be comparable with the exams of the last examinee due to a limited pool of questions and due to discussions among students. Furthermore, oral examinations are only suitable for certain types of exams, as the number of possible task forms is limited. Also factors such as nervousness of the candidates should not be overlooked in oral distance examinations.

Student presentation as distance examination

These are presentations held online, either as livestream or as an uploaded video. In the case of a livestream, the student must have the necessary hardware and a good Internet connection. In the case of a video stream, mobile phone camera and microphone should be sufficient and the requirements for the Internet are much lower. Data protection problems are limited, for example virtual backgrounds could be allowed for a livestream. Possibilities of cheating are comparable to those of regular presentations. Presentations are certainly an alternative for smaller seminars or lab courses, but this is also not a solution for large lectures.

Supervised computer-based written examinations

Supervised written examinations on the computer involve fillable exams on your computer while a webcam films the examinee and checks for any attempts to cheat. This examination is usually carried out using third-party programs (e.g. Proctorio), which use AI to analyse, for example, the line of vision or the desk of the examinee. Open book exams are not possible, however, as averting the gaze can be considered an attempt to cheat. It is important to mention that not every view out of the window is automatically evaluated as 5.0 cheating. After a certain time of not looking at the screen a suspicious situation is reported, which can be confirmed later by the recording or not. A great advantage of this method is that even large lectures could hold exams in this way and students outside of Munich also have the opportunity to take part in exams. The disadvantages, however, are the hardware and internet requirements. In addition, Proctorio stores and processes your data, which is a considerable data protection problem. Furthermore, the processing on the computer only allows multiple choice or free text tasks, which makes it hard to implement for example for math exams.

Supervised written exams with pen and paper

The tasks are uploaded to Moodle at a fixed time and can be answered by the examinee with pen and paper. During the processing, the examinee is monitored by a video conference with an exam supervisor. The possibilities for cheating are thus comparable to a normal examination in a lecture hall with supervision. After the examination has been processed, a photograph of the test is sent to the examiner and the original is later sent by mail. The number of students who can take this exam depends strongly on the number of possible exam supervisors, but could also be suitable for large lectures. The advantage here is that almost all familiar types of tasks are possible, so all 3 departments could benefit from this. However, filming of the examinee is also indispensable here, possibly also recording, but the recordings would have to be deleted when the grades are announced. Also the hardware and internet requirements hardly differ from the other formats.

Exercises during the semester

Exercises performed during the semester are partly already implemented in the form of lab work or grade bonuses and could be expanded this semester. During the semester, some projects or tasks are worked on which then (possibly in connection with a presentation) make up the overall grade. A big advantage is the independence from hardware or internet connection. This should also help students in other time zones. The disadvantage is that the semester has already started and therefore a concentrated workload could arise towards the end of the semester. Also for trainers and tutors there is a considerable amount of work involved in correction. In computer science subjects, programming tasks with already known plagiarism software could be checked for cheating, but this project-based work is difficult in mathematics and physics and fraud is very difficult to avoid. Also unofficial group work cannot be prevented.

Concluding words

Digital exams bring a lot of challenges. However, it should be possible to solve some of the problems. For example, one could try to buy hardware to lend or provide places in the computer halls. Mixed forms of the different types of exams are also imaginable. Fortunately, the TUM has been very accommodating in the current situation. So if you don’t want to write your exams, e.g. because of data protection problems, you should have the possibility to repeat your exam at a later date under more normal conditions without any disadvantages regarding the study progress review. Unfortunately, there is probably no such thing as the perfect solution, but we are working hard to achieve the best result for you. For this reason we need your support. What are your worries and concerns? Maybe even what kind of solutions do you have in mind? For this reason, the student council of the Department of Computer Science has set up a survey about how computer science students think about digital exams. The SC Physics and the SC Mathematics are also happy about your feedback!

Facebook: https://www.facebook.com/FSMPI.TUM Twitter: https://twitter.com/mpi_tum Instagram: https://www.instagram.com/mpi_tum/ Telegram Newsletter: https://t.me/fsmpi Telegram bulletin board: https://t.me/mpi_aushaenge

Image: Nick Morrison https://unsplash.com/photos/FHnnjk1Yj7Y