Educational innovation, development and discussion at ETH
Vereinfachtes Reporting in der Unterrichtsbeurteilung der ETH Zürich
An vielen Hochschulen sind die Prozesse der Unterrichtsbeurteilung eindeutig geregelt. Dies aber meist nur bis zum Ende der Datenerhebung. In den letzten Jahren rückte der Schlussteil des Evaluationsprozesses, also die Analyse der Resultate und die Festlegung von Massnahmen aufgrund dieser Resultate, vermehrt in den Fokus. Bisher mussten die 16 Departemente pro Semester je einen Bericht an die Rektorin senden, in dem die kritisch evaluierten Veranstaltungen eruiert und Ursachen und Massnahmen zu deren Verbesserung beschrieben wurden. Die Rektorin hat dann wieder in Briefform eine Rückmeldung an jedes Departement zum Bericht geschrieben.
Mit einer Weiterentwicklung unserer Evaluationssoftware EvaSys können wir nun diesen Feedback-Prozess mit einer Online-Applikation unterstützen. In einem ersten Schritt werden die kritisch evaluierten Lehrveranstaltungen und Prüfungen automatisch aufgrund von flexibel einstellbaren Kriterien erkannt und den Departementen in einer Liste präsentiert. Diskussionen, ob eine Lehrveranstaltung oder Prüfung nun kritisch evaluiert ist oder vergessen wurde, gehören so der Vergangenheit an. Kritisch bedeutet, dass das Departement bei der entsprechenden Lehrveranstaltung oder Prüfung genauer hinschauen und eine Rückmeldung an die Rektorin geben muss. Dieses «genaue Hinschauen» ist nun in einem eigenen Menü, dem Massnahmendialog, abgebildet, wie er in der Abbildung unten ersichtlich ist.
Somit erhält das Departement für jedes Semester eine Liste von kritisch evaluierten Lehrveranstaltungen und Prüfungen. Der Massnahmendialog muss für jede kritische Lehrveranstaltung oder Prüfung aufgerufen und ausgefüllt werden. Die Suche nach kritisch bewerteten Veranstaltungen, die in den Departementen bisher manuell erfolgt ist, entfällt damit. In der Liste ist zudem ersichtlich, ob die entsprechende Lehrveranstaltung oder Prüfung schon in früheren Semestern kritisch evaluiert war. Somit lassen sich Trends wie die Verbesserung einer kritischen Veranstaltung besser nachvollziehen. Am Ende der Liste der kritischen Lehrveranstaltungen und Prüfungen muss das Departement noch ein Fazit über die Unterrichtsbeurteilungen des aktuellen Semesters eingeben, zu welchem die Rektorin auch in einem Feld ein Feedback geben kann.
Die Weiterentwicklung unterstützt den Feedbackprozess zwischen Departementen und Rektorin. Auf umständliche Berichterstattung und Korrespondenz in brieflicher Form kann in Zukunft verzichtet werden. Das Erkennen von kritischen und von wiederholt kritischen Lehrveranstaltungen und Prüfungen ist automatisiert, was den administrativen Aufwand wesentlich verringert und der inhaltlichen Arbeit an der Verbesserung der Qualität von Lerneinheiten und Prüfungen zugutekommt. Natürlich ist mit der Weiterentwicklung auch die Hoffnung verbunden, dass der Feedbackprozess in Zukunft etwas beschleunigt werden kann.
Going paperless: The revised portal page in Online Examinations at ETH Zurich introduced in Spring Semester 2020
In online examinations at ETH Zurich, the portal page is the website students see first when they face the exam computer and is thus the entry point for every online examination with Moodle. A revised portal page was introduced in September 2020, focusing on improvements in several key areas: 1) going paperless, 2) exam administration, 3) visual redesign and 4) technical infrastructure.
Before the revision, students writing an online examination always found a piece of paper at their workspace. The piece of paper contained general information about online examinations and Moodle, served to track the specific computers used by students (students noted their name and computer number on it) and had to be signed (to confirm knowledge that technical problems are to be reported immediately and that screens are recorded).
The preparation and distribution of the paper sheets was time-consuming both on part of the LET staff and examiners. Therefore, the goal was to provide all required information to students digitally, at the same time ensuring availability not only during, but also before and after an examination, as it is the case with paper. Fortunately, SafeExamBrowser, which is used at ETH Zurich to provide a safe and intuitive exam environment in online examinations with Moodle, includes functionality to easily give students access to specific additional resources. We thus designed a “manual” webpage containing all relevant information that can be viewed by students at all times simply by clicking on the respective icon in the taskbar (see Figure 1, right side).
Instead of noting their name and used computers on the paper, the revised portal page allows students to enter their personal information directly on the webpage using a simple form. The information, including the used computer, is automatically and cleanly stored in a database. In addition, as a replacement for the signature on paper, the portal page was extended by a dialogue presenting information that must be confirmed to be able to proceed to the actual examination (see Figure 1, left side).
Due to the growing number of online examinations at ETH Zurich, it was getting increasingly difficult to maintain the previous portal page and store the papers in a way to provide quick access to the required information when needed (i.e., association of computers and students). The revised portal page therefore includes a separate space for LET to administer the examinations that are available for students and search used computers on an exam-by-exam basis.
Finally, while Moodle has undergone a visual overhaul in recent years, the design of the old portal page was lagging behind. In addition to updating the design in general, the theme of the revised portal page is now based on the Moodle theme to also provide a more consistent and streamlined exam environment.
Similarly, it was also time to update the technical infrastructure of the portal page. It was a plain HTML site that could only be updated by one person at a time to prevent that no changes were accidentally overwritten. In contrast, the revised portal page is a state-of-the-art application with a separate administration space that can be used by multiple users at once. Two load-balanced frontend servers ensure that all requests from the examination clients are reliably processed and all data are stored in a central database on a separate server.
The revised portal page was developed alongside the regular operations of the online examinations service and was planned to be tested in Spring Semester 2020 in a few select examinations. However, when it became apparent that on-site examinations in 2020 were to be conducted under special circumstances due to the Coronavirus pandemic, we decided early on to introduce it more broadly to facilitate the implementation of the hygienic measures by going paperless. Extensive security measures were taken and an organizational as well as a technical fallback was in place at all times, which – as expected – was not needed in the end. After the positive and smooth experience of the first examination session, slight improvements were added for Autumn Semester 2020 and analogous functionalities were developed for setups not using Moodle such as Linux examinations. In the future, the plan is to implement a functionality that allows easy pre-assignment of students to computers by displaying the corresponding name directly on the screen.
Examinations in exceptional times: one year of remote written examinations in Moodle during the COVID-19 pandemic
When it became apparent in the Spring Semester 2020 that on-site examinations were not possible during the semester due to the COVID-19 pandemic, alternative performance assessments without requiring students to be physical present were needed. In collaboration with LET, ETH then soon developed the legal and organisational and didactical framework to ensure that examinations during and at the end of the semester could also be carried out in this exceptional situation.
According to the directive “COVID-19: Measures with regard to teaching” for Autumn Semester 2020 and Spring Semester 2021, high-stakes written examinations during the examination sessions are to take place on site under a strict safety concept. In contrast, written examinations during or at the end of semester in the context of end-of-semester examinations, semester performances and continuous performance assessment tasks can either be conducted on site using a safety concept or as a remote examination. However, a remote version of the examination must always be planned as a backup, in case the government regulations change.
The shift from classical written exams in presence to remote written examinations posed a challenge for everybody. To help teachers to adapt their examinations to an appropriate remote format, LET compiled an overview of alternative ways of assessment. The focus of this text is on one such scenario: written examinations conducted remotely on Moodle.
Long before COVID-19, ETH has been using a separate Moodle instance (Exam Moodle) as an online-platform for summative on-site examinations on desktop computers and mobile devices. Therefore, fortunately, the infrastructure was already in place, although additional functionalities were integrated to better accommodate examinations that are conducted remotely (e.g. a plugin to request a declaration of originality).
In the course of the last two semesters, LET has supported over 200 remote written examinations in Moodle with 14’000 individual performances. Throughout this year, we have found several key factors for successfully conducting remote examination in Moodle.
Useful advice before you start preparing your exam
Setting up a course on the Exam Moodle and coordinate technical support well in advance: Send an email to email@example.com if you would like to conduct a remote examination in Moodle as early as possible. We will create a course for you on the Exam Moodle, help you setting up your examination and coordinate how you can reach us as technical support during the examination.
Keep the “open-book” character in mind while designing the exam questions: Even though certain practices have to be applied in order to prevent students to easily communicate the solutions among each other (e.g. open-ended questions, randomization of closed-ended questions), the use of the Internet cannot be prevented. For this reason, you either create questions that cannot be easily looked up on the Internet or explicitly include the Internet search as part of the examination.
Best practice for preparation and conduct
Take advantage of the new opportunities that arise:
Although preparing a remote examination in Moodle may seem challenging at first, it also holds the opportunity for new and creative ways to design an examination.
The biggest opportunity with remote examinations in Moodle is to embrace the fact that students have access to additional resources, such as databases and software, and make it an integral part of the examination. In this way, it is possible to assess the deep understanding of a topic and the application of knowledge rather than mere fact retrieval. This is ideally supported by open-ended rather than closed-ended questions, since the former give students the possibility to present their knowledge in detail. However, notwithstanding all the didactical opportunities, it is crucial that the alignment of the learning objectives with the examination is still maintained.
2. Give students the possibility to navigate freely:
We strongly recommend refraining from using a predefined navigation for the following reasons:
Students are not able to get an overview of the whole exam first
Students are not able to choose which part of the examination they want to begin with. Stress levels rise when students have no choice but to start with a topic they find difficult.
As students are not able to navigate backwards, they cannot revise and adjust their given answers. This is particularly relevant in the case of a momentary blackout.
Overall, this creates additional stress for students in an already stressful situation and generally leads to an examination that tests stress resistance rather than the actual learning outcome.
3. Use Zoom as a tool for communication
First things first: You do not have to use or proctor your students with Zoom during a written remote examination in Moodle. However, Zoom is a great way to stay in touch with your students throughout the examination and we recommend using it as your primary communication channel. It allows you to communicate information quickly to all students and students can contact you via chat if they have a question or problem. The students’ cameras do not have to be switched on for this.
The main advantages over other communication channels such as email or phone are the immediacy of communication and the possibility of screen sharing. The latter is particularly useful when technical difficulties arise.
However, it is crucial to establish a second, internet-independent communication channel (e.g. phone). This way, students who lose their internet connection can still contact you.
4. Things to consider for setting up your Zoom meeting
Once you have decided to use Zoom during the remote examination in Moodle, there are a couple of considerations and preparations that should be made beforehand:
Distribute roles: make sure everybody involved in the examination knows what their responsibilities are during the exam. For example, who is responsible for questions in the chat? Who will make the announcements and share the password for the exam? Who will handle breakout rooms?
Chat settings: make sure that students can write only to you and the Co-Hosts in the Zoom chat. The setting to chat with all participants of the meeting should be disabled.
Breakout rooms: including breakout rooms in your Zoom meeting is very helpful in supporting participants with questions or problems, without disturbing the other participants.
5. Conduct a mock examination
A mock exam reduces potential technical difficulties in advance and gives students the opportunity to familiarize themselves with the Moodle exam platform. This way, students already know how to access the examination on the exam day and are familiar with the interface. In order to achieve this, the mock examination does not have to be elaborate. It can also consist of general questions that are not specific to your subject.
6. Make sure that students contact you immediately in case of technical problems
You cannot stress this enough! If a student faces any technical difficulties and cannot continue working on the examination, the student has to contact you immediately. Only if they inform you immediately, it is possible to extend the examination time by the time lost due to the problem. If students inform you after the examination that they were unable to continue working, this is no longer possible.
If you would like to conduct your examination as a remote written examination in Moodle or would like to discuss the possibilities for your remote examination in Moodle, please do not hesitate to contact us via firstname.lastname@example.org.
The lecture series «Pharmazeutische Fallbeispiele» [Pharmaceutical Case Studies] is a compilation of seven 2-hour sessions for around 75 students of the BSc Pharmaceutical Sciences. Surrounded by lectures and lab work on basic science and pharmacotherapy in the third year of studies, our autumn series aims to showcase the complexity of and our fascination for later pharmacy practice issues, giving the students a new perspective on all the other courses’ material as well.
One of our primary learning objectives states that students should be able to analyse simple case studies from pharmacy practice and present, explain, and discuss them in plenary, based on their current pharmaceutical knowledge. To let students achieve this objective, we had already included group work and presentations, where they discuss their thoughts on a given case study with our and their own literature resources (e.g., which drug class is most appropriate for which kind of nausea).
In 2019, we realized that student participation dropped towards the end of the semester when the big exams of the other courses approached. The sessions were mainly visited by the presenting student groups, whilst their peers focused on learning for ECTS.
2020 challenged us to go digital. This simultaneously provided the opportunity to have Moodle supporting us in our different teaching elements. The Moodle group selection allowed our students to choose their own peers. Folders, surrounded by explanatory text, helped us in embedding the asynchronous learning material (i.e., preparatory reading).
Most importantly, however, we created a simple quiz with four questions for each of the seven 2-hour sessions, focusing on the day’s learning objectives. We specifically aimed to include questions on the preparatory reading, our frontal inputs, the presentations by their peers, and one additional pharmacy practice issue. Moodle badges allowed for a simple gamification of the quiz. We shortened our lessons and offered the time to complete the quizzes during the two hours to not increase the overall student workload.
Student feedback was overwhelmingly positive: They appreciated our efforts concerning the Moodle course, liked the variation with peer presentations, stated having fun completing our quizzes, and were happy about the interactive segments. There were still fewer students present live towards the end of the semester, but the completed quizzes suggest a shift towards asynchronous learning by watching the recordings when taking a break from learning for ECTS.
We will most certainly keep our Moodle course even when going back to physically present teaching. The students seemed to be engaged in the course material by asking us interested follow-up questions concerning the preparatory reading and even our quizzes. The administrative work for setting up the course was hefty, but well worth it. One advise to my previous self: Cramped shoulders won’t help you in troubleshooting issues in the Moodle group selector faster.
Arbeiten in der Abteilung LET während der Pandemie
Die Corona Pandemie hat auf der ganzen Welt zu Veränderungen und Anpassungen geführt. Dies trifft auch auf die Abteilung LET zu. Im August 2020 habe ich im Rahmen meiner Ausbildung zum Informatiker Fachrichtung Systemtechnik von Young ‘n’ Rising in die Abteilung LET gewechselt. Zu dieser Zeit waren die Meisten im Homeoffice tätig.
Die Abteilung in einer solchen Zeit zu wechseln ist etwas Eigenartiges. Ich hatte das Glück, dass ich mein Team persönlich kennenlernen durfte. Die ersten zwei Wochen war ich an jedem Arbeitstag mit einer anderen Person im Büro. «Kennenlernen» hielt sich hierbei natürlich in Grenzen, da es jeweils nur ein Tag war. Meine Teamkollegen gaben mir jeweils einen Einblick in ihre Arbeiten. So lernte ich was mein Team macht und womit ich in Zukunft arbeiten werde. Ausserhalb meines Teams habe ich drei bis vier Personen mal persönlich im Flur getroffen. Weitere Personen habe ich früher oder später online kennengelernt. Noch heute gibt es viele Mitarbeiterinnen und Mitarbeiter an der Abteilung LET, welche ich noch nie persönlich getroffen habe. Es gibt sogar welche, mit denen ich noch gar nie etwas zu tun hatte. Für gewisse Leute scheint es normal, dass man nicht alle Mitarbeiterinnen und Mitarbeiter aus der Abteilung kennt. Man muss hierbei jedoch bedenken, dass die Abteilung LET mit rund 50 Mitarbeiterinnen und Mitarbeiter eher eine kleine Abteilung ist.
Für mich war das ganze «Online die Leute kennen lernen» nicht wirklich schlimm oder komisch. Ich habe in meiner Freizeit viel mit Computer und den neusten Techniken zu tun und bin mir das deshalb gewohnt. Für mich war es nichts Neues über Tools wie Zoom oder Teams zu kommunizieren. Ich bin nun genau ein Jahr im Homeoffice. Die Tage, an welchen ich seit März 2020 im Büro war, liegen bei unter 20. Für die meisten wäre das nichts, was sie aushalten würden. Ich höre von vielen, dass sie ins Büro wollen um einen Austausch zu haben. Eine Veränderung und einen Perspektivenwechsel. Ich selbst merke das nicht sehr stark. Das einzige was ich eine Zeit lang gemerkt habe war, dass mein Zimmer nicht mehr nur mit Freizeit oder Privatleben zu tun hat. Es ist seit einem Jahr auch mein Arbeitsplatz. Ich kann mich selbst sehr glücklich schätzen, dass ich zuhause das nötige Material habe, um anständig arbeiten zu können. Ich musste mir aufgrund des Homeoffices keine neue Peripherie wie Bildschirme oder eine Tastatur kaufen.
Trotz allem war der Anfang und auch jetzt noch das tägliche Arbeiten an der Abteilung LET während dem Homeoffice sehr angenehm. Es gibt viele Personen in der Abteilung, welche sich enorm bemühen, die soziale Interaktion zwischen den Mitarbeiterinnen und Mitarbeiter hoch zu halten. Auf diese Art und Weise konnte ich mich auch schon mit Personen austauschen, mit welchen ich rein durch die Arbeit nicht viel bis gar kein Kontakt gehabt hätte.
New ideas for Innovedum, the ETH innovative teaching fund
With the Innovedum Fund, ETH has an extremely successful instrument for promoting innovative teaching, especially with regard to community building (cf. Reinhardt, Korner, Walter, 2019). Topics such as student engagement (Healey, Flint & Harrington, 2014) and Scholarship of Teaching and Learning (Martensson & Roxa, 2015) are increasingly being considered globally as an important part of educational development activities. With this in mind, the Innovedum application process became the focus of a rethink and revision in 2020. The application process was updated to a webform and new criteria were included in the application process. These were; inclusion of the student perspective, dissemination of Innovedum projects results and communication.
Inclusion of the student perspective in the project design and the planned project implementation
To encourage future applicants to take the student perspective into account, a new question was added to the application form. This is to meet the express wish of the Rector to further student perspectives and involvement when developing projects that innovate teaching and learning at ETH. Since the purpose of Innovedum is to have a positive effect on teaching and learning, it is important that the opportunity to include students in the application process is available:
Student Involvement: Describe whether and how students were involved in the preparation and review of this project application. How will students be involved in project implementation?
This question provides the project applicant with the freedom to decide if and how students can be involved in a possible project, while also pointing out easy steps how this could be done.
Dissemination of Innovedum Projects: Spreading good Teaching and Learning at ETH
Currently there is a public project database and various community events (Refresh Teaching; Learning and Teaching Fair) where Innovedum projects are made visible. To compliment this an explicit expectation to systematically reflect on the effectiveness of Innovedum projects is now also part of the application and reporting process. Applicants are now encouraged to consider the impact the project will have on teaching and learning and therefore develop a coherent evaluation strategy from the beginning.
Evaluation strategy: Describe the evaluation strategy you will use to check achievement of project goals and effects on teaching. What approaches will you use? Are you planning measures for identifying interim results? If so, how will these results flow back into the project?
For help with designing an evaluation strategy apropriate lecturers can always contact their LSPs or LET.
Project communication: Making project insights visible
Taking the findings made during the evaluation and sharing them with others will make it easier for new applicants to profit from the lessons others have learned and increase the quality of their own applications. Ultimately a clearer picture of how innovation in teaching in learning works at ETH will emerge and flow back in to educational development as a whole.
Project communication: How do you plan to publicise and document the progress of the project? What form will the final report for the Innovedum project database take? How will you disseminate project results?
There are a multitude of spaces both at ETH and beyond where results and experiences can be shared. At ETH the following spaces are available:
LET-Blog. The blog is a place where effective and innovative teaching is featured as well as general projects and activities relating to teaching and learning. www.blogs.ethz.ch/letblog
Refresh Teaching. A lunch-time seminar series where lecturers share and discuss their innovations in teaching. www.refreshteaching.ethz.ch
Innoview and Competence view are two different dynamic websites which respectively feature innovative teaching projects or projects where cross-disciplinary competencies are explicitly fostered.
Learning and Teaching Journal. The Journal publishes discussion as well as systematic reflections regarding discipline specific contributions.
Please contact LET (email@example.com) if you want to share your teaching project in one of these spaces. Any kind of projects are welcome, funded and non-funded.
Beyond ETH there are frequent conferences where teaching staff are welcome to present such projects. The Swiss Faculty Development Network hosts an annual conference of this nature and scholarship of teaching and learning (SoTL) conferences are also a great opportunity.
The benefits of classroom visits (and how to make them effective)
By Tommaso Magrini, PhD student, Departement of Materials
It requires effort, time and theoretical preparation to be truly able to deliver a good lecture, to properly plan a class, or to assess the performance of your students. As I am writing this document, I am still in this process of learning. Nevertheless, the more I am involved in teaching, the more I can experiment with new techniques or approaches, keeping the structures that worked, adjusting those that didn’t. One technique I plan to keep is classroom visits with peers.
Over the course of my doctoral studies I enrolled in the program “Learning to Teach”. One of the most interesting things I have learnt is the concept “learning by doing”. Not only is “learning by doing” more fun and engaging for the students, but it has also been proven to be the best ally for the lecturers to reach their learning objectives. Step by step, I introduced different activities in my classes, ranging from simple short discussions to more advanced posters and presentations. Those activities not only proved to be a good way to keep the students focused and active, but were also be extremely useful for us as teachers to assess if the concepts our class has been built on, are solid and stable in the students’ minds, or if they need repetition or consolidation. To understand whether the activities I have planned are meaningful and well aligned with the learning objectives, I have asked for peers to visit my classes, sit in and provide me with feedback.
Over the last semester I started implementing a new activity at the end of each lecture. Expecting my students to build on the concepts seen in my class and restructure them into a broader and more applied context, I proposed the following activity: divided in groups the students would need to come up with a shared idea, describe it schematically on a poster and then pitch it in front of the class. This would then foster a discussion between ‘critical friends’, that would openly challenge each group’s idea, with the goal of improving it and helping its realization. The classroom visit proved to be crucial in this phase.
As a lecturer, during such vivid and intense scientific discussions, I have to occupy several roles at the same time. Indeed, not only I have to moderate the discussion, but I also have to evaluate how deep and relevant the discussion is, while I assess whether the students have reached the milestones and the key learning objectives or not. For this reason, the presence of my colleague, that sits ‘outside the discussion’ and evaluates the classroom response to the activity is extremely important. If at the beginning he would observe only, at later stages we were also able to switch roles and evaluate the class activity in turns. Being able not only to take part in the discussion but also to observe it from outside and take notes gave me a more complete vision of the activity.
At the end of each class, I would always have a debriefing with my colleague. Its goal was to sum up the positive and negative aspects of the lecture in a constructive and unbiased way. These debriefings helped to correct the weak parts of the lecture and expand the positive ones. It was clear from the first time on, that the students were responding to the activity we planned with a positive attitude and with enthusiasm. Furthermore, through the classroom visits, we realized that we could more efficiently assess the classroom knowledge by using ‘exam-like questions’ during the discussion.
As a matter of fact, the ‘simple’ classroom visits have evolved, in our experience, into an open ideas exchange, built on honest and constructive feedback, that would help me improving my teaching style, the structure of my classes and the realization of more targeted and better structured learning activities.
Accessibility is an important topic for ETH, also in the institution’s teaching. ETH’s projectin this area comprises 14 sub-projects. In the first interview below, project manager Romila Storjohann describes the central points of the project with regard to teaching. The physical givens (rooms, access possibilities, etc.) are of course crucial to accessibility, but adapting electronic information is also important. LET has invested much time and energy in this topic, and has since January been working with a project coordinator for accessible teaching and teaching materials, Anton Bolfing. He describes his initial focus in the second interview below. LET is currently drafting various measures and identifying “quick wins”; for example, Mathjax formulas can be read aloud. LET organised an Accessible Teaching Day on 3 February 2020.
Accessibility is also an Innovedum focal point theme. Projects in this area are very welcome. Taking accessibility into account in teaching platforms and documents can provide improved semantics, better-structured, more flexibly displayed content and thus better usability in general. This benefits all users and also improves searchability.
What do you see as the long-term goals for accessibility in the area of teaching?
Seen in the long term, the goal is to offer barrier-free, inclusive teaching at ETH Zurich. This involves, on the one hand, the alteration of existing teaching materials and technologies, and on the other the readiness of teaching staff to make their teaching as accessible as possible so that all students may better take part.
What concrete links are there between ETH teaching and the three accessibility project categories “Construction”, “Technology” and “Organisation”?
The project “Barrier-free at ETH Zurich” comprises 14 sub-projects in total: nine under the category “Construction”, one under “Organisation” and four under “Technology”.
Sub-project 13, “Barrier-free teaching materials”, to commence in February 2021 at LET, is in the “Technology” category. Besides technical alterations (e.g. the subtitling of video recordings), non-technical measures will also be involved, such as as the drawing up of training concepts for teaching faculty.
Naturally measures in the “Construction” category will also contribute to barrier-free teaching by ensuring barrier-free access to seminar rooms, lecture halls and other ETH teaching locations.
Interview with Anton Bolfing, LET project coordinator for “Barrier-free teaching” since January 2021
My area of responsibility is the accessibility of electronic user interfaces. Here the main focus is on teaching and learning materials.
What do you see as the core themes for barrier-free teaching at ETH?
We envision an ETH where it goes without saying that accessibility is considered in all products and digital creations. Here information being digital is a prerequisite for its accessibility: print is not accessible!
What are your priorities?
In terms of existing accessibility issues, I see the most need for action in learning platforms and documents. Accessible teaching guarantees students access to all relevant information and systems irrespective of any physical, sensory or mental limitations. This involves not only the classical learning platforms, but also communication and information platforms and classical websites. Just as important are documents, including textbooks and lecture notes. Examinations must also become accessible.
It is clear to me that the road to our vision will be a long one. Therefore my first priorities are communication and the development of aids. All parties involved must be convinced of the need for accessible teaching.
Concretely, what will be altered on learning platforms and in documents?
Briefly summarised, there are two main points here:
Improved flexibility in the display of screen content (responsivity, various monitors, text enlargement, colour schemes etc.) and flexibility in the use of input devices (pointing devices such as mouse, joystick, eye-tracker etc. and serial input devices such as switches and keyboard)
Compatibility with assistive technologies (screenreader, language input)
How will the corona situation influence your efforts in 2021?
The corona situation is showing a broader public how important information and communication technologies are in the knowledge and service economy: think working from home. We are noticing how much we depend on these technologies. Many people with disabilities knew this even before the corona pandemic and will continue to know it afterwards. For them it is therefore even more important that these technologies are accessible.
H5P is a special toolset that enables teachers to enrich their Moodle courses. Teachers can firstly add interactive elements in Moodle content and secondly adjust the visual appearance of Moodle course pages. Both can enhance the learning process.
Research shows that digital learning is most effective when students interact with content, answer questions and most importantly, get immediate feedback. The critically important feedback loop that is naturally present in face-to-face learning is often missing in digital learning environments. Students want to find out immediately if their answer was correct or not. By providing performance feedback during digital learning, teachers can provide learners with a sense of the real-world consequences of decisions students make. Therefor the vast array of available interactive elements is especially important, since it offers options for almost every learning situation.
Effective digital learning should also provide learners with realistic practice opportunities; for example, simulations, scenario-based decision making, case-based evaluations, and authentic exercises.
Use H5P’s interactive elements to support reflection, application, rehearsal, elaboration, contextualisation, debate, evaluation, synthesisation, and so on. Focus on using H5P to add interaction and attractive graphic elements. H5P is not recommended for graded activities as tech savvy students can download and analyse the resulting XML file.
Example elements to increase interactivity
Select interactive elements, such as “Agamotto” which compares images as shown below.
“Find multiple hotspots” invites students to point out important aspects in images. Example: Find all the vegetables in this picture.
Or you can add a structural element to your course by adding the element “Accordion” which enables you to create collapsable paragraphs.
How to add H5P to your Moodle course
This must happen in two distinct steps. First you must create the content and save it. Secondly you embed the new content to your course.
To create the H5P content, access the “Content bank”. There is a shortcut in the navigation to the left of your screen.
Click on the “add” button and select the type of element you would like to create.
To help you decide which H5P element is best for your needs, we have created several exemplars to help you choose. You can view these exemplars in the “Building an effective Moodle course” in the section “Using special features“. (No enrollment key necessary).
In addition you can visit the H5P website to see more detailed examples, instructions and tutorials. Important note: There are more elements listed on the H5P website than are available on the ETH Moodle system.
After you have created H5P element, make sure you save it with a clear name so you can recognise it later.
Now you have two options for using the H5P element.
The first option is to add it as a separate activity. Simply add a new activity and select H5P. Then choose your pre-made element. This scenario make sense when you want to focus on the element as a stand alone activity and don’t want to embed it within additional text.
The second option is to add H5P as part of a text. Navigate to the exact spot where you would like to add the element (for example in a Moodle book or in a label on the course page) and begin editing.
In the editing toolbar, first expand the view of editing tools.
Then select H5P.
This will prompt you to “browse repositories”. The H5P content bank is shown as one of the repositories. All the elements you have already created are shown here. Select the one you need.
We recommend selecting the option “Create an alias/shortcut to the file”. This ensures that when you make a change to the original element in the content bank, it is automatically updated on your course page or wherever it has been embedded.
Preview how it looks by assuming the student role. (Switch roles by clicking on your profile picture.) We also recommend checking out it appears in the Moodle app. To make any changes to the element, you will need to go back to the content bank using your computer (not your mobile device), edit and save. It will update automatically if you have embedded it as an alias.
More information in the “Building an effective Moodle course” in the section “Using special features“. (No enrollment key necessary).
Among other things, ETH Zurich’s EduApp allows instructors to pose clicker questions during lectures. Instructors can interrupt lectures to ask questions from the students and get and give feedback on learning progress. Lecturers can also trigger phases of peer-instruction, where students discuss their initial answers to a question with one another and then reanswer the question – in effect, the students are teaching each other during those phases, thus “peer instruction”. By asking students to answer a question twice, lecturers gather data on student understanding. But how meaningful is this feedback data, in particular, when answering is voluntary and ungraded?
A group of mathematics instructors at ETH’s D-MATH worked with LET to analyze EduApp data using Item Response Theory (IRT), Classical Test Theory (CTT) and clustering methods. Over the course of the semester, 44 clicker problems were posed – 12 of them twice, as the instructor decided to insert a phase of peer-instruction. The following figure shows an example of the kind of problem being analyzed:
The problem shown was used in conjunction with peer-instruction; the gray bars indicate the initial student responses, the black bars those after the discussion. A simple, unsurprising observation is that after peer-instruction, more students arrived at the correct answer. What can we learn from these responses? CTT and IRT can provide psychometrics that help understand this instructional scenario.
When it comes to being “meaningful,” the “discrimination” parameter of a problem is of particular interest: how well does correctly or incorrectly answering a problem distinguish (“discriminate”) between students who have or have not understood the underlying concepts?
CTT simply uses the total score as a measure of “ability”, but also has a measure of discrimination (“biserial coefficient”). IRT estimates the probability of a student arriving at the correct answer for a particular problem (“item”) based on a hidden (“latent”) trait of the student called “ability” – typically, higher-ability students would have a higher chance of getting a problem correct. How exactly this probability increases depends on problem characteristics (“item parameters”).
In IRT, the ability-trait is determined in a multistep, multidimensional optimization process, where the difficulty and discrimination parameters of particular problems (“items”) feed back on how much correctly answering that problem says about the “ability” of the student; “high-ability” students are likely to get correct answers even on high-difficulty, high-discrimination problems.
The results of their study were extremely
encouraging: using both CTT and IRT, almost all 44 problems under investigation
exhibited strong positive discrimination in the initial vote. This means that
the better the student understood the underlying concepts, the much more likely
they were to give the right answers – and vice versa. A low
discrimination, on the other hand, means a problem provides less meaningful
feedback. For the handful of problems which had lower (yet still meaningful!)
discrimination, this could be explained by other problem characteristics, for
example, that at the time they were posed, they were still too hard or already
too easy – but even that feedback is meaningful to the instructor for future
The truly surprising result of the study was that in all cases of peer-instruction, the problem had even stronger discrimination afterwards! Yes, unsurprisingly more students answer correctly after discussion with their neighbors (the problem becomes “easier”), but: peer-instruction does not simply allow weaker students to enter the correct answer, it apparently helps them to perform at their true potential.
For the purposes of the study, the clicker data had to be exported manually, but the next version of EduApp, slated to be released in December 2020, will allow export of data for learning analytics purposes directly from the interface – the following figure shows a sneak preview of that new functionality.
The exported data format is compatible with
input for the statistics software R, and there are variety of guides
available for how to analyze this data (https://aapt.scitation.org/doi/abs/10.1119/1.5135788
(accessible through the ETH Library) provides a “quick-and-dirty” guide).
The full study, including results from
Classical Test Theory and clustering methods, as well an outlook for new EduApp-functionality
is available open-access in Issue 13 of e-learning and education (eleed)