Schlechte Unterrichtsbeurteilungen für innovativen Unterricht?
“Mit viel Engagement habe ich meine Lehrveranstaltung nach neuesten didaktischen Erkenntnissen umgestaltet. Sei es im Flipped Classroom oder mit erhöhtem Einsatz von Clicker-Fragen, die Studierenden waren während der Präsenz aktiv gefordert und haben auch gerne mitgemacht. Doch mit der Unterrichtsbeurteilung kam die grosse Enttäuschung. Die Studierenden bewerten mich und meinen Unterricht deutlich schlechter als vorher. Sie bevorzugen sogar Frontalunterricht, denn damit würden sie besser lernen. Habe ich etwas falsch gemacht? Soll ich wieder zurück zu meiner altbewährten Vorlesung?”
Was hier wie ein
Einzelfall klingt, ist doch sehr verbreitet. Zahlreiche Untersuchungen
dokumentieren, dass Unterrichtsformen mit Lerner zentrierten Methoden häufig zu
schlechten Evaluationsergebnissen führen (z.B. Seidel, 2013). In einer kürzlich publizierten Studie haben
Louis Deslauriers und Kollegen/innen der Harvard University genau diese
Problematik untersucht (Deslauriers,
2019). Sie gingen der Frage nach, ob es bei aktiv involvierten Studierenden
eine Diskrepanz zwischen dem tatsächlichen und dem empfundenen Lerngewinn gibt.
Falls Studierende den Eindruck haben, weniger als in einer Vorlesung gelernt zu
haben, dann führt dies zwangsläufig zu schlechteren Evaluationsergebnissen. Die
experimentell angelegte Untersuchung bestätigte die negative Korrelation
zwischen der Selbsteinschätzung und dem effektivem Lerngewinn. Folgende drei Gründe
sind dafür verantwortlich:
Interaktive Unterrichtsmethoden verlangen eine erhöhte kognitive Leistung, die von Studierenden dann nicht unbedingt mit dem Lernen in Verbindung gesetzt wird.
Insbesondere Studienanfänger/innen verfügen noch nicht über die Fähigkeit, ihr eigenes Wissen in einem neuen Fachgebiet korrekt einzuschätzen.
Die klare und sprachgewandte Präsentation beim Frontalunterricht verleitet Studierende dazu, ihr eigenes Verständnis in Vorlesungen deutlich zu überschätzen.
Aufgrund der Ergebnisse einer Nachuntersuchung schlagen Delauriers und Kollegen/innen einige recht simple Massnahmen vor, um dieses Missverhältnis zwischen gefühltem und tatsächlichem Lerngewinn zu unterbinden. Zentral dabei ist, die Befürchtungen und Ängste der Studierenden ernst zu nehmen und sie zu thematisieren. So kann eine kurze Darlegung der Lernvorteile von aktivem Unterricht (z.B. Freeman, 2014) während der ersten Unterrichtsstunde bereits erste Befürchtungen auffangen. Aber auch im weiteren Verlauf der Veranstaltung sollte immer wieder auf den erzielten Lernfortschritt hingewiesen werden. Damit erlangen die Studierende eine bessere Einschätzung ihres eigenen Lerngewinns. Hilfreich ist zudem, auf die Gefahr der Lernillusion bei eloquenter Redegewandtheit des Dozierenden in Vorlesungen hinzuweisen (z.B. Toftness, 2018).
Auch an der ETH konnten wir den Einfluss dieser Massnahmen bestätigen. In einer Studie am Departement Physik verglichen wir den Lerngewinn zwischen interaktivem Unterricht und Vorlesung. Bereits in der ersten Lerneinheit wiesen wir die interaktive Gruppe ausführlich auf die positiven Auswirkungen des interaktiven Unterrichts hin. Zusätzlich wurden Unsicherheiten bezüglich des eigenen Lerngewinns im Semester kontinuierlich thematisiert. Bei der Unterrichtsevaluation konnten wir daraufhin kein Missverhältnis zwischen effektiver und geschätzter Lernleistung feststellen. Studierende im interaktiven Unterrichtsformat erzielten einen höheren Lerngewinn und gaben signifikant bessere Werte bezüglich ihres eigenen Lernens an als jene in der parallel durchgeführten Vorlesung (Schiltz, 2018).
Tipp: Was hier jetzt speziell für interaktive Unterrichtsformen gilt, lässt sich sicher auch auf jeden anderen Wechsel der Lernform übertragen. Insbesondere wenn die neue Lernform noch nicht geläufig ist, sollte man die anfänglichen Bedenken der Studierenden ernst nehmen und ihnen klar vermitteln, welchen Nutzen sie vom Wechsel zu erwarten haben. Daneben ist es wichtig, Ergebnisse der Unterrichtsbeurteilung (ob gute oder schlechte) kritisch zu hinterfragen. Nicht immer ist der kausale Zusammenhang zwischen studentischer Zufriedenheit und tatsächlichem Lernerfolg gegeben (z.B. Carpenter, 2020).
Case study: Molegram Explorer – A mixed-reality framework for teaching drug design
As part of a series of case studies, staff at LET sat down to have a conversation with Prof. Gisbert Schneider and Dr. Jan Hiss from the Institute of Pharmaceutical Sciences in the Department of Chemistry and Applied Biosciences to discuss their mixed-reality project.
What is the project about?
The Molegram Explorer project provides a mixed-reality framework to facilitate and broaden students’ understanding of molecular structure. It is part of the Computer-Assisted Drug Design course run by Gisbert Schneider, professor in the field of the same name.
At the core of the project is the innovative hardware device “HoloLens” (comprising special glasses with 3D projection, motion sensor and environment scanning, produced by Microsoft). Users of the HoloLens see not only (real) furniture and people present in the room, but also a hologram. In our case, the virtual object is a protein that the students can explore, investigate and even walk through. They literally immerse themselves in the world of molecules.
This innovative concept presents a new way of perceiving molecular structure and facilitates new approaches to chemical structure analysis and design via human-machine interaction.
What motivated you to initiate the project?
This innovative project originated in answer to a call for projects on the use of the HoloLens in ETH teaching. Because we faced a teaching challenge where a 3D representation gave a very good use case, it was a perfect opportunity to apply for a pilot project using HoloLenses.
The idea underpinning our work is that this new technology can help our students to understand certain important principles of molecular structure which traditional teaching methods and media struggle to clarify. To identify or design a suitable drug molecule (the ligand) students must understand the protein’s surface, and particularly the cavities suitable for accommodating the ligand. The HoloLens device helps them visualise the regions of a protein that are accessible to the ligand.
How did you do it? Three groups collaborated on this project. First, Gisbert Schneider and Jan Hiss delivered the scientific content. Guided by specific learning objectives, ETH Zurich’s Educational Development and Technology (LET) unit helped organise and facilitate the development process and provided the required hardware. Finally, a specialised software company (afca) implemented the learning app software.
The HoloLenses are used in a two-week practical course in which students experience a condensed version of early-stage drug discovery. They learn how to computationally screen a catalogue of millions of molecules to identify those that might favourably interact with a particular protein. The students perform a computational analysis and select one or two molecules from the top-ranking candidates. Then they synthesise these compounds and test their activity in the laboratory.
An important basic aspect of protein-ligand interaction is the “solvent-accessible surface”. For beginners, this molecular representation often remains an abstract concept without suitable visualization. By using the HoloLens students can now create surface representations of a protein, interact with the holographic model, and simultaneously discuss it with peers and instructors.
Did you have the support you needed for the project? Is there any additional help you wish you had had?
We had excellent help from the company afca who designed a user-friendly app with an elegant interface. LET helped us with the legal aspects and provided the necessary contacts. The 12 ETH HoloLenses are stored at LET. Although we understand that HoloLenses are not easily available due to their comparably high acquisition cost, it would have been helpful to have faster and easier access to this hardware, especially when we needed to try out and check something quickly.
Please describe some of the key outcomes of the project.
The new tool proved to be a valuable addition to our course. It certainly does not replace traditional teaching and discussion, but it is an example of how technology can enhance the understanding of abstract scientific concepts which are otherwise hard to teach. Because students can virtually navigate the molecular hologram they gain a better understanding of the concept of protein structures and surfaces. In the learning sciences this effect is described by the principle of “embodied cognition”. We were also able to increase the attractiveness of our subject matter with this concrete visual experience. It was a kind of scientific marketing. We received several suggestions for additional projects in the context of other practical exercises. The positive feedback and the success of the pilot has driven us to expand the project with enhanced content and to reach out to other disciplines.
How did the project affect learners or the way in which you teach?
We observed that students became more curious, not only about the specific topic of the learning app but in general about many questions related to protein-structure-based drug design. Students certainly appreciate the new tool. The value of technology-enhanced learning apps for teaching of specific aspects in our field is obvious, and we intend to stay on this route.
What lessons learned would you share with your colleagues?
It is not always realistic or meaningful for scientists and teachers to address app programming and didactic concepts. Therefore, it is important to have experts from complementary fields working hand in hand. Experts on the subject matter can contribute the scientific content, and software developers can create user-friendly and visually appealing interfaces and functions. Learning professionals can then connect content with technological functions. They can also advise on how to transpose learning objectives into an appropriate and technology-enhanced learning process.
Overall, we encourage teachers to try out new methods in teaching, and there is much potential for combining proven learning approaches with new technology. In particular, teachers and students should not fear experiments that do not produce immediate success. “Productive failure” should be regarded as a natural part of the development process; it is a great way to learn.
What are your future plans for this work? How do you plan to sustain what you created through the project?
Based on the many positive outcomes, we plan to develop further apps. The ultimate goal is to adapt the work to a professional context by adding scientific content from our subject matter, together with advanced analysis tools. It would also make sense to develop HoloLens learning apps for selected (teaching) topics in medicine, chemistry and biology. HoloLenses are increasingly employed not only by the entertainment industry, but in business and education generally. We would welcome new, broader applications of this technology at ETH – but always with a critical double-check as to whether it actually provides added value for students compared to conventional teaching methods. In the case of Molegram Explorer, we are extremely satisfied with the learning success achieved.
Feedback from PhD students (Tutors)
What is your opinion about this course and the HoloLens process? How has it influenced your learning process?
Cyrill Brunner: Though it has been clear to me before since I have been doing research in that area before, the visualization of proteins by the HoloLens helped in getting a better feeling for 3D structure of a protein. My personal gain was clearly not so much as for the course participants, but that has nothing to do with the process itself, but the chosen protein (carbonic anhydrase II) which I’ve done research in before. I’m positive that application of the HoloLens process on a novel protein would have helped me clearly to get a better understanding of the 3D confirmation.
Dominique Bruns: The HoloLens is a useful next step in the visualisation of molecules. This hands-on experience allows the understanding of molecular properties, their definition and dependencies. In this regard, the application of the surface area visualisation and determination is an ideal showcase.
Do you think this approach could also be used in other courses?
Cyrill Brunner: Yes, indeed. The HoloLens should clearly be put into use in the lab course of medicinal chemistry as students there are already working with a 2D visualization program. A 3D full insight into what they have been studying beforehand would strengthen their understanding.
Dominique Bruns: I appreciate the 3D perspective and interactivity enabled by the HoloLens, a characteristic that might be especially useful for people that find it difficult to see in three-dimensions.
In my opinion, many further examples could be established and used for didactic purposes, e.g. chemical reaction mechanisms or biological folding events of proteins.
To find out more about how ETH teaching staff can start their own HoloLens project, visit theETH website.
Virtual field trips enhance Production Management classes
Using factory-visit apps and affordable cardboard viewers, Professor Torbjørn Netland and his team innovate how production management can be taught. By integrating Virtual Reality (VR) technology in the spring term course Global Operations Strategy at ETH Zurich, the teaching team provided students rare access to multiple factory sites and their inner workings. The students used VR apps in order to help them complete graded course assignments.
What triggered this experiment?
“Production and operations management is an applied field, but it is difficult to teach all the inner workings of a factory in a classroom. Because I cannot always bring students to the field, I wanted to bring the factory to the students. VR now offers unprecedented opportunities for doing so, and ABB had the app I needed. My impression is that students generally enjoyed this teaching innovation” – Prof. Dr. Netland
In optimal class conditions, students may be able to visit one factory in order to see how the concepts they are learning in class relate to the real world. However, field trips are resource intensive to organise and they have limitations. Not all students may be able to attend and they usually rely on their memory (or written notes) to recall relevant information later.
The use of the VR app enables students not only to revisit the factory as often as they would like, it also gives them access to spaces they may not have previously been able to see, such as a close-up view of a milling machine in action, or entry to a dust-free zone. Since the apps enable students to visit multiple company sites with a minuscule time and resource investment, the value of the virtual tours becomes very clear.
What exactly did they do?
With the help of two research associates, Oliver Flaeschner and Omid Maghazei, Prof. Netland compared the learning objectives of his course to the information already contained in the virtual tour apps produced by the ABB group. By adjusting their assignment questions for their teaching case, they created alignment between the content provided in the apps and the students’ assignment questions. With the help of the apps, their own smartphones and cardboard VR viewers, students navigated their way around five different factories in three countries, observed machinery and people in action and gathered information relevant to their assignments.
What were the results?
The teaching team conducted an evaluation survey, two focus groups and collected additional written feedback. The Faculty Development staff from the Educational Development and Technology unit (LET) provided support to ensure that student anonymity was preserved. The results showed that students initially found the opportunity to use VR fun and motivating. While there were some technical difficulties and temporary negative physical side-effects such as dizziness and headaches, in general students stated they enjoyed the opportunity to explore the virtual factory as often as they would like, and at their own pace and availability.
They also reported feeling a high degree of autonomy as they actively engaged with the information. The teaching team did not attempt to measure a learning gain this time, but are encouraged by this pilot project and are planning further development and evaluation of this teaching innovation. In the meantime, the results of the research have been written up and will be presented at the EurOMA Conference in Hungary in June this year.
Should readers wish to find out more, they are welcome to contact Prof. Dr. Netland directly for more information.