Category: Featuring teaching

Impartial group assessment. Using peer review and economic theory to grade groups fairly.

In a clear case of practicing what he preaches, Dr. Heinrich Nax has applied game theory to his teaching practice. After lecturing on game theory for several years, he realised that his methods for teaching, more specifically, for assessing did not follow the very theories he was espousing and so he set out to correct this incongruence.

In his course «Controversies in Game Theory» students work in groups and are assessed based on a group project. Social tensions can develop between individual and collective interests in group interactions. One such tension, free-riding, when one person rides the coat-tails of other hard-working group members is well known. There are however additional potential problems when assessing group work such as collusion on grades in cases of peer review. To eliminate these tensions, Dr. Nax decided to implement a mechanism from economic theory to his assessments.

What triggered this approach?

Previously Dr. Nax gave the same final mark to everybody in a particular group regardless of their individual efforts as these could not reliably be assessed. From a game theory perspective this constituted a big temptation for free-riding and Dr. Nax decided to devise something that would incentivize individual efforts but without giving up the benefits of group work altogether. 

What exactly did he do?

Influenced by the article Impartial division of a dollar by Clippel, G., Moulin, H., and Tideman, N. (2008), Dr. Nax and his colleague Sven Seuken implemented the article’s mechanism in a blockchain start-up company. The mechanism enables a group to split their financial earnings through peer review between the group members. Group members decide internally what a fair allocation of earnings should be. So he decided to try the mechanism in an educational setting where the “earnings” become the finite amount of points the group works towards, the total of which is determined by the grade he allocates to the group’s total project.

The key idea of the mechanism is that individual group members don’t evaluate their own performance and therefore don’t decide how many points they themselves have “earned”. Instead they allocate relative contributions to the other group members. So in a group of three, if student A thinks group member B did twice as much work as fellow group member C, then B should receive twice as many points as C. Using a specific formula (described in the paper) all three group members reports are then aggregated anonymously to make sure the resulting grades cannot be manipulated. In other words, student A only receives the (aggregated) amount of points, that their colleagues think student A deserves.

Courtesy of Spliddit

What were the results (for student learning)?

Not only was Dr. Nax convinced that the quality of the group projects improved, but the students were happier as well. They believed that the marking was much more fair. It is unclear if this grading method decreased free-riding, however students felt that freeriders did receive lower grades, thus increasing student satisfaction in comparison to grading methods where all members of the group receive the same grade, regardless of effort or contribution.

To see this mechanism in action, visit the Spliddit website which features a demonstration tool. Those interested in learning more should read the original paper or this second (less math-based) follow-up paper or contact Dr. Nax for further information. Dr. Nax is working on a tool to make his grading plan available to other professors.

 

Read more

Posted on ,

Case study: Molegram Explorer – A mixed-reality framework for teaching drug design

As part of a series of case studies, staff at LET sat down to have a conversation with Prof. Gisbert Schneider and Dr. Jan Hiss from the Institute of Pharmaceutical Sciences in the Department of Chemistry and Applied Biosciences to discuss their mixed-reality project.

 

What is the project about?

The Molegram Explorer project provides a mixed-reality framework to facilitate and broaden students’ understanding of molecular structure. It is part of the Computer-Assisted Drug Design course run by Gisbert Schneider, professor in the field of the same name.

At the core of the project is the innovative hardware device “HoloLens” (comprising special glasses with 3D projection, motion sensor and environment scanning, produced by Microsoft). Users of the HoloLens see not only (real) furniture and people present in the room, but also a hologram. In our case, the virtual object is a protein that the students can explore, investigate and even walk through. They literally immerse themselves in the world of molecules.

This innovative concept presents a new way of perceiving molecular structure and facilitates new approaches to chemical structure analysis and design via human-machine interaction.

 

What motivated you to initiate the project?

This innovative project originated in answer to a call for projects on the use of the HoloLens in ETH teaching. Because we faced a teaching challenge where a 3D representation gave a very good use case, it was a perfect opportunity to apply for a pilot project using HoloLenses.

The idea underpinning our work is that this new technology can help our students to understand certain important principles of molecular structure which traditional teaching methods and media struggle to clarify. To identify or design a suitable drug molecule (the ligand) students must understand the protein’s surface, and particularly the cavities suitable for accommodating the ligand. The HoloLens device helps them visualise the regions of a protein that are accessible to the ligand.

 

How did you do it?
Three groups collaborated on this project. First, Gisbert Schneider and Jan Hiss delivered the scientific content. Guided by specific learning objectives, ETH Zurich’s Educational Development and Technology (LET) unit helped organise and facilitate the development process and provided the required hardware. Finally, a specialised software company (afca) implemented the learning app software.

The HoloLenses are used in a two-week practical course in which students experience a condensed version of early-stage drug discovery. They learn how to computationally screen a catalogue of millions of molecules to identify those that might favourably interact with a particular protein. The students perform a computational analysis and select one or two molecules from the top-ranking candidates. Then they synthesise these compounds and test their activity in the laboratory.

An important basic aspect of protein-ligand interaction is the “solvent-accessible surface”. For beginners, this molecular representation often remains an abstract concept without suitable visualization. By using the HoloLens students can now create surface representations of a protein, interact with the holographic model, and simultaneously discuss it with peers and instructors.

 

Did you have the support you needed for the project? Is there any additional help you wish you had had?

We had excellent help from the company afca who designed a user-friendly app with an elegant interface. LET helped us with the legal aspects and provided the necessary contacts. The 12 ETH HoloLenses are stored at LET. Although we understand that HoloLenses are not easily available due to their comparably high acquisition cost, it would have been helpful to have faster and easier access to this hardware, especially when we needed to try out and check something quickly.

 

Please describe some of the key outcomes of the project.

The new tool proved to be a valuable addition to our course. It certainly does not replace traditional teaching and discussion, but it is an example of how technology can enhance the understanding of abstract scientific concepts which are otherwise hard to teach. Because students can virtually navigate the molecular hologram they gain a better understanding of the concept of protein structures and surfaces. In the learning sciences this effect is described by the principle of “embodied cognition”. We were also able to increase the attractiveness of our subject matter with this concrete visual experience. It was a kind of scientific marketing. We received several suggestions for additional projects in the context of other practical exercises. The positive feedback and the success of the pilot has driven us to expand the project with enhanced content and to reach out to other disciplines.

 

How did the project affect learners or the way in which you teach?

We observed that students became more curious, not only about the specific topic of the learning app but in general about many questions related to protein-structure-based drug design. Students certainly appreciate the new tool. The value of technology-enhanced learning apps for teaching of specific aspects in our field is obvious, and we intend to stay on this route.

 

What lessons learned would you share with your colleagues?

It is not always realistic or meaningful for scientists and teachers to address app programming and didactic concepts. Therefore, it is important to have experts from complementary fields working hand in hand. Experts on the subject matter can contribute the scientific content, and software developers can create user-friendly and visually appealing interfaces and functions. Learning professionals can then connect content with technological functions. They can also advise on how to transpose learning objectives into an appropriate and technology-enhanced learning process.

Overall, we encourage teachers to try out new methods in teaching, and there is much potential for combining proven learning approaches with new technology. In particular, teachers and students should not fear experiments that do not produce immediate success. “Productive failure” should be regarded as a natural part of the development process; it is a great way to learn.

 

What are your future plans for this work? How do you plan to sustain what you created through the project?

Based on the many positive outcomes, we plan to develop further apps. The ultimate goal is to adapt the work to a professional context by adding scientific content from our subject matter, together with advanced analysis tools. It would also make sense to develop HoloLens learning apps for selected (teaching) topics in medicine, chemistry and biology. HoloLenses are increasingly employed not only by the entertainment industry, but in business and education generally. We would welcome new, broader applications of this technology at ETH – but always with a critical double-check as to whether it actually provides added value for students compared to conventional teaching methods. In the case of Molegram Explorer, we are extremely satisfied with the learning success achieved.

 

Feedback from PhD students (Tutors)

What is your opinion about this course and the HoloLens process? How has it influenced your learning process?

Cyrill Brunner: Though it has been clear to me before since I have been doing research in that area before, the visualization of proteins by the HoloLens helped in getting a better feeling for 3D structure of a protein. My personal gain was clearly not so much as for the course participants, but that has nothing to do with the process itself, but the chosen protein (carbonic anhydrase II) which I’ve done research in before. I’m positive that application of the HoloLens process on a novel protein would have helped me clearly to get a better understanding of the 3D confirmation.

Dominique Bruns: The HoloLens is a useful next step in the visualisation of molecules. This hands-on experience allows the understanding of molecular properties, their definition and dependencies. In this regard, the application of the surface area visualisation and determination is an ideal showcase.

 

Do you think this approach could also be used in other courses?

Cyrill Brunner: Yes, indeed. The HoloLens should clearly be put into use in the lab course of medicinal chemistry as students there are already working with a 2D visualization program. A 3D full insight into what they have been studying beforehand would strengthen their understanding.

Dominique Bruns:  I appreciate the 3D perspective and interactivity enabled by the HoloLens, a characteristic that might be especially useful for people that find it difficult to see in three-dimensions. 

In my opinion, many further examples could be established and used for didactic purposes, e.g. chemical reaction mechanisms or biological folding events of proteins.

 

To find out more about how ETH teaching staff can start their own HoloLens project, visit the ETH website.

Read more

Posted on ,

Neue Funktionen für die EduApp

Die EduApp ist eine der wichtigsten Lehrapplikationen der ETH. Ziel der EduApp ist es einerseits, die Interaktion zwischen Studierenden und Dozierenden im Hörsaal zu verbessern. Anderseits möchte diese Lehre-App Studierenden der ETH Zürich einen Mehrwert im Studienalltag bieten.

Im letzten Frühlingssemester haben 100 Dozierende Clickerfragen in ihrem Unterricht eingesetzt und damit 8’694 Studierende erreicht. Auch aus Sicht der Dozenten ist die EduApp eine wertvolle Ergänzung.

Dr. Ghislain Fourny (D-INFK): «Ich benutze seit 2016 die EduApp in allen meinen Vorlesungen und bin davon sehr begeistert. Es ermöglicht eine reiche Interaktion mit den Studierenden und gibt mir ein konstantes Feedback»

Prof. Dr. Christoph Heinrich (D-ERDW): «Ich habe im HS2017 zum ersten Mal regelmässig Clicker-Fragen in meiner grossen Geologievorlesung für die Erstsemestrigen am D-BAUG eingesetzt. Es war ein grosser Erfolg, nicht zuletzt wegen der Auflockerung, und ich bekam spontan viele positive Feedbacks».

Dr. Markus Kalisch (D-MATH): «Mit der EduApp bekomme ich sofortiges Feedback von den Studenten, selbst wenn die Vorlesung mehrere hundert Teilnehmer hat».

Dr. Meike Akveld (D-MATH): «Die EduApp gibt mir direktes Feedback darüber, ob verstanden wurde, was ich unterrichtet habe. Ich bitte immer einen der Studierenden die richtige Antwort zu erklären, was oft hilfreich ist. Ausserdem ist es für sie eine angenehme Abwechslung».

Neue Funktionen für den Clicker

Pünktlich auf das aktuelle Semester wurden in der der EduApp neue Funktionen im Bereich Clicker hinzugefügt. Mit der Funktion «Clicker» können Dozierende via EduApp Fragen stellen, die meist sofort im Unterricht beantwortet werden.

1. Zwischenresultate: Neu können Dozierende die Abstimmung der Clickerfragen in zwei Runden machen und die Zwischenresultate anzeigen.

2. Erweiterter LaTeX-Editor: Der Funktionsumfang des LaTeX-Editors zur Anzeige von mathematischen Formeln in Clickerfragen wurde erweitert. Nicht nur können Dozierende jetzt Formeln und Gleichungen im Text eingebunden darstellen, es gibt auch mehr Textformatierungsmöglichkeiten.

3. Flashcards: Studierende können neu mit der Funktion «Flashcards» bestehende Clickerfragen durcharbeiten (z.B. zur Prüfungsvorbereitung). Die neue EduApp-Funktion “Flashcards” wurde durch den «the Rectors Impulse Fund» ermöglicht.

Mehr zu den neuen Funktionen finden Sie auf der EduApp Service-Seite und in der aktualisierte EduApp-Anleitung.

Read more

Posted on ,

Flipped learning helps evaluation of health delivery systems

When it comes to medical device and health services development, it is essential to conduct a rigorous analysis and propose fitting solutions. These should cover the specific situations that impact the user and the country’s system within which they are delivered. This not only requires knowledge in medical device evaluation and regulation, but also “soft” skills such as empathy. In order to maximise opportunities for his students to develop these competences, Prof. Dr. Walter Karlen flipped his class so that his students could practice the skills he intends for them to learn. The course in question is called Appropriate Health System Design, an elective course open to all master students of ETH.

What triggered this approach?

Comprehensive analysis and human centred design are very important competences in Dr. Karlen’s field. Encouraged and informed by what he learned in the didactic course Teaching at ETH: Committed and skilled, he adjusted his course design to become almost entirely flipped.

What exactly did he do?

Students work in groups to work their way through the challenges he sets for them every week. Readings assigned as homework prepare students to solve problems together during class time, while he is there to advise and support them. Using the principles learned during the didactic course, he ensures that he consolidates the results of their group activities and are shared in an online book so all students can benefit. This also means he can provide feedback on their progress towards the learning objectives.

Students attempt to improve existing devices (such as a standard asthma inhaler or an x-ray machine) or systems within which these devices are delivered. They do this according to the unique needs of fictional personas they collectively developed in the first weeks of the course. Then, they develop prototypes and evaluate them for appropriateness. They present their results in a final graded poster presentation to their peers and a panel of experts.

What were the results?

The response of the students has been very positive. They complimented the interactive course design and rated the course highly in the most recent evaluation. “I believe the students are inspired more. They will certainly remember this course next time they see a medical device. Even though adjusting the course plan has taken time, the effort was well worth it.”

 

Read more about Flipped Classroom

Read more

Posted on ,

Virtual field trips enhance Production Management classes

Using factory-visit apps and affordable cardboard viewers, Professor Torbjørn Netland and his team innovate how production management can be taught.  By integrating Virtual Reality (VR) technology in the spring term course Global Operations Strategy at ETH Zurich, the teaching team provided students rare access to multiple factory sites and their inner workings. The students used VR apps in order to help them complete graded course assignments.

What triggered this experiment?


A student using the cardboard viewer

“Production and operations management is an applied field, but it is difficult to teach all the inner workings of a factory in a classroom. Because I cannot always bring students to the field, I wanted to bring the factory to the students. VR now offers unprecedented opportunities for doing so, and ABB had the app I needed. My impression is that students generally enjoyed this teaching innovation” –  Prof. Dr. Netland

In optimal class conditions, students may be able to visit one factory in order to see how the concepts they are learning in class relate to the real world. However, field trips are resource intensive to organise and they have limitations. Not all students may be able to attend and they usually rely on their memory (or written notes) to recall relevant information later.

The use of the VR app enables students not only to revisit the factory as often as they would like, it also gives them access to spaces they may not have previously been able to see, such as a close-up view of a milling machine in action, or entry to a dust-free zone. Since the apps enable students to visit multiple company sites with a minuscule time and resource investment, the value of the virtual tours becomes very clear.

What exactly did they do?

With the help of two research associates, Oliver Flaeschner and Omid Maghazei, Prof. Netland compared the learning objectives of his course to the information already contained in the virtual tour apps produced by the ABB group. By adjusting their assignment questions for their teaching case, they created alignment between the content provided in the apps and the students’ assignment questions. With the help of the apps, their own smartphones and cardboard VR viewers, students navigated their way around five different factories in three countries, observed machinery and people in action and gathered information relevant to their assignments.

What were the results?

The teaching team conducted an evaluation survey, two focus groups and collected additional written feedback. The Faculty Development staff from the Educational Development and Technology unit (LET) provided support to ensure that student anonymity was preserved. The results showed that students initially found the opportunity to use VR fun and motivating. While there were some technical difficulties and temporary negative physical side-effects such as dizziness and headaches, in general students stated they enjoyed the opportunity to explore the virtual factory as often as they would like, and at their own pace and availability.

They also reported feeling a high degree of autonomy as they actively engaged with the information. The teaching team did not attempt to measure a learning gain this time, but are encouraged by this pilot project and are planning further development and evaluation of this teaching innovation.  In the meantime, the results of the research have been written up and will be presented at the EurOMA Conference in Hungary in June this year.

 

Should readers wish to find out more, they are welcome to contact Prof. Dr. Netland directly for more information.

Oliver Flaeschner, Torbjorn Netland and Omid Maghazei

Read more

Posted on ,