In a project-based course, students learned to apply materials knowledge and skills to the construction of a boat that could navigate an unknown terrain using artificial intelligence. We talked with the two lecturers Rafael Libanori and Henning Galinski, and the department’s Educational Developer Lorenzo De Pietro to find out more about this innovative course in the Department of Material Sciences.
Lukas Wooley and Sebastian Gassenmeier get their boat ready.
What triggered this experiment?
Originally, we were inspired by AP50, a project and team-based introductory physics course taught at Harvard. We wanted to do more with problem-based learning at ETH Zurich and achieve a different kind of learning environment. Students tend to expect that lectures just “give the knowledge”, but there is so much more to teaching. We realised it’s important to teach students how to learn more efficiently and take more responsibility for their own learning. In this course, we give students scientific questions to answer themselves. We wanted them to start taking risks and to have the freedom to fail, which is what science is all about. It’s not just theoretical input. Interpersonal and technical skills are just as important as academic skills.
What exactly did you do? We applied for Innovedum funding and when we were successful recipients we created a course that gives the students a project that has a connection to material sciences, as well as other areas such as controlling and artificial intelligence. We receive support from Antonio Loquercio in the controlling and computer vision part. He is currently a Post-Doc at the University of Berkley, California. Without him, it would have been very difficult to achieve the computational goals of the project. Students attended 4 weeks of theoretical classes and then started working in teams. The goal is to construct a model boat which can intelligently navigate a course using the Materials Design Lab at D-MATL. We also employed PhD students as coaches to support the students.
What were the results? We had 16 students who completed the course in the spring of 2021. The challenges were big and so we were thrilled by the final outcomes. The students took it seriously and at the end of the course there were four final boats. The students displayed great creativity, such as building small experimental set-ups along the way. They were able to solve problems on their own, in groups and learn from each other.
What is the student perspective? Students were frustrated initially because we took a passive approach to communicating knowledge, but they saw the benefit of this approach at the end. We believe that learning should strain their abilities and that it is iterative. But it is wonderful that it ended on a celebratory note with the functional boats that successfully navigated the terrain.
What lessons did you learn? We realised that in the future we need to spend more time explaining our approach to teaching and clarifying expectations right from the start. We also plan to pay close attention to the gender-balance among our students as we want to maintain a good mix as the course grows.
What are your plans for the future regarding this project? Due to the current curriculum revision projects in our department, there will likely be an increase in hands-on courses like this one. So, this course represents a new way of teaching, like a prototype for the new curriculum. The results will be looked at closely and are quite important for future decision-making in the department. Teaching this way is also a development opportunity for the lecturers.
What first steps do you advise for others who are interested in doing the same? We think it is important that teaching is viewed as a design science, in other words that it benefits from careful planning and time. We recommend visiting other courses that already use this kind of approach and speaking with the course leaders to gain inspiration and practical ideas for implementing project and problem-based learning in your own course. We would be happy to share our experiences with others.
New ideas for Innovedum, the ETH innovative teaching fund
With the Innovedum Fund, ETH has an extremely successful instrument for promoting innovative teaching, especially with regard to community building (cf. Reinhardt, Korner, Walter, 2019). Topics such as student engagement (Healey, Flint & Harrington, 2014) and Scholarship of Teaching and Learning (Martensson & Roxa, 2015) are increasingly being considered globally as an important part of educational development activities. With this in mind, the Innovedum application process became the focus of a rethink and revision in 2020. The application process was updated to a webform and new criteria were included in the application process. These were; inclusion of the student perspective, dissemination of Innovedum projects results and communication.
Inclusion of the student perspective in the project design and the planned project implementation
To encourage future applicants to take the student perspective into account, a new question was added to the application form. This is to meet the express wish of the Rector to further student perspectives and involvement when developing projects that innovate teaching and learning at ETH. Since the purpose of Innovedum is to have a positive effect on teaching and learning, it is important that the opportunity to include students in the application process is available:
Student Involvement: Describe whether and how students were involved in the preparation and review of this project application. How will students be involved in project implementation?
This question provides the project applicant with the freedom to decide if and how students can be involved in a possible project, while also pointing out easy steps how this could be done.
Dissemination of Innovedum Projects: Spreading good Teaching and Learning at ETH
Currently there is a public project database and various community events (Refresh Teaching; Learning and Teaching Fair) where Innovedum projects are made visible. To compliment this an explicit expectation to systematically reflect on the effectiveness of Innovedum projects is now also part of the application and reporting process. Applicants are now encouraged to consider the impact the project will have on teaching and learning and therefore develop a coherent evaluation strategy from the beginning.
Evaluation strategy: Describe the evaluation strategy you will use to check achievement of project goals and effects on teaching. What approaches will you use? Are you planning measures for identifying interim results? If so, how will these results flow back into the project?
For help with designing an evaluation strategy apropriate lecturers can always contact their LSPs or LET.
Project communication: Making project insights visible
Taking the findings made during the evaluation and sharing them with others will make it easier for new applicants to profit from the lessons others have learned and increase the quality of their own applications. Ultimately a clearer picture of how innovation in teaching in learning works at ETH will emerge and flow back in to educational development as a whole.
Project communication: How do you plan to publicise and document the progress of the project? What form will the final report for the Innovedum project database take? How will you disseminate project results?
There are a multitude of spaces both at ETH and beyond where results and experiences can be shared. At ETH the following spaces are available:
LET-Blog. The blog is a place where effective and innovative teaching is featured as well as general projects and activities relating to teaching and learning. www.blogs.ethz.ch/letblog
Refresh Teaching. A lunch-time seminar series where lecturers share and discuss their innovations in teaching. www.refreshteaching.ethz.ch
Innoview and Competence view are two different dynamic websites which respectively feature innovative teaching projects or projects where cross-disciplinary competencies are explicitly fostered.
Learning and Teaching Journal. The Journal publishes discussion as well as systematic reflections regarding discipline specific contributions.
Please contact LET (beratung@let.ethz.ch) if you want to share your teaching project in one of these spaces. Any kind of projects are welcome, funded and non-funded.
Beyond ETH there are frequent conferences where teaching staff are welcome to present such projects. The Swiss Faculty Development Network hosts an annual conference of this nature and scholarship of teaching and learning (SoTL) conferences are also a great opportunity.
Engaging students through technology enhanced Feedback
Teachers’ written commentary on student assignments is a fundamental
element of instruction in almost any discipline. However,
it is unclear what impact the feedback has on students. Consequently, teachers
face fundamental questions for which no ready answers are available: Which
components of commentary are most helpful, and how are they most effectively
delivered? How can students’ uptake of commentary be optimised, and how can
teachers be most efficient when providing commentary?
Giving and receiving feedback with Edword
Edword is an assignment feedback tool that provides answers to these
questions; it quantifiably improves the quality and efficiency of commentary
and its uptake and measures
previously unobserved aspects of learning. Edword is an online
platform to which students can upload written assignments of any kind set by
teachers.
Teachers can then add commentary to the text. These comments can be written individually as done in many traditional teaching settings, but Edword also enables the rapid application of prewritten comments from comment sets. These comment sets can address any aspects of written work in any discipline. They can be prepared by teachers working individually or shared between colleagues in teams.
Because they can be adjusted and augmented, a comment set can evolve as individual comments are added and improved. The quality and level of detail that can be delivered within the time available for commenting on assignments is thus substantially increased.
When the student opens the commentary, the most important comments,
selected by the teacher, are presented first and repeated comments bundled so
that the student sees every instance of the same comment in the assignment.
This allows the teacher to optimise individual students’ uptake of their
commentary. Edword enables further optimisation by measuring two aspects of
students’ engagement with the comments: the time the student spends with each
individual comment is automatically recorded, and the student gives one of
three responses—helpful, neutral, or unhelpful—to each comment. These data points
are automatically collated by comment and assignment to provide a fine-grained
evidence base for further adaptation of comment sets and commenting practice to
the specific requirements of programs and disciplines.
Successful pilot project
Edword’s suitability for use with UZH and ETH students was tested in a pilot project between February and May 2020. Writing instructors from the English unit of the Language Center of UZH and ETH Zurich (LC) attended a LET Refresh Teaching event at ETH on 4 September 2019 where selected EdTech startups from the Kickstart Accelerator program presented their tools; here, they were introduced to the Edword online writing assessment tool. Seeing its potential, four writing instructors collaborated with LET and ran a pilot project to test Edword in their courses comprising 167 students in all. The instructors created and shared comment sets containing a total of over 350 specialized comments. The participating students were surveyed online about their experience with Edword at the end of their courses (response rate 32%). Some 87% said they preferred commentary via Edword over traditionally delivered comments.
Potential for broader application at ETH
The feedback processes in Edword can be used to provide highly nuanced and sophisticated commentary for any kind of written assignment, and comment sets can be adapted to the demands of any discipline. The comment sets can be written centrally or developed collaboratively or individually, and the uptake of commentary is monitored in detail. Further test groups can demonstrate the range of academic contexts in which Edword is applicable. Please contact Melanie Walter if you are interested in trying out Edword in your ETH course.
Pilot am D-INFK: Blended Learning Kurs “Didactic Basics for Student Teaching Assistants”
Als wichtiger Bestandteil von innovativer Lehre an der ETH Zürich, haben sich Online-Lernangebote in den letzten Jahren stetig weiterentwickelt. Vor allem der Ansatz des Blended Learning sowie das Konzept des Flipped Classroom, mit der Verknüpfung von Online-Selbstlernphasen und kooperativem Präsenzlernen gewinnen immer mehr an Bedeutung.
Die didaktische Ausbildung für studentische Lehrassistierende an der ETH Zürich nimmt diesen Trend auf und setzt ihn zielgruppenorientiert um: Daher befindet sich das Kursangebot derzeit im Wandel – aus einer klassischen Präsenzveranstaltung soll zukünftig ein Blended Learning Kursangebot für alle Hilfsassistierenden mit Lehraufgaben entstehen.
Zur didaktischen Ausbildung von Hilfsassistierenden bietet das LET seit vielen Jahren entsprechende Kurse an, bei denen das Lernen zu ca. 80% in Präsenz und ca. 20% des Lernens online stattfindet. Da die didaktischen Kurse für Lehrassistierenden immer besser besucht werden und die Skalierung der Präsenzkurse eine natürliche Grenze hat, wurde vor einiger Zeit mit der Konzeptplanung eines Blended Formats begonnen. Nach eigenen Videoproduktionen und mehrmaligen Feedbackschleifen mit ehemaligen Lehrassistierenden konnte ein erster Pilot im Sommer 2019 fertiggestellt und im HS 2019 für die Hilfsassistierenden am D-INFK angeboten werden. Für die erste Durchführung entschieden wir uns, Präsenzkurs und Onlinekurs parallel anzubieten und die Studierenden selbst aussuchen zu lassen, welches Format sie bevorzugen: Für den neuen Blended Learning Kurs am D-INFK registrierten sich 88 Teilnehmende, während am “klassischen” Präsenzkurs lediglich 16 Studierende teilnahmen. Bereits an dieser Verteilung beim Pilotkurs wird deutlich, dass die Lehrassistierenden ein flexibles Format bevorzugen, wie wir dies mit dem neuen Blended Learning Kurs “Didactic Basics for Student Teaching Assistants” anbieten. Bei dem neuen Kurskonzept ist das Verhältnis zwischen Online- und Präsenzlernen genau umgekehrt zum bisherigen Konzept: ca. 80% des Lernens findet nun online anhand von Lernvideos und anderen Online-Lernmaterialien statt, während der Transfer des Gelernten in einem Präsenztermin umgesetzt wird und weiterhin Peer Hospitationen (gegenseitige Unterrichtsbesuche) stattfinden, die zusammen ca. 20% des Gesamtaufwands für die Teilnehmenden ausmachen.
Die Online-Lernumgebung in Moodle stand den angemeldeten Teilnehmenden zum Beginn des Herbstsemester 2019 zur Verfügung und begann mit einer dreiwöchigen Startphase, um den unterschiedlichen Anstellungszeitpunkten von Hilfsassistierenden gerecht werden zu können. Nach dieser Ankunftsphase wurden dann im wöchentlichen Rhythmus die vier Kapitel des Online-Kurses veröffentlicht.
Die vier Kapitel des Onlinekurses beinhalten die folgenden didaktischen Themengebiete:
Getting started with your course planning
Presentation and communication in class
Activating students in class
Feedback & Assessment
Im ersten Teil des Kurses reflektierten die Teilnehmenden ihre Rolle als Hilfsassistierende und erstellten einen Plan für eine Übungsstunde mit den dazugehörenden Lernzielen. Die Basis für die Erreichung der jeweilige Kompetenzstufe in den Kapiteln wurde durch vom LET produzierte Videos und mit den Videos verknüpften Aufgabenstellungen geschaffen. Mit Ausnahme vom dritten Kapitel enthält jedes Kapitel 1-3 Videos, welche wichtige Inputs zu den jeweiligen Hauptthemen geben. Am Ende von jedem Kapitel gab es zudem eine Reflexionsaufgabe, die eine individuelle Rückschau der Hilfsassistierenden auf Ihre wichtigsten Erkenntnisse ermöglicht und mit der eine Konsolidierung des Wissens erreicht werden soll.
Nach Abschluss der vierwöchigen Onlinephase fanden mehrere Transferworkshops in Form von Mittagsveranstaltungen in Kleingruppen mit jeweils 20-23 Personen statt. An diesen Präsenzterminen hatten die Hilfsassistierenden die Möglichkeit, die Lerninhalte aus der Onlinephase zu vertiefen und mit Peers sowie den Kursleitenden zu diskutieren. Zudem wurden am Präsenztermin Tandems für die Peer-Hospitation gebildet, deren Ergebnisse wiederum online präsentiert und diskutiert werden.
Bei den Transferworkshops wurden die Lerninhalte aus dem Onlinekurs mit eigenen Erfahrungen in der Lehre verglichen und gegenseitiges Feedback zu individuellen Lösungsansätzen der Aufgabenstellungen, wie z.B. dem Lesson Plan für eine Übungsstunde gegeben. Zum Abschluss des Onlinekurses wurde eine finale Reflexionsaufgabe gestellt, um den Hilfsassistierenden die Möglichkeit zu geben, ihre neue Sicht auf das Lehren und Lernen in Übungen in eigene Worte zu fassen, diese mit der vorhergehenden Sichtweise zu vergleichen und den Kurs in seiner Gesamtheit nochmals Revue passieren zu lassen.
Insgesamt beinhaltet der Onlinekurs zehn schriftliche Leistungsnachweise, welche von den Teilnehmenden arbeitet und eingereicht werden müssen. Zusätzlich gehören die beiden Onlinebefragungen am Anfang und am Ende der Kurses, die Teilnahme am Transferevent und die Peer-Hospitation ebenfalls zu den Pflichtaufgaben des Kurses. Die Deadline für die Abgabe aller Leistungsnachweise steht noch bevor. Aus diesem Grund ist noch keine Aussage darüber möglich, über wie viele Lehrassistierenden den Onlinekurs erfolgreich bestanden haben.
Die grosse Herausforderung bei der Erstellung eines Onlinekurses ist die Vorausplanung. Der gesamte Kurs sowie sämtliche Materialien und Abläufe müssen vor Kursbeginn vollständig feststehen. Aufgaben und Leistungsnachweise sollten von Anfang an klar definiert sein, damit die Teilnehmenden den Arbeitsaufwand einschätzen können. Eine weitere wichtige Überlegung ist die der Kommunikation zwischen Kursleitenden und Teilnehmenden: Wie soll die Kommunikation erfolgen? Innerhalb von welchem Zeitraum können die Teilnehmenden mit einer Rückmeldung rechnen? Wie bewerkstelligen wir die Rückmeldungen bei einer Vielzahl an Teilnehmenden? Klar festgesetzte Regelungen sind notwendig, um den reibungslosen Ablauf des Blended Learning Kurses zu gewährleisten. Zusammengefasst muss eine klare Struktur, vordefinierte Prozesse und eine sorgfältige Vorausplanung gegeben sein, um einen Onlinekurs erfolgreich durchführen zu können.
Anhand des positiven Feedbacks der Teilnehmenden sehen wir in Blended Learning Kursen eine neue Perspektive für didaktische Kursangebote an der ETH Zürich. Zudem haben wir festgestellt, dass der Onlinekurs hinsichtlich Ablauf und Struktur noch weiterentwickelt und ausgebaut werden muss. Beispielsweise haben sich die teilnehmenden Studierenden gewünscht, dass gewisse Inhalte des Kurses bereits zu einem früheren Zeitpunkt freigeschaltet werden sollen, damit die Teilnehmenden diese bereits für ihre erste Übungsstunde als Lehrassistierenden einsetzen können. Die stärkere Orientierung an der Lehrpraxis der Teilnehmenden steht daher ganz oben auf unserer Agenda für die Weiterentwicklung des hybriden (Verknüpfung von Online- und Präsenzlernen) Kursangebots.
Ausblick
Für das Frühjahrssemester 2020 ist geplant, den Blended Learning Kurs für Hilfsassistierende aus anderen Departementen anzubieten. Inhaltlich soll die Transferveranstaltung verlängert werden, um ein Microteaching (Simulation von Unterrichtssituationen) einbauen zu können und den Kurs damit noch praxis- und transferorientierter gestalten zu können. Ein weiterer wichtiger Bestandteil der Weiterentwicklung des Onlinekurses ist die Produktion von weiteren Videos zu verschiedenen Themen der Hochschullehre. Dabei setzen wir den Akzent auf realitätsnahe Inhalte, in denen didaktische Themen anhand von realen Unterrichtssituationen erklärt werden. Didaktische Themen sollen daher auch zukünftig weder durch Animationsvideos noch durch idealisierte oder nachgespielte Unterrichtssequenzen thematisiert werden, sondern anhand von realen Situationen aus unterschiedlichen Übungen und Lehrveranstaltungen. Diese realitätsnahe Form von Lernvideos findet sich bislang eher selten, da es nicht einfach ist, die komplexen sozialen Interaktionen im Unterricht zu extrahieren und aufzuzeigen. Unserer Erfahrung nach haben jedoch die realen Unterrichtssituationen einen stärkeren Lerneffekt als nachgespielte Szenen. Und die Rückmeldungen der Studierenden bestätigen diese Überlegung, weshalb wir überzeugt sind, dass sich der Aufwand für diese Art von selbstproduzierten Videos aus pädagogisch-didaktischen Gründen langfristig auszahlen wird.
It was some years ago, that the Teaching Commission asked LET (the unit for Educational Development and Technology at ETH) to consider ways to involve student in teaching and learning innovation.
In response, the programme “Student Innovedum” was specifically developed. Students were invited to develop prototypes of their own ideas over the duration of a semester. It ran for three years and the results of the student projects were presented each year at the annual Innovedum event and the Learning and Teaching Fair.
While this did increase awareness of the potential of involving students in educational innovation and sparked valuable discussions at ETH, the actual projects and ideas of students did not come to fruition as had originally been hoped. Supporting the students would have required more resources than were available and placed a high burden of work on the (already very busy) students.
Therefore it was decided not to continue Student Innovedum in 2019. Instead, it is our intention to continue the discussion with students, the Teaching Commission and the Rector of ETH in order to decide how to best honour the original request of integrating students in educational innovation.
A working group will be looking at the latest literature and other inspiring examples from around the world to consider ways of engaging students more deeply and in more meaningful ways in funded educational innovation projects at ETH.
We are still at the beginning of this process but would like to take this opportunity to thank all the people who helped make Student Innovedum happen. This includes the wonderful staff at the Student Project House, the Rector Prof. Dr. Sarah Springmann, Vice-rector Prof. Dr. Andreas Vaterlaus, the members of the Teaching Commission, staff at LET and of course all the students who participated and poured so much passion into the process. Thank you to all and watch this space!
Using Polybook to create interactive lecture notes together
As a depository of digital
lecture notes the Polybook has been popular among ETH teaching staff for some
time. In Polybook instructors can enrich conventional lecture material with
interactive elements such as quizzes and videos, and question students on
particularly important material. This increases student engagement with the
material and improves knowledge uptake.
The very name “Polybook” is an
indication of this tool’s many functions and the large number of books and
lecture notes which are stored there.
Using Polybook it is possible to work together on lecture notes or documents and to give them an ordered structure and presentation. Access to individual books is steered via a Moodle link, ensuring that all students in a course have access to its books.
The Polybook is part of LET’s eCollab Service, and can be used in a number of collaborative scenarios. These include:
Author and publish texts collaboratively: In Polybook students can author texts from group or project work either alone or collaboratively, and then make them accessible to others. These texts can be augmented with images or interactive elements such as quizzes and videos. Polybook can also be deployed for preparation or wrap-up of lectures and seminars.
Peer review / peer assessment: Student or working group texts or projects can be exchanged with other student groups and assessed. The results can be used for revision purposes.
Learning journal: Learning processes can be published by students or student groups for purposes of self-reflection or the evaluation of a group process.
Interactive lecture notes: Polybook can be deployed in place of conventional lecture notes, with additional possibilities: students can use interactive elements, or create them themselves; and they can discuss the material via the comments function.
Do you teach at ETH? Have we sparked your interest? How to Polybook contains comprehensive instructions on how you, as teaching faculty, can set up Polybook in Moodle and take the first steps towards collaborative or interactive lecture notes. We would also be happy to provide personal guidance on implementing a collaborative scenario or using Polybook: please contact Melanie Walter, the person responsible for the eCollaboration service. We look forward to working with you.
Case study – Peer Review Food Chemistry Laboratory – Writing reports
As part of a series of case studies, staff at LET sat down to have a conversation with Prof. Laura Nyström and Dr. Melanie Erzinger from the Department of Health Sciences and Technology to discuss their food chemistry laboratory project.
What
is the project about?
We introduced a new way to write lab reports,
combined with a peer review method to foster collaboration and critical
thinking skills among students. In the past students did not have clear
criteria as to what makes a good report. Assistants also needed too much time
to read the reports and give repeated feedback. Thus we looked for a way to
help assistants spend less time on the review process.
We transformed the format of our Food
Chemistry Laboratory Course (Food Science, BSc level, 4th semester)
from a classical lecture format with lab exercises to a blended learning
format. With new videos, we can achieve better coverage of basic knowledge
(i.e. security, handling of equipment).
What
motivated you to initiate the project?
Student numbers have increased over the
past 10 years and we have been losing too much time in covering basic knowledge
repeatedly. Using concept videos, students will be able to review key topics on
their own. Overall, we also wanted to make the entire course more attractive. A
key intention was to develop student skills in report writing and improve
report quality.
How
did you do it?
We defined additional, clear quality
criteria for a good report. During a first round students give each other
feedback, such that final review by teaching assistants and lecturer approval involve
less effort. For each experiment, every student has to review another student’s
report. In total, each reviews four reports over the semester.
Students don’t get a grade for the peer review (semester performance in the lab course is also ungraded). They have one week for each of the four peer reviews, and must complete each by the respective set deadline. They answer various questions related to the quality of the respective report (these involve five aspects plus overall feedback; see the annex at the end of this case study). Students do not “grade” the reports, but give feedback in their own words.
Assistants are aware of what is asked in
the reports and are therefore able to provide targeted and helpful feedback in
the lab which addresses the quality criteria for reports.
We provide the students with online material on how to write reports (short videos, documents etc.). Previously we had a short lecture with examples. Until now, however, we did not train them in conducting proper peer reviews. We have now realised that we need to do this (especially for Bachelor’s degree students), and will include peer review training with the short lecture next year.
Did
you have the support you needed for the project? Is there additional support
you wish you had had to help you to achieve your goals?
We learned about a module inside our LMS for
administering the peer-review process (“workshop module” in Moodle). It would
have been helpful to have had practical tips from others, but apparently not
many lecturers have used this tool. Although the general instructions were
useful, it took quite some time to learn all the aspects of the tool.
Please
describe some of the key outcomes of the project.
Various things changed for the better. Students learned a lot by reading and reviewing the reports of their peers. They gained important input for their own reports. For many it was the first time they had had to give feedback in such a structured way. They also had to find a way to critique something in a good, constructive manner. Overall, students were introduced to a new way of critical thinking and took important first steps in this skill, which is important for their later careers.
We can say clearly that through the new
review method we were able to improve the quality of reports and reduce the
time needed by lecturers to grade them.
How
did the project impact learners or the way in which you teach?
In general the peer review method was well received in the BSc course, and we used the same approach in an MSc-level course. We therefore realised that Bachelor’s degree students need more help and training in peer review than Master’s students.
Overall we saw that the blended learning
approach and the peer review methods work to improve our courses, addressing
the above-mentioned challenges of lack of student preparation and the need to
constantly repeat basic knowledge. Students themselves clearly realised the
value and potential of better collaboration, peer feedback and critical
thinking skills.
What
lessons learned do you want to share with your colleagues?
Not every cohort is the same. While things worked quite well in 2017, in 2018 fewer students adhered to the schedule and deadlines – even though everything was communicated and documented in the same way as in the previous year.
What
are your future plans for this work? How do you plan to sustain what you have
created through the project?
More and more assistants will become competent in providing full reviews of the already peer-reviewed reports. Currently lecturers still have to do this. Lecturers will thus gain more time to be present in the labs and to give 1:1 feedback to students in the lab and online.
We will definitely create some training
material for assistants for this purpose, but it is not available yet. We also want
to create a short, ready-to-use document about giving feedback in our specific
context: what is constructive feedback, what are the do’s and don’ts? Students,
assistants and fellow instructors can use it.
We are interested in learning whether other
lab courses at ETH do something similar, and how. We also need to improve the
support situation with the Moodle review tool “workshop module”. We will
continue to work with it, but it is a bit tricky sometimes.
One additional idea is to make the videos
interactive. Students will see in-built questions in the videos which they have
to answer right away.
Additional
notes regarding resources and tools used.
We used a programme called Labster to create virtual labs in some cases to extend the experience to experiments which were not doable in reality in our labs.
We learned from other courses and departments regarding effective feedback (ETH “Foundations of Teaching and Learning” course).
To conduct the peer review we work with the “workshop module” in Moodle.
To make the videos interactive we will work with the new Moodle “interactive video suite” plugin.
Student voices:
What is your opinion about this course and the peer review
process (lab reports). How has it influenced your learning process?
Robert Spiess: I think peer-reviewing was a great way to see other students’ work. It gave me the opportunity to experience and compare different ways of writing. I could always detect things that I wanted to include in my reports. At the same time, I could see in which points my reports were better, where my advantages were.
I think this procedure is particularly useful when writing.
But the reports should not be too long, because, otherwise, students have to
spend too much time on their own report and neglect, as a consequence, the
peer-reviewing of someone else’s report. Other courses usually required longer
reports. But if the reports were shortened, the method could also be applied in
other laboratory courses (such as in the food processing or in the
biotechnology lab course).
Aline Candrian: I’m glad I did the course, I think it gives students a first impression of laboratory work. The lab report writing is an essential part of the course to understand the experiment and the obtained data. The peer review approach was fine, even though nobody was eager to do them. VERY little time was invested into peer review by most groups, as far as I’m concerned. Of course, sometimes you could benefit from your peer’s feedback but most of the times we didn’t act on them. That’s probably because it was our first time writing (semi-) real reports. We didn’t really know what we were doing and you mostly think you know better than others, especially if someone reviewed your report who you rate less familiar with chemistry.
At the same time you’re well aware that you know nothing
about report writing, so how can you evaluate someone else’s work?
Additionally, motivation was minimal since you were just
glad to be done with writing your report. Having to assess another report and
then correct your own report again was just another ‘burden’. So, altogether, I
would say report-writing was a crucial part of the course but peer review not
so much since we had no experience at all. I think peer review makes more sense
in the courses in our last semester.
Making the students just do a peer review on the last report
might work. They’ll see how it works, they’ll have had written a few reports
(and got more familiar with it) and might be more confident in providing
feedback. But I’m not an expert, it may not work the way I envision it, what do
I know 🙂
Since 2015, the Educational Development and
Technology unit (LET) at ETH has been running lunch-time seminars designed to
provide short insights into innovative teaching at ETH.
These events feature ETH teaching staff speaking and sharing their own ideas and experiences with didactic initiatives. Each event includes an opportunity for attendees to ask questions and discuss ideas among themselves. After the discussion, an optional lunch is provided which creates more time for networking and exchange.
The topics for 2019 have now been announced and include many relevant topics such as “Assessment drives learning”, “Interactive videos” and “Data literacy”. Each year, the topics are crowdsourced both from attendees of previous events, Educational Developers in the departments, as well as LET staff who scan current trends in the higher education teaching world.
Following the individual events, the presentation materials are uploaded on the Refresh Teaching website along with additional relevant resources and photos to help document the event.
ETH staff interested in attending a Refresh Teaching event can check the website for the current presentation topics and are asked to register for catering purposes.
Impartial group assessment. Using peer review and economic theory to grade groups fairly.
In a clear case of practicing what he preaches, Dr. Heinrich Nax has applied game theory to his teaching practice. After lecturing on game theory for several years, he realised that his methods for teaching, more specifically, for assessing did not follow the very theories he was espousing and so he set out to correct this incongruence.
In his course «Controversies in Game Theory» students work in groups and are assessed based on a group project. Social tensions can develop between individual and collective interests in group interactions. One such tension, free-riding, when one person rides the coat-tails of other hard-working group members is well known. There are however additional potential problems when assessing group work such as collusion on grades in cases of peer review. To eliminate these tensions, Dr. Nax decided to implement a mechanism from economic theory to his assessments.
What triggered this approach?
Previously Dr. Nax gave the same final mark to everybody in a particular group regardless of their individual efforts as these could not reliably be assessed. From a game theory perspective this constituted a big temptation for free-riding and Dr. Nax decided to devise something that would incentivize individual efforts but without giving up the benefits of group work altogether.
What exactly did he do?
Influenced by the article Impartial division of a dollar by Clippel, G., Moulin, H., and Tideman, N. (2008), Dr. Nax and his colleague Sven Seuken implemented the article’s mechanism in a blockchain start-up company. The mechanism enables a group to split their financial earnings through peer review between the group members. Group members decide internally what a fair allocation of earnings should be. So he decided to try the mechanism in an educational setting where the “earnings” become the finite amount of points the group works towards, the total of which is determined by the grade he allocates to the group’s total project.
The key idea of the mechanism is that individual group members don’t evaluate their own performance and therefore don’t decide how many points they themselves have “earned”. Instead they allocate relative contributions to the other group members. So in a group of three, if student A thinks group member B did twice as much work as fellow group member C, then B should receive twice as many points as C. Using a specific formula (described in the paper) all three group members reports are then aggregated anonymously to make sure the resulting grades cannot be manipulated. In other words, student A only receives the (aggregated) amount of points, that their colleagues think student A deserves.
What were the results (for student learning)?
Not only was Dr. Nax convinced that the quality of the group projects improved, but the students were happier as well. They believed that the marking was much more fair. It is unclear if this grading method decreased free-riding, however students felt that freeriders did receive lower grades, thus increasing student satisfaction in comparison to grading methods where all members of the group receive the same grade, regardless of effort or contribution.
To see this mechanism in action, visit the Spliddit website which features a demonstration tool. Those interested in learning more should read the original paper or this second (less math-based) follow-up paper or contact Dr. Nax for further information. Dr. Nax is working on a tool to make his grading plan available to other professors.
Case study: Molegram Explorer – A mixed-reality framework for teaching drug design
As part of a series of case studies, staff at LET sat down to have a conversation with Prof. Gisbert Schneider and Dr. Jan Hiss from the Institute of Pharmaceutical Sciences in the Department of Chemistry and Applied Biosciences to discuss their mixed-reality project.
What is the project about?
The Molegram Explorer project provides a mixed-reality framework to facilitate and broaden students’ understanding of molecular structure. It is part of the Computer-Assisted Drug Design course run by Gisbert Schneider, professor in the field of the same name.
At the core of the project is the innovative hardware device “HoloLens” (comprising special glasses with 3D projection, motion sensor and environment scanning, produced by Microsoft). Users of the HoloLens see not only (real) furniture and people present in the room, but also a hologram. In our case, the virtual object is a protein that the students can explore, investigate and even walk through. They literally immerse themselves in the world of molecules.
This innovative concept presents a new way of perceiving molecular structure and facilitates new approaches to chemical structure analysis and design via human-machine interaction.
What motivated you to initiate the project?
This innovative project originated in answer to a call for projects on the use of the HoloLens in ETH teaching. Because we faced a teaching challenge where a 3D representation gave a very good use case, it was a perfect opportunity to apply for a pilot project using HoloLenses.
The idea underpinning our work is that this new technology can help our students to understand certain important principles of molecular structure which traditional teaching methods and media struggle to clarify. To identify or design a suitable drug molecule (the ligand) students must understand the protein’s surface, and particularly the cavities suitable for accommodating the ligand. The HoloLens device helps them visualise the regions of a protein that are accessible to the ligand.
How did you do it? Three groups collaborated on this project. First, Gisbert Schneider and Jan Hiss delivered the scientific content. Guided by specific learning objectives, ETH Zurich’s Educational Development and Technology (LET) unit helped organise and facilitate the development process and provided the required hardware. Finally, a specialised software company (afca) implemented the learning app software.
The HoloLenses are used in a two-week practical course in which students experience a condensed version of early-stage drug discovery. They learn how to computationally screen a catalogue of millions of molecules to identify those that might favourably interact with a particular protein. The students perform a computational analysis and select one or two molecules from the top-ranking candidates. Then they synthesise these compounds and test their activity in the laboratory.
An important basic aspect of protein-ligand interaction is the “solvent-accessible surface”. For beginners, this molecular representation often remains an abstract concept without suitable visualization. By using the HoloLens students can now create surface representations of a protein, interact with the holographic model, and simultaneously discuss it with peers and instructors.
Did you have the support you needed for the project? Is there any additional help you wish you had had?
We had excellent help from the company afca who designed a user-friendly app with an elegant interface. LET helped us with the legal aspects and provided the necessary contacts. The 12 ETH HoloLenses are stored at LET. Although we understand that HoloLenses are not easily available due to their comparably high acquisition cost, it would have been helpful to have faster and easier access to this hardware, especially when we needed to try out and check something quickly.
Please describe some of the key outcomes of the project.
The new tool proved to be a valuable addition to our course. It certainly does not replace traditional teaching and discussion, but it is an example of how technology can enhance the understanding of abstract scientific concepts which are otherwise hard to teach. Because students can virtually navigate the molecular hologram they gain a better understanding of the concept of protein structures and surfaces. In the learning sciences this effect is described by the principle of “embodied cognition”. We were also able to increase the attractiveness of our subject matter with this concrete visual experience. It was a kind of scientific marketing. We received several suggestions for additional projects in the context of other practical exercises. The positive feedback and the success of the pilot has driven us to expand the project with enhanced content and to reach out to other disciplines.
How did the project affect learners or the way in which you teach?
We observed that students became more curious, not only about the specific topic of the learning app but in general about many questions related to protein-structure-based drug design. Students certainly appreciate the new tool. The value of technology-enhanced learning apps for teaching of specific aspects in our field is obvious, and we intend to stay on this route.
What lessons learned would you share with your colleagues?
It is not always realistic or meaningful for scientists and teachers to address app programming and didactic concepts. Therefore, it is important to have experts from complementary fields working hand in hand. Experts on the subject matter can contribute the scientific content, and software developers can create user-friendly and visually appealing interfaces and functions. Learning professionals can then connect content with technological functions. They can also advise on how to transpose learning objectives into an appropriate and technology-enhanced learning process.
Overall, we encourage teachers to try out new methods in teaching, and there is much potential for combining proven learning approaches with new technology. In particular, teachers and students should not fear experiments that do not produce immediate success. “Productive failure” should be regarded as a natural part of the development process; it is a great way to learn.
What are your future plans for this work? How do you plan to sustain what you created through the project?
Based on the many positive outcomes, we plan to develop further apps. The ultimate goal is to adapt the work to a professional context by adding scientific content from our subject matter, together with advanced analysis tools. It would also make sense to develop HoloLens learning apps for selected (teaching) topics in medicine, chemistry and biology. HoloLenses are increasingly employed not only by the entertainment industry, but in business and education generally. We would welcome new, broader applications of this technology at ETH – but always with a critical double-check as to whether it actually provides added value for students compared to conventional teaching methods. In the case of Molegram Explorer, we are extremely satisfied with the learning success achieved.
Feedback from PhD students (Tutors)
What is your opinion about this course and the HoloLens process? How has it influenced your learning process?
Cyrill Brunner: Though it has been clear to me before since I have been doing research in that area before, the visualization of proteins by the HoloLens helped in getting a better feeling for 3D structure of a protein. My personal gain was clearly not so much as for the course participants, but that has nothing to do with the process itself, but the chosen protein (carbonic anhydrase II) which I’ve done research in before. I’m positive that application of the HoloLens process on a novel protein would have helped me clearly to get a better understanding of the 3D confirmation.
Dominique Bruns: The HoloLens is a useful next step in the visualisation of molecules. This hands-on experience allows the understanding of molecular properties, their definition and dependencies. In this regard, the application of the surface area visualisation and determination is an ideal showcase.
Do you think this approach could also be used in other courses?
Cyrill Brunner: Yes, indeed. The HoloLens should clearly be put into use in the lab course of medicinal chemistry as students there are already working with a 2D visualization program. A 3D full insight into what they have been studying beforehand would strengthen their understanding.
Dominique Bruns: I appreciate the 3D perspective and interactivity enabled by the HoloLens, a characteristic that might be especially useful for people that find it difficult to see in three-dimensions.
In my opinion, many further examples could be established and used for didactic purposes, e.g. chemical reaction mechanisms or biological folding events of proteins.
To find out more about how ETH teaching staff can start their own HoloLens project, visit theETH website.