Author: Pascal Xavier Schmidt

Case study – Peer Review Food Chemistry Laboratory – Writing reports


As part of a series of case studies, staff at LET sat down to have a conversation with Prof. Laura Nyström and Dr. Melanie Erzinger from the Department of Health Sciences and Technology to discuss their food chemistry laboratory project.


What is the project about?

We introduced a new way to write lab reports, combined with a peer review method to foster collaboration and critical thinking skills among students. In the past students did not have clear criteria as to what makes a good report. Assistants also needed too much time to read the reports and give repeated feedback. Thus we looked for a way to help assistants spend less time on the review process.

We transformed the format of our Food Chemistry Laboratory Course (Food Science, BSc level, 4th semester) from a classical lecture format with lab exercises to a blended learning format. With new videos, we can achieve better coverage of basic knowledge (i.e. security, handling of equipment).

What motivated you to initiate the project?

Student numbers have increased over the past 10 years and we have been losing too much time in covering basic knowledge repeatedly. Using concept videos, students will be able to review key topics on their own. Overall, we also wanted to make the entire course more attractive. A key intention was to develop student skills in report writing and improve report quality.

How did you do it?

We defined additional, clear quality criteria for a good report. During a first round students give each other feedback, such that final review by teaching assistants and lecturer approval involve less effort. For each experiment, every student has to review another student’s report. In total, each reviews four reports over the semester.

Students don’t get a grade for the peer review (semester performance in the lab course is also ungraded). They have one week for each of the four peer reviews, and must complete each by the respective set deadline. They answer various questions related to the quality of the respective report (these involve five aspects plus overall feedback; see the annex at the end of this case study). Students do not “grade” the reports, but give feedback in their own words.

Assistants are aware of what is asked in the reports and are therefore able to provide targeted and helpful feedback in the lab which addresses the quality criteria for reports.

We provide the students with online material on how to write reports (short videos, documents etc.). Previously we had a short lecture with examples. Until now, however, we did not train them in conducting proper peer reviews. We have now realised that we need to do this (especially for Bachelor’s degree students), and will include peer review training with the short lecture next year.

Did you have the support you needed for the project? Is there additional support you wish you had had to help you to achieve your goals?

We learned about a module inside our LMS for administering the peer-review process (“workshop module” in Moodle). It would have been helpful to have had practical tips from others, but apparently not many lecturers have used this tool. Although the general instructions were useful, it took quite some time to learn all the aspects of the tool.

Please describe some of the key outcomes of the project.

Various things changed for the better. Students learned a lot by reading and reviewing the reports of their peers. They gained important input for their own reports. For many it was the first time they had had to give feedback in such a structured way. They also had to find a way to critique something in a good, constructive manner. Overall, students were introduced to a new way of critical thinking and took important first steps in this skill, which is important for their later careers.

We can say clearly that through the new review method we were able to improve the quality of reports and reduce the time needed by lecturers to grade them.

How did the project impact learners or the way in which you teach?

In general the peer review method was well received in the BSc course, and we used the same approach in an MSc-level course. We therefore realised that Bachelor’s degree students need more help and training in peer review than Master’s students.

Overall we saw that the blended learning approach and the peer review methods work to improve our courses, addressing the above-mentioned challenges of lack of student preparation and the need to constantly repeat basic knowledge. Students themselves clearly realised the value and potential of better collaboration, peer feedback and critical thinking skills.

What lessons learned do you want to share with your colleagues?

Not every cohort is the same. While things worked quite well in 2017, in 2018 fewer students adhered to the schedule and deadlines – even though everything was communicated and documented in the same way as in the previous year.

What are your future plans for this work? How do you plan to sustain what you have created through the project?

More and more assistants will become competent in providing full reviews of the already peer-reviewed reports. Currently lecturers still have to do this. Lecturers will thus gain more time to be present in the labs and to give 1:1 feedback to students in the lab and online.

We will definitely create some training material for assistants for this purpose, but it is not available yet. We also want to create a short, ready-to-use document about giving feedback in our specific context: what is constructive feedback, what are the do’s and don’ts? Students, assistants and fellow instructors can use it.

We are interested in learning whether other lab courses at ETH do something similar, and how. We also need to improve the support situation with the Moodle review tool “workshop module”. We will continue to work with it, but it is a bit tricky sometimes.

One additional idea is to make the videos interactive. Students will see in-built questions in the videos which they have to answer right away.

Additional notes regarding resources and tools used.

  • We used a programme called Labster to create virtual labs in some cases to extend the experience to experiments which were not doable in reality in our labs.
  • We learned from other courses and departments regarding effective feedback (ETH “Foundations of Teaching and Learning” course).
  • To conduct the peer review we work with the “workshop module” in Moodle.
  • To make the videos interactive we will work with the new Moodle “interactive video suite” plugin.

Student voices:

What is your opinion about this course and the peer review process (lab reports). How has it influenced your learning process?

Robert Spiess: I think peer-reviewing was a great way to see other students’ work. It gave me the opportunity to experience and compare different ways of writing. I could always detect things that I wanted to include in my reports. At the same time, I could see in which points my reports were better, where my advantages were.

I think this procedure is particularly useful when writing. But the reports should not be too long, because, otherwise, students have to spend too much time on their own report and neglect, as a consequence, the peer-reviewing of someone else’s report. Other courses usually required longer reports. But if the reports were shortened, the method could also be applied in other laboratory courses (such as in the food processing or in the biotechnology lab course).

Aline Candrian: I’m glad I did the course, I think it gives students a first impression of laboratory work. The lab report writing is an essential part of the course to understand the experiment and the obtained data. The peer review approach was fine, even though nobody was eager to do them. VERY little time was invested into peer review by most groups, as far as I’m concerned. Of course, sometimes you could benefit from your peer’s feedback but most of the times we didn’t act on them. That’s probably because it was our first time writing (semi-) real reports. We didn’t really know what we were doing and you mostly think you know better than others, especially if someone reviewed your report who you rate less familiar with chemistry.

At the same time you’re well aware that you know nothing about report writing, so how can you evaluate someone else’s work?

Additionally, motivation was minimal since you were just glad to be done with writing your report. Having to assess another report and then correct your own report again was just another ‘burden’. So, altogether, I would say report-writing was a crucial part of the course but peer review not so much since we had no experience at all. I think peer review makes more sense in the courses in our last semester.

Making the students just do a peer review on the last report might work. They’ll see how it works, they’ll have had written a few reports (and got more familiar with it) and might be more confident in providing feedback. But I’m not an expert, it may not work the way I envision it, what do I know 🙂

Read more

Posted on ,

Case study: Molegram Explorer – A mixed-reality framework for teaching drug design

As part of a series of case studies, staff at LET sat down to have a conversation with Prof. Gisbert Schneider and Dr. Jan Hiss from the Institute of Pharmaceutical Sciences in the Department of Chemistry and Applied Biosciences to discuss their mixed-reality project.

 

What is the project about?

The Molegram Explorer project provides a mixed-reality framework to facilitate and broaden students’ understanding of molecular structure. It is part of the Computer-Assisted Drug Design course run by Gisbert Schneider, professor in the field of the same name.

At the core of the project is the innovative hardware device “HoloLens” (comprising special glasses with 3D projection, motion sensor and environment scanning, produced by Microsoft). Users of the HoloLens see not only (real) furniture and people present in the room, but also a hologram. In our case, the virtual object is a protein that the students can explore, investigate and even walk through. They literally immerse themselves in the world of molecules.

This innovative concept presents a new way of perceiving molecular structure and facilitates new approaches to chemical structure analysis and design via human-machine interaction.

 

What motivated you to initiate the project?

This innovative project originated in answer to a call for projects on the use of the HoloLens in ETH teaching. Because we faced a teaching challenge where a 3D representation gave a very good use case, it was a perfect opportunity to apply for a pilot project using HoloLenses.

The idea underpinning our work is that this new technology can help our students to understand certain important principles of molecular structure which traditional teaching methods and media struggle to clarify. To identify or design a suitable drug molecule (the ligand) students must understand the protein’s surface, and particularly the cavities suitable for accommodating the ligand. The HoloLens device helps them visualise the regions of a protein that are accessible to the ligand.

 

How did you do it?
Three groups collaborated on this project. First, Gisbert Schneider and Jan Hiss delivered the scientific content. Guided by specific learning objectives, ETH Zurich’s Educational Development and Technology (LET) unit helped organise and facilitate the development process and provided the required hardware. Finally, a specialised software company (afca) implemented the learning app software.

The HoloLenses are used in a two-week practical course in which students experience a condensed version of early-stage drug discovery. They learn how to computationally screen a catalogue of millions of molecules to identify those that might favourably interact with a particular protein. The students perform a computational analysis and select one or two molecules from the top-ranking candidates. Then they synthesise these compounds and test their activity in the laboratory.

An important basic aspect of protein-ligand interaction is the “solvent-accessible surface”. For beginners, this molecular representation often remains an abstract concept without suitable visualization. By using the HoloLens students can now create surface representations of a protein, interact with the holographic model, and simultaneously discuss it with peers and instructors.

 

Did you have the support you needed for the project? Is there any additional help you wish you had had?

We had excellent help from the company afca who designed a user-friendly app with an elegant interface. LET helped us with the legal aspects and provided the necessary contacts. The 12 ETH HoloLenses are stored at LET. Although we understand that HoloLenses are not easily available due to their comparably high acquisition cost, it would have been helpful to have faster and easier access to this hardware, especially when we needed to try out and check something quickly.

 

Please describe some of the key outcomes of the project.

The new tool proved to be a valuable addition to our course. It certainly does not replace traditional teaching and discussion, but it is an example of how technology can enhance the understanding of abstract scientific concepts which are otherwise hard to teach. Because students can virtually navigate the molecular hologram they gain a better understanding of the concept of protein structures and surfaces. In the learning sciences this effect is described by the principle of “embodied cognition”. We were also able to increase the attractiveness of our subject matter with this concrete visual experience. It was a kind of scientific marketing. We received several suggestions for additional projects in the context of other practical exercises. The positive feedback and the success of the pilot has driven us to expand the project with enhanced content and to reach out to other disciplines.

 

How did the project affect learners or the way in which you teach?

We observed that students became more curious, not only about the specific topic of the learning app but in general about many questions related to protein-structure-based drug design. Students certainly appreciate the new tool. The value of technology-enhanced learning apps for teaching of specific aspects in our field is obvious, and we intend to stay on this route.

 

What lessons learned would you share with your colleagues?

It is not always realistic or meaningful for scientists and teachers to address app programming and didactic concepts. Therefore, it is important to have experts from complementary fields working hand in hand. Experts on the subject matter can contribute the scientific content, and software developers can create user-friendly and visually appealing interfaces and functions. Learning professionals can then connect content with technological functions. They can also advise on how to transpose learning objectives into an appropriate and technology-enhanced learning process.

Overall, we encourage teachers to try out new methods in teaching, and there is much potential for combining proven learning approaches with new technology. In particular, teachers and students should not fear experiments that do not produce immediate success. “Productive failure” should be regarded as a natural part of the development process; it is a great way to learn.

 

What are your future plans for this work? How do you plan to sustain what you created through the project?

Based on the many positive outcomes, we plan to develop further apps. The ultimate goal is to adapt the work to a professional context by adding scientific content from our subject matter, together with advanced analysis tools. It would also make sense to develop HoloLens learning apps for selected (teaching) topics in medicine, chemistry and biology. HoloLenses are increasingly employed not only by the entertainment industry, but in business and education generally. We would welcome new, broader applications of this technology at ETH – but always with a critical double-check as to whether it actually provides added value for students compared to conventional teaching methods. In the case of Molegram Explorer, we are extremely satisfied with the learning success achieved.

 

Feedback from PhD students (Tutors)

What is your opinion about this course and the HoloLens process? How has it influenced your learning process?

Cyrill Brunner: Though it has been clear to me before since I have been doing research in that area before, the visualization of proteins by the HoloLens helped in getting a better feeling for 3D structure of a protein. My personal gain was clearly not so much as for the course participants, but that has nothing to do with the process itself, but the chosen protein (carbonic anhydrase II) which I’ve done research in before. I’m positive that application of the HoloLens process on a novel protein would have helped me clearly to get a better understanding of the 3D confirmation.

Dominique Bruns: The HoloLens is a useful next step in the visualisation of molecules. This hands-on experience allows the understanding of molecular properties, their definition and dependencies. In this regard, the application of the surface area visualisation and determination is an ideal showcase.

 

Do you think this approach could also be used in other courses?

Cyrill Brunner: Yes, indeed. The HoloLens should clearly be put into use in the lab course of medicinal chemistry as students there are already working with a 2D visualization program. A 3D full insight into what they have been studying beforehand would strengthen their understanding.

Dominique Bruns:  I appreciate the 3D perspective and interactivity enabled by the HoloLens, a characteristic that might be especially useful for people that find it difficult to see in three-dimensions. 

In my opinion, many further examples could be established and used for didactic purposes, e.g. chemical reaction mechanisms or biological folding events of proteins.

 

To find out more about how ETH teaching staff can start their own HoloLens project, visit the ETH website.

Read more

Posted on ,