Tag: Peer feedback

Case Study – Peer Review Mastering Digital Business Models

As part of a series of case studies, staff at LET sat down to have a conversation with Prof. Elgar Fleisch, Johannes Hübner and Dominik Bilgeri from the Department of Management, Technology, and Economics (D-MTEC) to discuss their Mastering Digital Business Model (MDBM) course.

What is the project about?

In this Mastering Digital Business Model (MDBM) course, Prof. Elgar Fleisch, Dominik Bilgeri, George Boateng and Johannes Huebner teach Master’s level students a theory- and practice-based understanding of how today’s information technologies enable new digital business models and transform existing ones. The course contains a novel examination mode, a video group project is introduced as a core element contributing to the overall course grade. In addition, students are asked to participate in a peer-to-peer review of the videos produced by other student groups, which is independent of the grading and is geared towards giving students insights in how other groups solved the challenge. The best-rated videos are then shared with the entire class in the end of the semester.

As part of this newly created examination element, course participants (in teams of two to three students) explain one of the major lecture topics (theoretical lenses) in the first half of their video.Then they apply the same lens by analysing a company, aiming to better understand its underlying business model. Companies are pre-selected and allocated to students for fairness reasons. Every year, we choose a pool of interesting companies in the context of digital transformation, the Internet of Things, Blockchain, e-health, etc.

What motivated you to initiate the project?

The core idea was to improve students’ learning success by using an examination format that not only requires learners to reiterate theoretical contents, but also apply the theory in a practical context. The students have different backgrounds, and do not necessarily have a strong business focus, which means that many of the concepts taught in class may be rather abstract. We used the video format and specific companies as case studies, because we think this is a good way to trigger curiosity, show concrete examples of modern companies in a compact form, and force students to reflect deeply upon theoretical frameworks compared to other examination formats.

How did you do it?

Aside from the weekly input lectures, we ask students to form groups in the beginning of the semester. We then provide a list of theoretical core topics from which each group can choose one. In addition, we randomly assign each group to a case company. The theoretical topic then first needs to be explained in the first half of the video, and then be applied to the case company in the second half. Here we thus used a prosumer approach, where students become part of the course because they create a small section of the content. The best videos are shared with the class, and can be reused as additional learning materials for future cohorts. This set-up generally resulted in high-quality videos, perhaps also since students knew their videos will be used again.

Students also had to review the video projects of five other groups. They had to clearly describe whether and how their peers used certain perspectives (called “lenses” in the course) which played a role in the video and in their feedback. In this way they analysed once more how the newly learned concepts were visible in other companies – a positive side effect being that they also honed their reflection and feedback skills.

Did you have the support you needed for the project? Is there additional support you wish you had had to help you to achieve your goals?

We asked two students from previous cohorts to join us as tutors, and support this year’s groups primarily with technical questions about video-making (e.g. tools, quality considerations etc.). In addition, we designed one of the lecture slots as a coaching session during which we would further support student groups with their questions. In total, this approach allowed us to provide the students with high-quality supervision with reasonable effort.

Please describe some of the key outcomes of the project

To most students, the task of creating a video was new. We received feedback that while the initial effort for learning how to make a video was higher compared to other examination formats, it was also fun and very helpful to really understand and apply the new concepts. They said that they learned things more deeply and more sustainably because they had to consider all details and aspects – compared to the practical exercises they are familiar with in other courses. By carefully phrasing their arguments in giving feedback on peer videos, students became more aware of their own thinking and argumentation.

We observed that the questions asked by students once they start creating videos were different and went deeper, i.e. their reflections were based on many concrete examples of companies, and the concepts were put into perspective. The same sub-concepts have a different meaning in another context, and students now see the overarching principles better and can argue more precisely about theoretical aspects. Without these concrete examples, it would have been harder to concretely grasp the theoretical aspects.

How did the project impact learners or the way in which you teach?

We were surprised by the high quality of the best student videos. The teaching team is now really motivated to continue innovating on our approaches in other courses. We saw clearly that when students are very active we get better results, deeper learning and better reflection.

What lessons learned do you want to share with your colleagues?

It can really pay off to try things and to experiment. We think that nowadays the classic format of passive lectures and final exams may not always be the best choice. We believe the improved outcomes through students who were actively engaged by the video assignment justified the investment in developing new approaches and tools.

When considering videos as an examination format, you should define the entire course/project very clearly. When describing what production options students have for videos, you should be very precise. Offering too many options can be counterproductive. It is better to present 3-4 crystal-clear examples and stick to them.

Also, we would recommend managing students’ expectations clearly in the beginning of the semester, and highlighting both the benefits and challenges of this examination format. Of course, this becomes easier after the first year, when you can draw from the experience of the first cohort, and also provide examples of prior videos to illustrate what is expected of the groups. Because the students are co-creators you get new and relevant content which enriches the course and can serve to motivate both students and teachers.

What are the future plans for this work? How do you plan to sustain what you have created through the project?

We plan to optimize some details of this course, and to go even more in the direction of a flipped classroom to use this teaching approach in other courses. We will create a library of the student videos to provide it as additional learning materials in future editions of the course.

Student feedback

By MDBM Student Cristina Mercandetti (mercandc@student.ethz.ch)

  1. Your opinion about this course and the peer review & video production process – how has it influenced your learning process?
    Cristina Mercandetti: I really enjoyed both the course and the video production process. I think they complemented each other very well and we were able to directly apply the theoretical knowledge learned in the course to work on our project. It helped me to think more critically about the course content, and really dive into some of the lenses and models presented. I don’t think this would have been possible without the video production, so it definitely improved my learning process.
  • Do you think this approach could be used in other courses?
    Cristina Mercandetti: Yes, I think this approach could easily be used in other classes. However, I think part of the fun in this class was that the video production was something very new and refreshing (a side effect was that I learned how to cut a short movie). I imagine that if several classes introduced this it would lose some of its novelty and could be stressful, as it took a lot of time.
  • Final remarks about the course
    Cristina Mercandetti: I really enjoyed the whole class, and heard a lot of good things from other students too.

Read more

Posted on ,

Case Study – Peer Review Corporate Sustainability

As part of a series of case studies, staff at LET sat down to have a conversation with Prof. Volker Hoffmann (SusTec, the Group for Sustainability and Technology) and Erik Jentges (Educational Developer) from the Department of Management, Technology and Economics (D-MTEC) to discuss their corporate sustainability project.

What is the project about?

The course “Corporate Sustainability” aims to enable students to become advocates of sustainable business practices in their later careers. Each year it attracts 150-200 students with diverse disciplinary backgrounds and different educational levels (BSc, MSc, and MAS). We adapted the Six Sentence Argument (6SA) method for this course. The method focuses on enhancing critical thinking skills through structured writing and guided, double-blind peer-review.

What motivated you to initiate the project?

We wanted students to get a clearer picture of what sustainability really is. In the course, they develop not only a deeper understanding of corporate sustainability but also the skills to give and receive feedback.

How did you do it?

At the core are four topics that relate to the sustainability of corporations. These are assessment, strategy, technology, and finance. We developed digital learning modules (videos, some with interactive elements) that explain key concepts to support the most relevant and difficult parts of the lecture. Also, we want to develop students’ critical thinking skills. In e-modules, students learn to formulate concise and short arguments with the 6SA method. The core idea builds on the assumption that writing is thinking.

In the e-modules, students face a decision (a micro case based on the lecture content) and argue for their preferred course of action using a logical structure of exactly six sentences. Each sentence fulfils a specific function in the overall argument and has a 20-word limit. A clear grading rubric enables students to assess 6SAs in double-blind peer reviews. These have been continuously adapted and improved since 2015. The specialized online tool “peergrade” also helped us to conduct a smooth process – for both students and teachers.

Through the peer assessment, students engage critically with their peers’ arguments and receive constructive feedback on their own arguments. With the 6SA exercise, students learn to argue with clarity, and it helps them to reflect on the way they and others think.

During the second half of the semester, students work in diverse teams to prepare mock debates, consulting strategies, economic models and campaign videos. In this phase, they are coached by several postdoctoral and doctoral researchers from SusTec, the Group for Sustainability and Technology. The students then present their projects and display their skills in a group puzzle session and are debriefed in the following final lecture session. Students receive grades for both individual and group performance and can earn a bonus on their exam grade when completing the critical thinking exercises.

Did you have the support you needed for the project? Is there additional support you wish you had had to help you to achieve your goals?

The project received funding from different sources. This helped us to hire academic staff to assist the development of new teaching approaches and the production of high-quality videos. In addition, we received specialist guidance in the instructional design and production of videos.

Please describe some of the key outcomes of the project

With regard to our feedback modules, we think that the quality of the argumentation and peer reviews has increased over the years. For example, we learned that the effective design of such peer assessment exercises for students requires training on how to give constructive feedback and that it should involve several feedback loops to support the development and refinement of critical thinking skills. Overall, the course now integrates many innovative teaching elements and was a finalist in the 2018 ETH KITE award.

How did the project impact learners or the way in which you teach?

When students are able to write better and concise arguments that convince critical readers, and if they can give constructive feedback to arguments that are being made to justify strategic decisions, then they are able to actively shape good decisions in a company setting – they can be change-makers for corporate sustainability. The students were motivated by the new teaching approaches such as the supporting videos, interactive questions inside the videos, and the critical thinking exercises. Peer assessment is “homework” for the students, but they know that they can earn a bonus on their exam grade – and they are already rehearsing for some parts of the final exam.

With regard to students’ learning, the peer review process itself is convincing. What is unique to our teaching situation is the incredible diversity in the classroom. A 19-year-old Swiss environmental science student may be sitting next to a 25-year-old Chinese student who is pursuing a master’s degree in management, who in turn sits next to a 35-year-old American part-time student with a PhD in chemistry and a management position with responsibilities for 20 employees in a multinational company. Peer feedback is a powerful solution to bridge these gaps of different levels of experience and cultural backgrounds. It allows younger students to write a creative and brilliant argument without being intimidated by more senior students. It allows a shy and quiet student to gain confidence by formulating a convincing argument whose strengths are recognized in their peers’ feedback. It creates a space for older students to learn how to coach younger classmates with constructive feedback to improve their reasoning.

That is why at D-MTEC, we use peer feedback in other courses as well. Students learn more when actually giving feedback compared to when only submitting an assignment.

What lessons learned do you want to share with your colleagues?

At the beginning, it was a lot of work and many people were involved, but it was worth it. Today, with regard to the critical thinking exercises, we have continuously refined our processes. Every student writes three reviews, thereby ensuring that everyone also receives much more feedback than a single lecturer could provide. The main work for lecturers is providing an overview of the themes in the arguments and summarizing the activity for all students. This lets them know that their individual contribution becomes part of a collective intelligence. There are always truly smart and innovative solutions that need to be shared with the whole class. Also, there is little effort involved in re-grading/moderating student questions about feedback, because we train students to write helpful and considerate feedback and make them aware of that they also have to learn how to receive feedback, especially if it is feedback that they don’t want to, but need to hear.

For the production of videos, we recommend planning enough time and engaging with video experts and instructional designers early on. Especially writing a concise script for a short video requires a surprising amount of time until it effectively conveys your key points.

If you are interested in applying these concepts in your own courses please contact LET.

Note: The project received funding from different sources (Innovedum, Emil Halter Foundation, ETH Critical Thinking Initiative).

Additional resources and comments

Read more

Posted on ,