Calling Bullshit: The Art of Skepticism in a Data-Driven World

book

by Carl Bergstrom and Jevin West

Random House, New York (2020), 318 pages

https://www.callingbullshit.org

“Bullshit involves language, statistical figures, data graphics, and other forms of presentation intended to persuade by impressing and overwhelming a reader or listener, with a blatant disregard for truth and logical coherence.”

For most of the book, the authors dig into how scientific results can be compromised at different levels (publications, press releases, social networks). They point at well-known fallacies in statistics and visualizations by providing some surprising examples (many from biology) that they have debunked and well documented by themselves.

In chapter 9, and this was for me the most important part, the authors discuss the primary aims of scientific research, its threats, and why scientific reasoning and scientific methodology still helps us understanding the world better. Especially in the context of the actual discussion on distrust in science (related to the pandemics) this chapter becomes even more relevant.

In their final two chapters the authors provide some general hints on how to identify and on how to refute (call) bullshit. They stress that refuting false claims should be done in a humble, respectful but nonetheless stringent way. Calling bullshit is not a matter of impressing your audience, but it is a moral imperative and at the end it makes ourselves as researchers “more vigilant , a little more thoughtful, a little more careful when sharing information.” (last sentence of the book).

Unfortunately, the book was written in pre-covid19 times. The bulk of actual misinformation and scientific misinterpretation, however, gets easily seizable with the help of this book.

The book is based on a course from the University of Washington, that the authors now have been teaching for some time. Elements from this course and transferred to other subjects suits well for all of our ETH courses and they will empower our students to think more critically on their role as a future researcher and societal policymaker.

I really enjoyed reading this book, written in a funny and personal style, and I can highly recommend it to anyone who is teaching science.

One week of Corona online teaching: experiences and outlooks

The decreed transition from classroom to online teaching caught us off guard at the Department of Physics, as anywhere else. Within a few days, almost 25 central lectures together with five laboratory courses, covering over 3,500 students, had to be switched to online mode. It is due to the high engagement and cooperative commitment of all members from the department that this enormous challenge could be mastered without any transition. None of the courses had to be canceled and since last Monday, all teachers, lecturers as well as teaching assistants, are teaching online.

D-PHYS goes online

Now, after one week of online teaching, it is possible to review the impacts of this switch. In addition, a look into the near and even into the far future should be allowed. How will teaching develop in the coming weeks? What lasting impact will the online shift have on higher education?

In the first days following the shift, the focus was mainly linked to technical problems: What tools should I use to teach online? Here we benefited from the fact that ETH relied on the Zoom platform right from the start. Although other options for teaching, such as recording self-made videos or using Moodle were also recommended, we mainly advertised Zoom at our department and offered an appropriate support. With Zoom, all lectures can be streamed live and students can even ask questions during the event. For courses relying on blackboard writing and experiments, we have set up a special installation in our main lecture halls where lecturers can stream their lecture via Zoom (without students in the hall). Lecturers who do not rely on the large blackboards and on the experiments give their lectures from home by using Zoom as well.

streamed lecture
Live lecture streamed with Zoom from the lecture hall with 141 students in the virtual audience.

With the setting offered by Zoom, it was possible to transfer the concept of the traditional classroom lecture more or less smoothly to online teaching. This was particularly important in the initial phase because the lecturers did not have to redesign their courses. The shift to technical new ground was already demanding enough.

Now, with the lecturers gaining more confidence in using Zoom, didactic reflections are emerging.

Does it really make sense to teach online the same way I have done it before?
Should I try to redesign my course in a more appropriate way for online teaching?

Scenarios such as flipped classroom, question and answer sessions or small project assignments are coming on the agenda. After the technical shift, we now experience a merging didactic shift. Of course, technology still plays an important role, but the focus lies on didactic implications. Lectures profit from their research experience in using communication and cooperation tools. Now they start to transfer those or similar tools to teaching and learning.

At least, the didactic phase is no longer under imminent time pressure. Lecturers have time to plan and to think about creative didactic experiments. Thus, in the near future supporting lecturers in new didactic implementation will become paramount, be it through didactic expertise or by disseminating tested scenarios.

It may seem premature at this point to look at the post Corona period. However, it is already clearly noticeable that traditional teaching is moving to new grounds. An increased number of online courses will continue to be part in higher education when we switch back to face-to-face instruction. Lecturers are now gaining insight in new didactic settings that they will be able to transfer to regular classes. Even brand new ideas might pop up and at the end, students will profit from an increased portfolio of many different teaching and learning opportunities.

With the next exam period ahead in May, the problem of assessments still remains unresolved. Even if oral exams offer a short-term alternative in a few cases, we still have to think about further online alternatives. The next coming months will keep us busy, anyway …

RTOP – ein Bewertungsraster für gute Lehre

Lecture

Wie bewertet man eine Lehrveranstaltung im Rahmen von einer Hospitation? Von der Hochschuldidaktik werden dazu unterschiedliche Instrumente zur Verfügung gestellt, die meist auf eigenen Erfahrungen beruhen, oder auf spezielle Aspekte des Unterrichts ausgerichtet sind.

An der vergangenen AAPT Konferenz hatte ich die Möglichkeit, im Rahmen eines Workshops, RTOP (Reformed Teaching Observation Protocol) auszuprobieren. RTOP wurde in den späten 1990er Jahren an der Arizona State University entwickelt, um Lehrveranstaltungen aus den MINT-Fächern nach den Standards des studierendenzentrierten Unterrichtens zu evaluieren. Es handelt sich um einen Fragebogen mit insgesamt 25 Items, die jeweils auf einer fünfstufigen Likert-Skala («nicht vorhanden» bis «sehr anschaulich») zu beantworten sind. Die Items sind dabei in drei Hauptkategorien aufgeteilt:

  1. Planung und Umsetzung (lesson design and implementation)
  2. Inhalt (content)
  3. Gegenseitiger Umgang (classroom climate)

Typische Items aus RTOP lauten:

The instructional strategies and activities respected students’ prior knowledge and the preconceptions inherent therein.

Connections with other content disciplines and/or real world phenomena were explored and valued.

There was a climate of respect for what others had to say.

Der gesamte Fragebogen (in Englisch) kann hier eingesehen werden:
http://physicsed.buffalostate.edu/AZTEC/rtop/RTOP_full/PDF/RTOPform_IN001.pdf

Üblicherweise protokolliert der Hospitant seine Beobachtungen während der Lehrveranstaltung und füllt den Fragebogen nachträglich aus. Um einheitliche, vergleichbare und zuverlässige Ergebnisse zu ermöglichen, stehen diverse Anleitungen und Videos mit Trainingsbeispielen zur Verfügung.

Das Instrument basiert auf Ergebnissen der Lehr- und Lernforschung, es ist validiert und kann sogar für Forschungszwecke verwendet werden. In einer grossangelegten Studie benutzen Granger et al. (2012) RTOP zur Bewertung der Lehre und konnten damit die erhöhte Lernwirksamkeit von studierendenzentriertem Unterricht nachweisen.

Bei einer Hospitation lässt sich das Instrument jedoch auch, ohne grossen Trainingsaufwand, als Diskussionsbasis im Rahmen einer Nachbesprechung mit den Dozierenden verwenden. Nächste Woche beginnt das Frühlingssemester und ich werde RTOP bei meinen Hospitationen auf jeden Fall ausprobieren. Zu den konkreten Erfahrungen mit RTOP berichte ich dann gerne hier im Blog.

Alle Infos und Materialien zu RTOP finden sich hier:
http://physicsed.buffalostate.edu/AZTEC/RTOP/RTOP_full/index.htm

Eine deutsche Übersetzung liegt vor und kann bei der Projektverantwortlichen Kathleen Falconer (http://www.physikdidaktik.uni-koeln.de/falconer.html) beantragt werden.

Master Thesis Evaluation – a Web Tool

The problem

Grades do not tell anything precise about the student’s abilities in the areas of scientific work, project management, critical thinking and social skills. And, in Switzerland especially, they do not offer any meaning to non-Swiss.

The solution

Additionally to the grade, we offer a report that describes the student’s abilities thoroughly. Since report writing is arduous, we developed a web tool MTE (Master Thesis Evaluation) that will calculate the grade and produce a report in a few clicks. You can still individualize the report afterwards, deleting and adding parts.

Further Advantages

  • Compliance: Grade calculation and partition follow the study guidelines.
  • Fairness: The professors all assess along the same criteria.
  • Service to the student’s future: The report can be part of the student’s portfolio.

Info

More info on MTE can be found in the moodle course “Master Thesis Evaluation“.

Here is an fictive report: