Tips for Writing Effective Multiple Choice Questions

Writing test questions is a daunting task for many instructors. It can be challenging to come up with questions that correctly assess students on the comprehension of course objectives. Multiple choice questions are no exception; despite being very popular, instructors often struggle to create well-constructed questions.

Piece of notebook paper with Questions at the top, followed by numbers and ABCD for each of the six numbers. Answers are circled in red.Multiple choice questions have several advantages. They lend themselves to covering a broad range of content and assessing a wide variety of learning objectives. They are very useful when testing a student’s lower level knowledge of a topic, such as factual recall and definitions, but if written correctly, they can be used to assess at the higher levels of analysis, evaluation, and critical thinking skills. Multiple choice questions are scored efficiently (even automatically, if an electronic test is used), therefore, they are frequently the evaluation method preferred by instructors of large courses.

There are some disadvantages, including the fact that this type of question can be time-consuming to construct. Multiple choice questions are made up of two parts: the stem, which identifies the question, and the alternative responses which include the correct answer as well as incorrect alternatives, known as distractors. Coming up with plausible distractors for each question can be a difficult task. And, while some higher level thinking skills can be addressed, multiple choice questions cannot measure a student’s ability to organize and express ideas.  Another thing to consider is that student success when answering multiple choice questions can be influenced by factors unrelated to the subject matter, such as reading ability, deductive reasoning, and the use of context clues.

The following guidelines are offered to help streamline the process of creating multiple choice questions as well as minimize the disadvantages of using them.

General guidelines for writing stems:

  1. When possible, prepare the stem as a clearly written question rather than an incomplete statement.

Poor Example: Psychoanalysis is….

Better example: What is the definition of psychoanalysis? 

  1. Eliminate excessive or irrelevant information from the stem.

Poor example: Jane recently started a new job and can finally afford her own car, a Honda Civic, but is surprised at the high cost of gasoline. Gasoline prices are affected by:

Better example: Which of the following are factors that affect the consumer price of gasoline? 

  1. Include words/phrases in the stem that would otherwise be repeated in the alternatives.

Poor example: Which of the following statements are true?
1. Slowing population growth can prevent global warming
2. Halting deforestation can prevent global warming
3.  Increasing beef production on viable land can prevent global warming
4.  Improving energy efficiency can prevent global warming

Better example: Which of the following techniques can be used to prevent global warming?
1. Slowing population growth
2. Halting deforestation
3. Increasing beef production on viable land
4. Improving energy efficiency 

  1. Avoid using negatively stated stems. If you must use them, highlight the negative word so that it is obvious to students.

Poor example: Which of the following is not a mandatory qualification to be the president of the United States?

Better example: Which of the following is NOT a mandatory qualification to be the president of the United States?

General guidelines for writing alternative responses:

  1. Make sure there is only one correct answer.
  1. Create distractors that are plausible to avoid students guessing the correct answer.

Poor example:
Who was the third president of the United States?
1. George Washington
2. Bugs Bunny
3. Thomas Jefferson
4. Daffy Duck

Better example: Who was the third president of the United States?
1. George Washington
2. Benjamin Franklin
3. Thomas Jefferson
4. John Adams 

  1. Make sure alternative responses are grammatically parallel to each other.

Poor example: Which of the following is the best way to build muscle?
1. Sign up to run a marathon
2. Drinking lots of water
3. Exercise classes
4. Eat protein

Better example: Which of the following is the best way to build muscle?
1. Running on a treadmill
2. Drinking lots of water
3. Lifting weights
4. 
Eating lots of protein 

  1. When possible, list the alternative responses in a logical order (numerical, alphabetical, etc.)

Poor example: How many ounces are in a gallon?
1. 16
2. 148
3. 4
4. 128

Better example: How many ounces are in a gallon?
1. 4
2. 16
3. 128
4. 148

  1. Avoid using ‘All of the above’ or ‘None of the above’ to prevent students from using partial knowledge to arrive at the correct answer.
  2. Use at least four alternative responses to enhance the reliability of the test.

References:

Brame, C., (2013) Writing good multiple choice test questions. Retrieved December 14, 2016 from https://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/

Burton, S. J., Sudweeks, R. R., Merrill, P.F., and Wood, B. (1991). How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty. Provo, Utah: Brigham Young University Testing Services and The Department of Instructional Science.

“Multiple Choice Questions.” The University of Texas at Austin Faculty Innovation Center, 14 Dec. 2016, https://facultyinnovate.utexas.edu/teaching/check-learning/question-types/multiple-choice.

Amy Brusini, Blackboard Training Specialist
Center for Educational Resources

Image Source: Pixabay.com

The Dead Grandmother Syndrome and How to Treat It

Gravemarker with angel lying face down in grief, holding a wreath.If you are a woman of a certain age, with grandchildren attending college, please watch out for yourself over the next couple of weeks. Your mortality rate is about to increase dramatically.

This is a well-documented phenomenon, first described in a scholarly journal, the Annals of Improbable Research in the November/December 1999 Special Education Issue with The Dead Grandmother/Exam Syndrome by Mike Adams, Department of Biology at Eastern Connecticut State University. Although Adams’ article reports on the results of serious data collection, you should take the conclusions with a small amount of salt. The Annals of Improbable Research — also known as AIR — is a science humor magazine that publishes “…research that makes people laugh and then think.” But even as you may smile reading Adam’s research on the dead grandmother syndrome, it is likely because you recognize it from personal experience. As the end of the semester approaches and exams and papers are due, students who fall behind may resort to excuses for extensions or make-up dates. Your syllabus makes it clear that you don’t offer exceptions. Enter the death of a beloved grandmother.

While some faculty take a hard line on these excuses [Dear Student: Should Your Granny Die Before The Midterm … Chronicle Vitae, January 29, 2015], others have learned from personal experience that sometimes students’ grandmothers actually do die. Brian Thill writes of the conundrums faculty face in dealing with students excuses in The Time of Dead Grandmothers [Inside Higher Ed, March 14, 2006]. He writes: “As teachers, it seems to me we finally have a choice with respect to student excuses: to become cynics or fools. Cynics disbelieve all excuses. (It’s as if they all dissolve into dead grandmothers.) Fools believe them all. … How rightly to regard a student who is lying to you? No question about teaching is harder to answer because no question is less attractive.”

Karen Eifler, associate professor in the School of Education at the University of Portland, Oregon, offers a practical solution in Dealing with Student Deceptions: What to do with ‘Death in the Family’ Excuses [Faculty Focus, March 12, 2009]. Eifler, while well aware of the dead grandmother syndrome, also recognized that students do have deaths in the family and these events demand a sympathetic and courteous response, even while not wanting to encourage students to practice deception. This was her answer to the problem: “[W]hen a student informs me that a close relative has died, I immediately send a condolence card to the whole family, expressing my sympathy for their loss. If the student has been explicit (“It was my grandmother”), I am too. I can also match their vagueness. If the loss was authentic, the family is touched at the gesture, and I am truly glad to have extended that civility. However, if the story was a fabrication, the student finds he or she has some uncomfortable explaining to do to the family, which usually curbs that behavior.” It only took a couple of semesters before word got around about her practice, and it worked to her advantage to be seen as compassionate. Students quickly realized that it was better to speak with her honestly about their need for an extension as “…they figured anyone willing to call their bluff by sending condolences to the whole family would probably treat them with reasonable due process anyway.”

Ultimately it may be most useful to take a look at the underlying cause of the syndrome and address the stress that students are experiencing.

In his article, ‘Tis the Season of Dead Grandmothers [Chronicle Vitae, November 2, 2016], David Gooblar, lecturer in the Rhetoric department at the University of Iowa, questions  “…the assumption that strict discipline is the same thing as demanding a lot from our students.” He states that it is possible to care about your students without being a pushover. Strict policies with no exceptions may “…signal to students that adherence to the rules is more important than any other learning goal we have for them.” The end result may be detrimental to long-term learning. Gooblar prefers to create a “cohesive and supportive” learning community for his students. For example, he allows his students to come up with policies on device use in the classroom, having learned that students are more likely to adhere to policies when they have had a voice in the decision. He writes: “We should strive to create courses in which students want to do the work on time — because we’ve successfully made the case that doing the work on time will benefit them. We should also look to make students trust us enough that if tragedy does strike — sometimes family members do die, you know — they feel comfortable coming to us and explaining why they need some extra time.”

How do you handle student excuses and/or requests for extensions or makeup exams? Please share your policies and solutions in the comments section.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Tweeting the Iliad

Two years ago I wrote a post on Using Twitter in Your Course that described how Margaret Rubega, Associate Professor in the Department of Ecology and Evolutionary Biology at the University of Connecticut with a PhD in ornithology, had used Twitter to promote active learning in a large lecture course. The post also provides some basics on how Twitter works for those unfamiliar with the social networking application. Recently, a colleague, gave me a link to an example of a faculty member using Twitter in a humanities course.

Twitter Logo Blue BirdSjoerd Levelt teaches at Bilkent University, in Ankara, Turkey in the program Cultures, Civilizations and Ideas, a year-long intensive course focusing on the meaning of culture. At Bilkent, instruction is in English. Levelt’s students are from diverse backgrounds and departments “…including computer sciences, mechanical, electrical and industrial engineering, law, archaeology, and management.” Levelt has a blog, and recently he posted on a course he is teaching on ancient and classical civilizations, covering texts ranging from the Epic of Gilgamesh to Plato’s Republic. The blog post, #Iliad, discusses the challenges he faced in teaching this text (The Iliad) and how having his students use tweeting, provided a solution.

Levelt writes: “The Iliad is not an easy text to read. Robert Fagles’ translation is not an easy translation to read. This would be true for most students (actually, most readers); and my students are further disadvantaged in that for the vast majority of them, English is not their native language, and many of them, they don’t read (or even like reading) literature all that much to begin with.” He wanted to “… explore with my students how we can engage with the classical text through various media…”.

First he had students look at other examples of how tweeting had been used to comment on literary works. Then he asked them to discuss how and why tweeting was effective in these cases. Students recognized that tweeting provides summary, explanation, commentary, and humor, among other things. In fact, reading with the idea that one will need to summarize, comment, and explicate the text in short sentences, forces students to read closely in order to grasp complexity and subtlety.

Levelt did not require the students to set up Twitter accounts. Instead he had them tweet on prepared “tweet sheets” that replicated the look of a Tweet. The assignment was structured; students were assigned to tweet as one of the characters in Book 3 (the focus of the assignment).

#Iliad, Book Three
Write “tweets” –on paper!– describing (a selection of) the narrative of book 3, from the perspective of the person whose identity you have been handed in class:
Paris (@FoolForLove)
Menelaus (@BattleHungry)
Helen (@TooPretty)
The Achaeans (@NotAHorse)
The Trojans (@HorseBreakers)
Use all 6 “tweets” of your handout to give your version of Book 3. Each tweet: maximum 140 characters. Bonus for creative use of hashtags and @mentions. Perspective: think, for example, of what your character can know, what they would find important, how they would view certain actions and events, what kind of language they would use.

After the students had completed the assignment, he had them come together in groups in class based on their characters (all the Helens, all the Trojans, etc.). The groups selected tweets from the combined sheets to share. Levelt then, with the permission of the students, shared their tweets on his Twitter account. He teaches three sections of the course, so there are three classes represented. These are their tweets: Class 1, Class 2, Class 3.

Levelt writes: “I was very impressed with the range and variety of aspects of the text reflected in my students’ tweets – from Helen’s conflicted internal monologues to Menelaus’ asking the troupes for retweets, and from a baffled Menelaus wondering what just happened after Paris disappeared to Helen’s shocked ‘selfie’ watching the battlefield, there are many very interesting readings of the text, and very few poor ones. Many of the tweets also provided opportunity for further discussion in class.” Moreover, students commented that the assignment made them look at the text with fresh eyes and engage with it based on their own experiences, leading to reading in a way that they had not done before.

This is an innovative use of social media to stimulate student engagement and higher level learning. Please share in the comments section examples you might have of similar assignments.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Twitter blue logo https://about.twitter.com/press/brand-assets

Quick Tips: Teaching in Challenging Times and Facilitating Difficult Discussions

In the days following the election faculty and students across the country were faced with Image of a stylized human figure peering into the opening of a large circular maze.teaching and learning in a climate that made both activities difficult. The issues that divided our nation could not be ignored in the classroom. The Center for Teaching at Vanderbilt University published a thoughtful guide for faculty: Teaching in Response to the Election, by Joe Bandy, CFT Assistant Director. The suggestions are practical, reference additional resources, and are useful not just today, but in thinking about supporting students in general. Three other CFT guides are referenced: Teaching in Times of Crisis for when “communities are united in grief or trauma,” Difficult Dialogues will be useful whenever topics of discussion in the classroom touch on “hot button” issues, and the guide for Increasing Inclusivity in the Classroom is relevant at all times.

We welcome your suggestions in the comments for facilitating difficult discussions and teaching in challenging times.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image source: Pixabay.com

Consider the OER (Open Educational Resource)

I should first disclose that I am not a longstanding, seasoned user of online strategies in my pedagogy. In fact, aside from very basic use such as posting images online for my students to review, my first real foray into systematic, thought-through online pedagogical strategies began in the summer of 2015.

A stitched image showing the Ishtar gate of Babylon in full view. Pergamon Museum, Berlin.In my discipline, specifically the study of Mesopotamian art but more broadly art history, I see two somewhat different audiences for online resources: 1) students (or student-like users) looking for content about art history; and 2) educators looking for pedagogical support/sharing related to the teaching of art history.

With respect to online resources for student-like users, two main trends in online pedagogy are apparent: 1) how to recreate and/or enhance the kind of activities that take place in face-to-face teaching; 2) how to add to, that is do something different, from the kind of activities that take place in face-to-face teaching.

My own foray into online pedagogy was primarily aimed at student-like users, although a secondary audience of other educators is also relevant because of the open-access nature of my project.

The arts of Mesopotamia – the “land between the rivers” in what is today Iraq and Syria – represent some of the earliest complex artworks dating back to 3500. Works from intricately carved seals to sculpture offer a wealth of arts that inform on the social, political, economic, and religious spheres of multiple ancient cultures, including Sumer, Babylonia, and Assyria. The cultural heritage of Mesopotamia is particularly threatened at the moment due to the current political situation in Iraq and Syria.

Teaching this material at the undergraduate level, however, is a challenge as there is no reliable, up-to-date textbook available; the most recent usable textbook dates to 1954 (H. Frankfort, Art and Architecture of the Ancient Orient). Publication of a traditional, hard-copy textbook now is considered financially impractical.

In the spring of 2015, pursuing a Technology Fellows grant from the CER, I proposed a solution: to create on-line modules to be used in teaching my course Palaces, Temples and Tombs in Mesopotamia in fall 2015. These modules are designed as Open Education Resources (OER) using a pre-existing Internet platform, OpenStax CNX, hosted through Rice University, which promotes the production of small “knowledge chunks” in an open license venue. Materials for the modules consist of freely available content and content created by me and my graduate student fellows, Megan Lewis and Avary Taylor.

What is an OER? From the William and Flora Hewlett Foundation “Open Educational Resources are teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use and repurposing by others. OER include full courses, course materials, modules, textbooks, streaming videos, tests, software, and any other tools, materials, or techniques used to support access to knowledge.”

OER modules of instruction permit multi-media and non-traditional formats for conveying information, including virtual reconstructions and walk-throughs, videos, and hyperlinking in addition to providing up-to-date informational entries for the ancient artworks. For my course, I envisioned these modules as a means of engaging students before actual face time in the classroom in order to concentrate on discussion and exploration of the complex conceptual aspects of Mesopotamian art and culture during class time.

Over the 6-month period of the fellowship, five different modules were created and posted to the website at OpenStax CNX. They were an enormous asset to the class, because they provided background information and discussion points that were up-to-date in their content and specifically formulated to align with my class lectures and discussion. The modules also included helpful videos and virtual reconstructions of the ancient art that provided a fuller understanding for the students.

The online modules were evaluated through an online survey, developed with the aid of CER, and available to the students through JHU’s Blackboard (learning management system). All 12 students completed the anonymous survey, which consisted of 5 questions. 83.3% of the respondents said the modules were “very successful” in providing information related to the course content, while the remaining 16.7% said they were “somewhat successful.” The responses to the other questions were also generally quite positive, with appreciation for the multimedia components and for the fact that the modules aligned well with the lectures. Respondents found least useful about the modules some formatting issues inherent in the platform we used, and a few noted that they were slow to download.

Beyond the student reactions, I have had positive responses from colleagues in the field who expressed gratitude for making freely accessible materials on Mesopotamian art available.

The one downside for me was that the OERs did not necessarily promote a higher level of discussion as I had hoped; the modules were still too close to a textbook in terms of how students interacted with the materials

There were a few issues that we faced in developing the content, one of which was copyright.  We had to rely on what was freely available online and that sometimes meant using videos that contained inaccurate material. We also had to work with the OpenStax CNX version of an html coding program that made certain things difficult to manipulate and constrained format in terms of relationship of image to text.

These drawbacks did not discourage me from using OERs. In spring 2016, I received an additional grant through the CER’s Technology Fellows program to produce more modules for my teaching with the assistance of graduate student fellows Megan Lewis and Avary Taylor.

The modules can be accessed through various search mechanisms on the OpenStax CNX website, including through the authors’ names: Marian Feldman, Megan Lewis, and Avary Taylor. They are:

  1. Cylinder Seals and the Development of Writing in Early Mesopotamia http://cnx.org/contents/863d1f28-bad9-42ab-a74c-c602256f9908@1/Cylinder-Seals-and-the-Develop
  2. Ur III: Continuity and Erasure http://cnx.org/contents/30f1bbbc-6341-4e2a-8d2a-53600a36a30d@1/Ur-III-Continuity-and-Erasure
  3. Late Bronze Age Internationalism and the International Artistic Style http://cnx.org/contents/98680d11-2374-4a98-aa91-d2708e2beff1@3/Late-Bronze-Age-Internationali
  4. Neo-Assyrian Palace Reliefs of Kings Tiglath-Pileser III and Sargon II http://cnx.org/contents/299a9d11-5c05-49c8-9844-6f042208b15c@1/Neo-Assyrian-Palace-Reliefs-of
  5. The Ancient City of Babylon http://cnx.org/contents/d49e45c8-931e-4dfd-a3e3-1d0dc0008d55@1/The-Ancient-City-of-Babylon
  6. Mesopotamian Votive Statuary from the Early Dynastic Period https://cnx.org/contents/k64PgmY0@1/Mesopotamian-Votive-Statuary-f
  7. Mesopotamian Cosmology and Mythology https://cnx.org/contents/OCYI18Df@1/Mesopotamian-Cosmology-and-Myt
  8. The Development of Sumerian Temple Architecture in Early Mesopotamia https://cnx.org/contents/Yip68Fa2@7/The-Development-of-Sumerian-Te
  9. Sargon the Great and the Charismatic Rulers of Ancient Akkad of Mesopotamia https://cnx.org/contents/4LSqiUv0@2/Sargon-the-Great-and-the-Chari
  10. The Babylonian Map of the World: A Portrayal of Mytho-Historic Reality https://cnx.org/contents/yM0T6acv@2/The-Babylonian-Map-of-the-Worl
  11. The ‘Victory Stele’ of Naram-Sin of Akkad and the Development of the Public Monument in Ancient Mesopotamia https://cnx.org/contents/YUbLWN2X@1/The-Victory-Stele-of-Naram-Sin

Marian Feldman, Professor, Departments of the History of Art and Near Eastern Studies, Johns Hopkins University

Image Source: A stitched image showing the Ishtar gate of Babylon in full view. Pergamon Museum, Berlin. Photo CC Radomir Vrbovsky, Wikimedia Commons.

 

Lunch and Learn: Flipped courses: What is the purpose? What are the strategies?

Logo for Lunch and Learn program showing the words Lunch and Learn in orange with a fork above and a pen below the lettering. Faculty Conversations on Teaching at the bottom.On Thursday, October 20, the Center for Educational Resources (CER) hosted the first Lunch and Learn—Faculty Conversations on Teaching for the 2016-1017 academic year. A panel of faculty including Avanti Athreya, Assistant Research Professor Applied Mathematics & Statistics; Michael Falk, Professor Materials Science & Engineering; Bob Leheny, Professor Physics & Astronomy; and Soojin Park, Assistant Professor Cognitive Science; spoke briefly on their experiences and engaged in a lively discussion with attendees on Flipped courses: What is the purpose?  What are the strategies?

Avanti Athreya described flipping a large lecture course in Fall 2015 with her colleague, Dan Naiman, Professor, Applied Math & Statistics. The 4 credit course, Statistical Analysis I had previously met four times a week for 50 minutes – three lectures by faculty and one small-group meeting led by a TA.  Starting in Fall 2015, students watched several short videos (5-15 minutes each) before the week started.  The videos were created by Athreya and Naiman using Camtasia. Students then met once for a 75-minute lecture with the instructor and twice in small-groups with a TA.  During these sessions students, working in teams of three, solved problems with a TA available for help as needed.  Clicker quizzes were given at the beginning of each lecture to motivate students to watch the videos. Athreya noted that clear learning objectives were listed at the beginning of each video. Challenges included initial resistance from the students (she stated that there had been less of that this semester, the second iteration of the flipped course), and that students often need alternative explanation for concepts. Typically, the videos cover an idea in one way. In a lecture, the instructor noting confusion may offer another explanation for clarification.

Soojin Park co-teaches Cognitive Neuroscience: Exploring the Living Brain with Brenda Rapp, Professor, Cognitive Science. This 3 credit course has an enrollment on average of 250 students. Park and Rapp flipped their course in Spring 2016, with a goal of putting more emphasis on student exploration. They videotaped scripted lectures (these videos were shorter and more focused than the lectures in the traditional course) and posted them on Blackboard. Students took quizzes on the video content. Students met twice a week in sections of about 25. One section was structured as a review section, the other as an active learning section. The challenge was to create the active learning activities. They decided to emphasize practical skills, such as exercises to learn spatial areas of the brain using 3-D software. These activities were all group based. There were worksheets for each session. For the final project, students developed a mock NIH proposal. Park and Rapp found a 5% learning improvement on the final exam (the questions were reused from the previous year to allow comparison) as well as higher course evaluations.

Bob Leheny reported that he is in the fourth year of teaching an active-learning version of Introduction to Physical Sciences, which incorporates a flipped classroom model. The course serves 700 students each semester. Before class, students watch videos that were developed at the University of Illinois. Leheny noted that there is a great deal of video content already developed for teaching introductory physics, so the faculty developing the course here were spared having to create their own. Faculty are able to track how much time students spend watching the videos. The course was developed with funding from a JHU Gateway Sciences Initiative grant, which included the design and implementation of an active learning classroom that seats 80 students. In the classroom, students review the video content, then work collaboratively in groups of three on exercises and experiments that explore the topic for the day. The course is supported by three graduate student TAs and four undergraduate TAs. Leheny said that one of the challenges was time management in the active learning setting. He compared the instructor and TAs to “waiters working the tables” where students were doing the activities and exercises. There is a constant monitoring of where students are and what they need.

Michael Falk was an early adopter of flipping the course. He now flips two courses: his undergraduate Computation and Programming for Materials Scientists and Engineers, with an enrollment of 35, and a graduate course, Thermodynamics of Materials. For the undergraduate class he created his own videos using Screen Flow. Students take quizzes on the video content before class. In class students work through exercises collaboratively. Falk uses Class Spot to facilitate this work. Class Spot allows screen sharing; students can see how their classmates worked out solutions to problems. For his graduate course in thermodynamics, Falk made short, Khan Academy-style videos using Quick Time. The students watch the videos before class and use class time for problem solving. He also made use of an application called Perusall for annotation exercises. His found in general that his students like it better if there is a short recap of the video material at the beginning of class. Falk feels that the biggest challenge with flipping is finding meaningful activities for class time.

Some key points covered during discussion included:

  1. Making sure that students aren’t assigned too much to do outside of class–videos should replace some of the reading or other homework assignments.
  2. It may be necessary to incentivize students to watch the videos. This can be in the form of quizzes.
  3. If group or collaborative work is done in class, follow best practices for creating groups. Groups of three are ideal. It is best not to have two males and one female in a group as has been shown in research on gender construction of teams. Group work presents valuable experiences for students. For those going into STEM fields, collaboration will be the norm, thus is a good skill to acquire. Group work can help minimize the negative aspects of competition in a classroom.
  4. Base in-class activities on the student learning goals for the course.
  5. Keep videos short, even, or especially when using a lecture-style delivery of the content. Scripting of lecture delivery was advised, as well as adopting a modular concept. Each lecture video should focus on one idea.
  6. Faculty who had flipped their courses noted that preparation for the initial offering of the course took a tremendous investment of time, but that the results had been worth the effort involved.
  7. Several faculty from the humanities discussed whether a flipped model could be used in their class situations, and specifically whether video delivery offered any advantage over reading a text. Certainly offering a variety of learning modalities can be valuable for students coming to a course with different backgrounds and understanding. A humanities course might not benefit from being flipped in total, but having students work together in class to develop specific skills, such as close reading, could prove valuable.

In all, the session was interesting and informative. If you are an instructor on the Homewood campus, staff in the Center for Educational Resources will be happy to talk with you about flipping a course.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image source: Lunch and Learn logo by Reid Sczerba, Center for Educational Resources.

Silence is Golden

A recent post in Tomorrow’s Professor by Joseph Finckel, Associate Professor of English at Asnuntuck Community College in Connecticut, suggested an innovative approach to teaching courses that have a discussion-based component. He writes: “I teach English, and midway through the spring 2013 semester, I lost my voice. Rather than cancelling my classes, I taught all my courses, from developmental English to Shakespeare, without saying a word.”

Black and white drawing of a man with his mouth taped shut.In The Silent Professor, Finckel notes that with an instructor-centric approach, talking is often confused with teaching. What he observed when he had laryngitis has compelled him to “lose his voice” at least once a semester since. “A wealth of literature focuses on active learning and learner-centered instruction, but I submit that nothing empowers learners as immediately and profoundly as does removing the professor’s voice from the room.”

Finckel points out that there are non-verbal actions the instructor can employ such as writing on the board, posing questions by typing into a projected document, and using gestures. Further, he tells us that considering when and for what reasons to speak assists developing “…an intentional, reflective teaching practice.” Student response has been positive. Finckel feels that is because he is creating a situation where learning will occur. “Teaching without talking forces students to take ownership of their own learning and shifts the burden of silence from teacher to student. It also forces us to more deliberately plan our classes, because we relinquish our ability to rely on our knowledge and experience in the moment.”

Although such an approach wouldn’t be appropriate for a large lecture class, it is useful to think about whether talking too much or too soon inhibits students. In working with faculty who teach discussion-based courses, one pitfall is being afraid of the silence after asking a question. It’s all too easy to fall into the habit of answering the question yourself when the silence is deafening. That simply reinforces the students’ belief that if they wait long enough, they’ll be off the hook.

Check out the article for more details on implementing the silent approach. Maybe that next case of laryngitis will be an opportunity rather than bad luck.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image source: Pixabay.com

Making Maps Making Connections

Using mapping as a learning tool for students offers several outcomes. Students develop skills in framing material within temporal and geospatial constructs. The ability to layer data and various media types in creating a map furthers critical thinking and gives students opportunities to understand course content in a complex spatial context. Mapping can be thought of beyond the sense of traditional cartography; we can use images of the universe, floor plans of a building, or molecular structures as the basis for maps on which students can build a story pertaining to their course work and/or research. Fortunately, there are some great tools, freely available, for you and your students to use for mapping projects.

Previously in a post on Resources for Multimedia Creation (October 8, 2014) I mentionedAn 1691 French map of the city of Kamianets-Podilskyi, located in western Ukraine. Google Maps for developers. “With Google Maps Application Programming Interface (API) users can expand, customize, and embed maps and mapping tools into their websites. This includes combining Flickr (the photo sharing website) content with maps. These work well with Google Sites and Google Docs.” Check out the tutorials and articles to get an idea of the types of projects Google Maps will support.

Harvard World Map, developed at Harvard University, is described as “…an online, open source mapping platform developed to lower barriers for scholars who wish to explore, visualize, edit, and publish geospatial information.  The system attempts to address the gap between desktop GIS which is generally light on collaboration, and web-based mapping systems which often don’t support the inclusion of large datasets.” Harvard World Map allows users to import and make visual large GIS data sets. The application facilitates the use of multiple layers to create complex visualizations. Maps can be kept private or shared. There are examples on the homepage as well as a large number of shared maps found under View a Map. This would be a good option for someone wishing to examine correlations among several data sets without having to deal with the steeper learning curve of a program such as ArcView GIS.

For those using Omeka [see Omeka.org, Omeka.net, and a previous Innovative Instructor post, Omeka for Instruction], the Neatline plugin offers a set of tools to allow “…scholars, students, and curators to tell stories with maps and timelines.” Neatline was developed at the Scholars’ Lab at the University of Virginia Library. Omeka and Neatline are designed specifically to support online collections and exhibitions. Take a look at the demos to get a sense of the rich and complex ways in which cultural heritage artifacts, photographs, or other documentation can be layered over maps to provide complex and nuanced interpretive readings of the collected materials.

If you are teaching in the Krieger School of Arts & Sciences or the Whiting School of Engineering at Johns Hopkins, there is another option: Reveal.  Developed here at the Center for Educational Resources, Reveal uses mapping, in the sense of the term that refers to hierarchical image mapping, combined with annotation. “Reveal is a web application for annotating images with rich multimedia content. Using Reveal, you can create a website where image annotations link to image, audio and video resources to illustrate visual relationships.” Watch the video to get a better idea of how Reveal works. Reveal uses JHU authentication and for the present is available only to those teaching on the Homewood Campus.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image source: Pixabay.com – An 1691 French map of the city of Kamianets-Podilskyi, located in western Ukraine.

PowerPoint in the Classroom

Do you use PowerPoint (or Keynote, Prezi or other presentation software) as part of your teaching? If yes, why? This is not meant to be a question that puts you on the defensive, rather to ask you to reflect on how the use of a presentation application enhances your teaching and fits in with other strategies to meet your learning objectives for the class.

Cartoon-like drawing of a presenter showing a slide to a sleeping/snoring audience.It’s been almost three years since The Innovative Instructor wrote on using PowerPoint in the classroom. See Polishing your PowerPoints, a post that covered some tips for creating more effective slides, citing a book by Nancy Duarte called Slide:ology [Nancy Duarte, Slide:ology,  O’Reilly Media, Inc., 2008].

A key point from that post to reiterate: “Duarte reports on research showing that listening and reading are conflicting cognitive processes, meaning that your audience can either read your slides or listen to you; they cannot do both at the same time. However, our brains can handle simultaneous listening to a speaker and seeing relevant visual material.”

It’s important to keep this in mind, particularly if your slides are text heavy. Your students will be scrambling to copy the text verbatim without actually processing what is being said. On the other hand, if your slides are used as prompts (presenting questions or key points with minimal text) or if you don’t use slides at all, students will have to listen to what you are saying, and summarize those concepts in their notes. This process will enhance their understanding of the material.

An article in Focus on Teaching from August 1, 2012 by Maryellen Weimer, PhD asks us to consider Does PowerPoint Help or Hinder Learning? Weimer references a survey of students on the use of PowerPoint by their instructors. A majority of students reported that all or most of their instructors used PowerPoint. Weimer’s expresses the concern that “Eighty-two percent [of students surveyed] said they “always,” “almost always,” or “usually” copy the information on the slides.” She asks, “Does copying down content word-for-word develop the skills needed to organize material on your own? Does it expedite understanding the relationships between ideas? Does it set students up to master the material or to simply memorize it?” Further, she notes that PowerPoint slides that serve as an outline or use bulleted lists may “oversimplify” complex content, encourage passivity, and limit critical thinking.

Four journal articles from Cell Biology Education on PowerPoint in the Classroom (2004 Fall) present different points of view (POV) on the use of PowerPoint. Although written over a decade ago, most of the concepts are still relevant. Be aware that some of the links are no longer working. From the introduction to the series:

Four POVs are presented: 1) David Keefe and James Willett provide their case why PowerPoint is an ideal teaching software. Keefe is an educational researcher at the Center for Technology in Learning at SRI International. Willett is a professor at George Mason University in the Departments of Microbial and Molecular Bioscience; as well as Bioinformatics and Computational Biology. 2) Kim McDonald highlights the causes of PowerPointlessness, a term which indicates the frequent use of PowerPoint as a crutch rather than a tool. She is a Bioscience Educator at the Shodor Education Foundation, Inc. 3) Diana Voss asks readers if PowerPoint is really necessary to present the material effectively or not. Voss is a Instructional Computing Support Specialist at SUNY Stony Brook. 4) Cynthia Lanius takes a light-hearted approach to ask whether PowerPoint is a technological improvement or just a change of pace for teacher and student presentations. Lanius is a Technology Integration Specialist in the Sinton (Texas) Independent School District.

These are short, op-ed style, pieces that will further stimulate your thinking on using presentation software in your teaching.

For more humorous, but none-the-less thought provoking approach, see Rebecca Shuman’s anti-PowerPoint tirade featured in Slate (March 7, 2014): PowerPointless. With the tagline, “Digital slideshows are the scourge of higher education,” Shuman reminds us that “A presentation, believe it or not, is the opening move of a conversation—not the entire conversation.”

Shuman offers a practical guide for those, like her, who do use presentation software, but seek to avoid abusing it. “It is with a few techniques and a little attention, possible to ensure that your presentations rest in the slim minority that are truly interactive and actually help your audience learn.” Speaking.io, the website Shuman references, discusses the use of presentation software broadly, not just for academics, but has many useful ideas and tips. 

For a resource specific to academic use, see the University of Central Florida’s Faculty Center for Teaching & Learning’s Effective Use of PowerPoint. The experts at the Center examine the advantages and challenges of using presentation software in the classroom, suggest approaches to take, and discuss in detail using PowerPoint for case studies, with clickers, as worksheets, for online (think flipped classes as well) teaching, the of use presenter view, and demonstrate best practices for delivery and content construction.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: CC Oliver Tacke https://www.flickr.com/photos/otacke/12635014673/

To Curve or Not to Curve Revisited

Yellow traffic signs showing a bell curve and a stylized graph referencing criterion-referenced grading.The practice of normalizing grades, more popularly known as curving, was a subject of an Innovative Instructor post, To Curve or Not to Curve on May 13, 2013. That article discussed both norm-referenced grading (curving) and criterion-referenced grading (not curving). As the practice of curving has become more controversial in recent years, an op-ed piece in this past Sunday’s New York Times caught my eye. In Why We Should Stop Grading Students on a Curve (The New York Times Sunday Review, September 10, 2016), Adam Grant argues that grade deflation, which occurs when teachers use a curve, is more worrisome than grade inflation. First, by limiting the number of students who can excel, other students who may have mastered the course content are unfairly punished. Second, curving creates a “toxic” environment, a “hypercompetitive culture” where one student’s success means another’s failure.

Grant, a professor of psychology at the Wharton School at the University of Pennsylvania, cites evidence that curving is a “disincentive to study.” Taking observations from his work as an organizational psychologist and applying those in his classroom, Grant has found he could both disrupt the culture of cutthroat competition and get students to work together as a team to prepare for exams. Teamwork has numerous advantages in both the classroom and the workplace as Grant details. Another important aspect is “…that one of the best ways to learn something is to teach it.” When students study together for an exam they benefit from each other’s strengths and expertise. Grant details the methods he used in constructing the exams and how his students have leveraged teamwork to improve their scores on course assessments. One device he uses is a Who Wants to Be a Millionaire-type “lifeline” for students taking the final exam. While his particular approaches may not be suitable for your teaching, the article provides food for thought.

Because I am not advocating for one way of grading over another, but rather encouraging instructors to think about why they are taking a particular approach and whether it is the best solution, I’d like to present a counter argument. In praise of grading on a curve by Eugene Volokh appeared in The Washington Post on February 9, 2015. “Eugene Volokh teaches free speech law, religious freedom law, church-state relations law, a First Amendment Amicus Brief Clinic, and tort law, at UCLA School of Law, where he has also often taught copyright law, criminal law, and a seminar on firearms regulation policy.” He counters some of the standard arguments against curving by pointing out that students and exams will vary from year to year making it difficult to draw consistent lines between, say an A- and B+ exam. This may be even more difficult for a less experienced teacher. Volokh also believes in the value of the curve for reducing the pressure to inflate grades. He points out that competing law schools tend to align their curves, making it an accepted practice for law school faculty to curve. As well, he suggests some tweaks to curving that strengthen its application.

As was pointed out in the earlier post, curving is often used in large lecture or lab courses that may have multiple sections and graders, as it provides a way to standardize grades. However, that issue may be resolved by instructing multiple graders how to assign grades based on a rubric. See The Innovative Instructor on creating rubrics and calibrating multiple graders.

Designing effective assessments is another important skill for instructors to learn, and one that can eliminate the need to use curving to adjust grades on a poorly conceived test. A good place to start is Brown University’s Harriet W. Sheridan Center for Teaching and Learning webpages on designing assessments where you will find resources compiled from a number of Teaching and Learning Centers on designing “assessments that promote and measure student learning.”  The topics include: Classroom Assessment and Feedback, Quizzes, Tests and Exams, Homework Assignments and Problem Sets, Writing Assignments, Student Presentations, Group Projects and Presentations, Labs, and Field Work.

Macie Hall, Instructional Designer
Center for Educational Resources


Image Source: © Reid Sczerba, 2013.