Category Archives: Best Practices

How Pretesting Can Help Your Students Fail Well

Posted on by
1

Our previous post When Failure is a Good Thing, looked at an initiative at Smith College called Failing Well, a set of programs that helps student understand that failing can lead to better learning. Today, The Innovative Instructor offers a concrete way in which you can introduce students to that concept.

Piece of lined, loose leaf notebook paper with six multiple choice questions. ABC or D is circled in red for each question.In Why Flunking Exams Is Actually a Good Thing (New York Times Magazine, September 4, 2104), Benedict Carey discusses the benefits of pretesting. He asks us to imagine that on the first day of a course we illicitly got a copy of the final exam. Would it help us to study more effectively and better attend to course readings, lecture materials, and class discussions? Undoubtedly it would. He then asks, “But what if, instead, you took a test on Day 1 that was just as comprehensive as the final but not a replica? You would bomb the thing, for sure. You might not understand a single question. And yet as disorienting as that experience might feel, it would alter how you subsequently tuned into the course itself — and could sharply improve your overall performance.” This is the concept of pretesting.

Carey calls it one of the most exciting developments in the science of learning field. “Across a variety of experiments, psychologists have found that, in some circumstances, wrong answers on a pretest aren’t merely useless guesses. Rather, the attempts themselves change how we think about and store the information contained in the questions. On some kinds of tests, particularly multiple-choice, we benefit from answering incorrectly by, in effect, priming our brain for what’s coming later.” The failure on the pretest is an example of failing well. It sets students up for better learning during the course. A study by U.C.L.A. psychologist Elizabeth Ligon Bjork found that “…pretesting raised performance on final-exam questions by an average of 10 percent compared with a control group.”

Carey cites additional studies of pretesting with the insight that “testing might be the key to studying” and a way of “enriching and altering memory.” More traditional ways of studying do not seem to produce the same depth of learning that frequent testing, including the kind of self-examination that includes recitation, appears to yield. Other studies have shown that immediacy of feedback—getting the correct answers soon after the pretest—led to the greatest learning gains.

Why does pretesting work? There are several theories. First it gives students a preview of the material and helps them “prime the brain” to absorb what is most important. A pretest sets up a hierarchy and adjusts student thinking. Secondly, it exposes false impressions, things students think they know but don’t, by conveying multiple possible answers that they may not have considered as possibilities. Biological factors may come into play as well. Guessing at an answer on a pretest works differently from the memory functions at play in remembering and studying. Guessing embeds an unfamiliar concept into the brain that will be recognized when come across again, particularly if that happens within a short timeframe.

There are limitations. For example, a pretest for an intro course in a foreign language using unfamiliar characters (Russian, Chinese, Arabic) wouldn’t work because students have no “scaffolding of familiar language to work with.” In fact, “[t]he research thus far suggests that prefinals will be much more useful in humanities courses and social-science disciplines in which unfamiliar concepts are at least embedded in language we can parse.”

What can we take away from Carey’s article? Because pretests don’t need to be graded, this can be an easy innovation to implement in your courses. A short multiple choice quiz given before your lecture or class discussion asking questions pertaining to the key points you will cover could make a big difference in your students’ learning of the material. To be sure that they leave with the right information, review the quiz and the correct answers at the end of class.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image source: Pixabay.com

When Failure is a Good Thing

Posted on by
2

Smith College, and some other institutions of higher learning, are taking a new approach to failure and we should all be paying attention. At many colleges and universities, students strive for success at all costs; failure is not an option. There is a prevailing sense that peers are easily achieving great things with little effort, a B- is perceived as a bad grade, stress levels run high. At Smith, the time had come to call out both the ubiquity and the benefits of failure.

In On Campus, Failure is on the Syllabus (The New York Times, June 24, 2017), Jessica Bennett describes the initiative at Smith where students and faculty were videotaped describing their worst failure. These have been played at a campus hub during fall orientation and again during the final-exam period. In an atmosphere where everything seems pressured and competitive it was helpful to students to see that everyone struggles and that that is O.K.

The initiative, Failing Well, “…is a set of programs dedicated to the discussion of failure, risk taking and mistakes. …the mission is to increase student resilience by teaching, telling stories, and opening a campus conversation about failure.” The idea is to show students that their self-worth shouldn’t be tied solely to success, and that “failing well” can lead to unexpected bonuses. Given a set of skills and permission to “screw up” actually leads students to better learning and helps them to develop networks of resources.

Smith is not alone in this endeavor. Bennett lists other institutions engaged in similar “…remedial education that involves talking, a lot, about what it means to fail.” Today’s students have different needs in the real world and higher education should be preparing them appropriately.

As to the causes for the intense need to succeed, complicated forces and factors are at play, including child-rearing and cultural practices, “college admissions mania,” economic fears, social media, and a need to be busy, so called “competitive stress.”

Although an individual instructor may not be able to implement a campus-wide initiative, it is worth thinking about ways in which faculty can help students understand that failure is valuable to learning. In the next post, The Innovative Instructor will look at a specific practice of failure that enables better learning that anyone can use in teaching.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image source: Pixabay.com

Faculty Office Hours: The Instructor is In

Posted on by
Reply

Peanuts comic showing Lucy in the psychiatrist's booth, re-labelled to say "Course Help Free; The Instructor is In."What if you hold office hours and nobody comes? That’s the issue that Dr. Richard Freishtat, Director of the Center for Teaching and Learning at UC Berkeley addresses in Don’t Be Alone During Office Hours (Tomorrow’s Professor, January 10, 2017). He describes a student panel at Berkeley on facilitating student success. When the student panelists were asked by a faculty member about attending office hours, there was silence. Why was this opportunity not being pursued? One student responded, “Reason #1: ‘I think office hours is for students who are struggling with the material and need extra help. I wouldn’t want my professor to know I’m struggling, even if I was.’ Reason #2: ‘I was fascinated by the Professor, the discipline, and research in the field beyond what we were focusing on in class. I would have loved to drop into office hours and just talk with my professor about her own research, but figured there’s no way she’d want to take the time to do that with me.’” Faculty in the audience expressed just how much they would appreciate students coming to see them for the latter reason.

Freishtat goes on to suggest some ways to increase student attendance at office hours including making it an assignment, telling students how to use/what the purpose is of office hours, explaining to students how to start an office hours conversations (bring a question, a quote of interest, a story), scheduling varying days and times to accommodate different schedules, and promoting office hours throughout the semester/quarter.

For more on practical measures to take to improve the participation and outcome of your office hours, see the Stanford Teaching Commons website on Office Hours That Work and the University of Washington Center for Teaching and Learning on Face to Face Office Hours.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: CC Image from Flickr [https://www.flickr.com/photos/frederickhomesforsale/16107671226] modified by Macie Hall

How to get students to focus on learning, not grades

Posted on by
2

Here at Johns Hopkins we have a significant number of undergraduates who are pre-med students majoring in a range of mostly STEM disciplines. For many, their undergraduate studies are a milestone to be marked on the way to a degree in medicine. Getting into medical school is the goal, and grades are seen as critical to their success in meeting that objective. Of course, it’s not a problem just for pre-meds, we see it across the board. Instructors understand that grades are important, but do not necessarily equate to future success. It’s the learning that counts. So how do we get students to focus on the learning not the grades?

Students watching demonstration of frog dissection.An editorial from Inside Higher Ed Too Smart to Fail? (August 16, 2016), by Joseph Holtgreive, Assistant Dean and Director of the Office of Personal Development at Northwestern University’s McCormick School of Engineering, summarizes the challenges that face faculty and students. The “fear of failing to be perfect, ideally an effortless perfection, versus the joy of learning” creates situations where students opt for an easy grade as opposed to challenging themselves to learn. Holtgrieve has found this to be a problem particularly for students who did well academically in high school with little effort. Such students come to college focused on the “wrong outcome”—a high GPA—thinking “they’re keeping their eyes on the ball, they are actually just staring at the scoreboard.” While this affirms their measure of performance as long as their grades exceed their efforts, it can create a problem when their efforts exceed their grades.

Holtgreive points out that “[f]ocusing on the measurement of our performance reinforces what researcher Carol Dweck calls a fixed mind-set. If students believe that how they perform at one moment in time exposes the limits of their potential rather than serving merely as a snapshot of where they are in the process of growing their abilities, feelings of struggle and uncertainty become threatening rather than an opportunity to grow.” Focusing attention on grades may limit learning. On the other hand, when students can be convinced to “…set their intention to be genuinely curious and authentically excited by the challenge of finding connections between their current knowledge and new opportunities to understand, they experience the true joy of learning and all of the spoils that attend it.”

To find ways to help students “…reposition thinking about grades and learning,” Maryellen Weimer, PhD, offers some practical ideas in Five Ways to Get Students Thinking about Learning, Not Grades, from Faculty Focus, April 12, 2017.

  1. Position assignments as learning opportunities by discussing the “knowledge and skills” required rather than as something they are doing to “please the teacher.” Ask students to consider what they will learn in doing the work.
  2. Help students reflect on learning experiences throughout the course. Ask them to think about their professional ambitions and the skills and knowledge they will need. Have them make a list of those and use the list after every assignment or activity to write a short reflection on how the work they completed furthered their development.
  3. Create evolving assignments rather than one time tasks or activities. “One-time assignments don’t illustrate how learning is an evolving process and they don’t teach students how to do more work on something they have already done.” Instead, have students write a paper one step at a time (research a topic and create a bibliography, submit a thesis statement and an outline, write a first draft, revise, etc.), complete a multi-phase project, write a series of reflections and responses on a subject. Provide feedback but not grades for each phase.
  4. Encourage peer collaboration by structuring group learning and making sure that students are asking the right questions of each other.
  5. Change the conversation by talking about learning with students. Help students see how learning, not grades, will relate to their future professional goals.

Shifting the focus from grades to learning requires faculty to go against the tide of today’s prevailing academic culture. But making a few changes in how you think about teaching can go a long way to improving student perceptions of the importance of learning.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Learning from Student Evaluations

Posted on by
4

The end of the semester is looming and with it the specter of student course evaluations.There are three face emoticons--yellow neutral face on the left, green smilely face in the center, and red frown face on the right. A hand with the index finger extended is pointing in the direction of the viewer at the yellow smiley face. On the whole, instructors dread dealing with these. Many question the value of institutionally administered evaluations.  Some ignore them, others read them and weep. It’s hard to consider student evaluations and not feel personally affronted by negative comments. You’ve worked hard, done your best, and the students are ungrateful. But maybe there is a better approach. Rather than going on the defensive, how can faculty use these evaluations to improve their teaching, and ultimately, improve student feedback?

Maryellen Weimer, What Can We Learn from End-of-Course Evaluations? (Faculty Focus, March 8th, 2017) suggests developing an “improvement mindset.” “Rather than offering answers, [student evaluations] can be used to raise questions. ‘What am I doing that’s causing students to view my teaching this way?’ Such questions need to lead us to specific, concrete behaviors—things teachers are or aren’t doing.” Weimer suggests looking at the Teaching Practices Inventory developed by Carl Wieman, Professor of Physics at Stanford,  and Sarah Gilbert, Department of Physics and Graduate School of Education at Stanford, as “…a great place to start acquiring this very detailed, nuts and bolts understanding of one’s instructional practice.” Although the inventory was developed with the STEM disciplines in mind, it is easily adaptable for teaching in other fields. An article by Weiman and Gilbert, The Teaching Practices Inventory: A New Tool for Characterizing College and University Teaching in Mathematics and Science, [CBE Life Science Education. 2014 Fall; 13(3):552-69. doi: 10.1187/cbe.14-02-0023], provides additional background and context.

There are other ways to improve teaching and thus evaluations. An underused resource to improve one’s teaching is classroom observation.  Weimer mentions peer observations in her article, but teaching and learning centers such as ours, the Center for Educational Resources, often provide this service for instructors. She also notes the value of formative assessment—seeking timely feedback from students during the course “…when there’s still time to make changes and students feel they have a stake in the action.”

For some solid, common sense advice that’s easy to put into practice, see David D. Perlmutter’s How to Read a Student Evaluation (The Chronicle of Higher Education, October 30, 2011). Perlmutter recommends “scanning for red flags”—the comments/complaints that a number of students make that, by paying attention and correcting, “…can help you head off longer-term troubles.”

He also recommends thinking ahead. Prepare yourself for the evaluations by taking the questions asked into account when designing your courses, and using them as a checklist for your teaching. Perlmutter also suggests how to tease out the qualitative data in open comments, and advises balancing negative comments against positive quantitative ratings. His take is to “…pay attention to student evaluations, try to understand them, and, equally important, communicate that you do not dismiss them as irrelevant.”

Finally, the Vanderbilt Center for Teaching offers a full set of resources, articles, and advice on their website page: Student Evaluations of Teaching. From talking with students, to making sense of evaluation feedback, to research on student evaluations, it’s all here in one convenient location.

Now there is no excuse. Prepare yourself and vow to use student evaluations as a means to improve your teaching skills. Better evaluations will await you in the future.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Scaffolding Part 2: Build Your Students’ Notetaking Skills

Posted on by
1

A few weeks back The Innovative Instructor posted on teaching your students how to read a journal article, essentially how to provide a scaffold for your students to effectively read scholarly writings. Another place where faculty can provide a framework for students is in the area of notetaking. Students who have grown up using laptops in class may not understand either the value of or the means to taking effective notes. Recognizing that schedules are already jam-packed, I am not suggesting that you spend a lot of class time to cover this. But, taking a few minutes on the first day of class to let students know why using laptops to take notes may not be a good idea, and providing them with some resources for several notetaking methods, may go a long way to improving their learning outcomes.

The image shows a green ball point pen resting on a blank page of graphing paper in a blue covered, open spiral notebook.Why not just let your students use their laptops to take notes? As I wrote in a previous post [May 21, 2014 Summer Reading: Three Articles for Your Consideration] in The Pen is Mightier Than the Keyboard Advantages of Longhand Over Laptop Note Taking  Pam A. Mueller and Daniel M. Oppenheimer [Psychological Science, April 23, 2014, doi: 10.1177/ 0956797614524581], reported on the benefits students gain by taking lecture notes longhand rather than on a laptop. Although using laptops in class is common (and instructors complain about the distractions laptops present), this study “…suggests that even when laptops are used solely to take notes, they may still be impairing learning because their use results in shallower processing.” “In three studies, [the researchers] found that students who took notes on laptops performed worse on conceptual questions than students who took notes longhand.” The authors conclude “…that whereas taking more notes can be beneficial, laptop note takers’ tendency to transcribe lectures verbatim rather than processing information and reframing it in their own words is detrimental to learning.”

As for resources on notetaking, the James Madison University Special Education Program offers the JMU Learning Toolbox, developed with a U.S. Department of Education grant on Steppingstones in Technology Innovation for Students with Disabilities. It features “tools and resources to enable students with learning difficulties to become better learners.” A section on notetaking outlines several different strategies for taking notes, based on common problems students may experience.

Students who have trouble keeping up with the fast pace of a lecture or discussion may benefit from the I SWAM method. If a student needs a better strategy for organizing notetaking, Cornell Notes may be the answer. For taking notes from a recorded talk PP 123 may be helpful. SCROL is beneficial for notetaking while reading course materials. TASSEL is a method offered for those who are easily distracted.

Although TASSEL is designed to help students not to doodle when they get distracted, Sketch Notes encourages drawing as a means to enhance notetaking. The webpage referenced mentions visual learners. In fact, learning styles such as visual, verbal, or kinetic have been debunked (for an overview or the research study see: Learning Styles Debunked: There is No Evidence Supporting Auditory and Visual Learning, Psychologists Say; for the full article see: Learning Styles: Concepts and Evidence by Harold Pashler, Mark McDaniel, Doug Rohrer, and Robert Bjork, Psychological Science in the Public Interest, Volume 9 Number 3 December 2008). Nevertheless, Sketch Notes will likely have the most appeal for those with artistic/creative leanings who like to doodle and draw.

In introducing notetaking strategies to your students, you will want to be sensitive to those who may have learning disabilities that make laptop notetaking a necessity. However, sharing the research on laptop notetaking with students will give them an understanding of why hand notetaking strategies may improve their learning. Providing them with resources to investigate will give them choices based on their needs.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Scaffolding: Teach your students how to read a journal article

Posted on by
3

Recently I have had conversations with faculty and librarians about students and journal articles, specifically, that students don’t come to college knowing how to find or how to read a journal article. It may seem tedious to have to take time out of your already packed class schedule, but it will be valuable (for you and them) to provide some scaffolding and introduce them to these practices.

A pair of glasses and a highlighter are shown on top of an open text book.Here at Johns Hopkins, our Academic Liaison librarians will be happy to come to your class and discuss with students how to search for and locate appropriate materials for their research. Those at other institutions may have similar resources available. But you may also find it worthwhile to give some guidance on the reading aspect.

The Consortium for Political and Social Research (ICPSR) at the University of Michigan has a great three page guide, How to Read (and Understand) a Social Science Journal Article (pdf) that breaks down the parts of a journal article (e.g., title, abstract, introduction, literature review, etc.) and describes what each is and what it tells the reader. It’s aimed at social science students, but is broad enough to be useful for any discipline. After introducing the parts, the guide describes how to read an article by first determining your purpose, then devising a reading strategy.

The ICPSR guide references an article in Inside Higher Ed, It’s Not Harry Potter (Rob Weir, March 9, 2011), which starts off by asking the question “We tell them, but do we show them how?” referencing reading journal articles. Weir recommends starting by introducing students to the concept of audience and have them consider “…for whom and for what purpose a journal article is written.” He lists things students should consider when reading an article, but emphasizes, as is picked up in the ICPSR article, that having students identify their purpose for reading the article is a critical first step. Strategies such as determining the writer’s method, examining the footnotes to evaluate evidence, and skimming are described.

I usually avoid linking to commercial sites/resources, with the exception of apps and software references, however a blog post from ProfHacker (Chronicle of Higher Education) on another subject [Switching from Evernote to OneNote, part 1 by Amy Cavender, August 11, 2016] alerted me to an article by Michael Hyatt: How to Make Your Non-Fiction Reading More Productive, that I thought was worth citing.

Although Hyatt’s advice may be geared towards those in corporate environments, he offers a succinct guide to reading non-fiction books that will be useful for your students. He suggests starting with reviewing and recording the basic bibliographic information, then summarizing the author’s main premise and argument. “Think of this section like an elevator pitch. If you had to tell someone what the book is about in less than a minute, what would you say?” Then Hyatt advises readers to note the insight they gained before identifying their disagreements with the argument. What was missing from the book? What were the main takeaways? Are they quotes that are notable? Having this kind of template for analyzing a book (or article) will give students a concrete platform for tackling scholarly reading.

If you have tips for scaffolding reading or other assignments for students, please share them in the comments section.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Lunch and Learn: Constructing a Comprehensive Syllabus

Posted on by
Reply

Logo for Lunch and Learn program showing the words Lunch and Learn in orange with a fork above and a pen below the lettering. Faculty Conversations on Teaching at the bottom.On Thursday, February 16, the Center for Educational Resources (CER) hosted the third Lunch and Learn—Faculty Conversations on Teaching—for the 2016-1017 academic year. Katie Tifft, Lecturer Biology, and Jane Greco, Associate Teaching Professor Chemistry, shared best practices for creating a comprehensive syllabus.

Tifft and Greco presented as a team, reflecting their commitment to collaboration, and gave an impressive overview of the process they follow. Here are their slides for review. They started by sharing a quote by Gary Gutting “Why Do I Teach?” [New York Times 5/22/2013]: “College education is a proliferation of . . . possibilities: the beauty of mathematical discovery, the thrill of scientific understanding, the fascination of historical narrative, the mystery of theological speculation. We should judge teaching not by the amount of knowledge it passes on, but by the enduring excitement it generates. Knowledge, when it comes, is a later arrival, flaring up, when the time is right, from the sparks good teachers have implanted in their students’ souls.”

This represents an ideal, but in real world practice your experience may differ. One way to ensure that students leave your classroom with the knowledge you hope they will gain is to think about how to construct your course so that the desired learning outcomes align with your pedagogical approaches.

Tifft and Greco noted that standard course planning path is to choose a textbook/readings, produce a syllabus, write or revise lectures and prepare slides, and then create assessments (exams and assignments). This is a teacher-centric approach as it revolves around the content that you as the instructor plan to disseminate.

But what if you wanted to develop a course that was student-centric? Then you might take an approach known as backward design. With backward design you start the course planning process by formulating broad learning goals, then defining specific, measurable learning objectives. To clarify, learning goals express what you want students to get out of the course, while learning objectives detail the specific skills and level of understanding you want students to obtain. Next you design the assessments that will be used to evaluate the students’ mastery of the learning objectives. Finally, you develop the course content and activities and choose supporting texts and readings. This process will help you to create a syllabus that informs the students what you expect them to be able to do at the end of the course, as you will share both the broad learning goals for the course and the learning objectives for each course section on the syllabus.

Tifft and Greco reported that research has shown that the longer and more detailed a syllabus is, the more comfortable students will be, because they can see ahead to what will be coming in the class. They suggest keeping a positive tone, focusing on rewards rather than consequences. They both emphasize collaborative work in their courses, and on the syllabi, which fosters a student-centric environment.

What should the syllabus include? The course schedule in some detail, along with the A sign with an orange background reading "Keep calm and read the syllabus."detailed learning objectives for each unit. The course content will be a major part of your syllabus. Policies for absences and missed work should be included and should be transparent, fair, and set an easily achievable bar by accommodating situations that are bound to occur, such as illness, sports team events, etc. One way to do this is to drop the lowest score if you give multiple quizzes, exams, or homework assignments. Tifft and Greco noted that well thought out and clearly written policies are essential in a large enrollment course, and will help reduce the number of emails from students.

The syllabus should give information about assessments and assignments including due dates, descriptions, the link to learning objectives. Setting the test and assignment dates in stone, so to speak, on the syllabus will help your students know what to expect when. Having a variety of assignments is a good practice as it speaks to the diversity of student learning styles. This isn’t always practical in a large lecture class, but should be considered.

If you are using clickers (classroom polling devices) you will want to include policies for use, credit given for participation, credit for correctness, and contribution to grade. Tifft and Greco do not give credit for correctness as they see that getting something wrong contributes to the student’s learning process.

Grades are a major concern for students at Johns Hopkins; Tifft and Greco said that it is important to be as specific and transparent as possible when describing grading criteria and distribution on the syllabus. Doing so will reduce student complaints and misunderstandings. Some practices to consider in creating a grading scheme include the concept of revision/redemption—giving students a chance to drop a low score or revise a paper. They recommend against grading on a curve to reduce competition and facilitate student collaboration.

Don’t forget to list sources of help for students: office hours, names and contact information for teaching assistants, dates and times for recitations/review sessions, and information about the Learning Den tutoring program or PILOT (peer led team learning) program if applicable.

Finally, Tifft and Greco mentioned the required and recommended statements of policy, such as those on ethics, accommodations for students with disabilities, and copyright compliance. And in closing, they recommend adding a line in your syllabus that reads: “The information on this syllabus is subject to change at any time for any reason.”

Discussion by the faculty in attendance followed. One question asked was “How do you get students to read the syllabus? Should you go over the syllabus in class?” Greco stated that since she is teaching first semester freshman, she spends about 20 minutes on the first day of class going over key points, especially the learning goals and her teaching philosophy. Tifft, who teaches upperclassmen does give a brief summary of key points.

Faculty also shared experiences with grading schemes. Many like the idea of dropping the lowest scores on tests and/or assignments and the concept of redemption, especially when based on how the student has done on other parts of the course work. Some faculty give several section-based exams followed by a comprehensive final. Students who have aced the section exams, are not required to take the final.

The use of extra-credit and make-up work to improve grades was debated. It was agreed that it was important to be transparent in these cases, and to make sure that all students are offered the same opportunities. Greco recommended that faculty not allow students to wait until the end of semester to do make up or extra-credit work as it puts too much burden on you as a grader.

The session ended with Tifft and Greco sharing this cartoon from PhD (Piled Higher and Deeper) by Jorge Cham, something anyone who has ever created a syllabus will relate to.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Image generated by http://www.keepcalm-o-matic.co.uk/

Tips for Writing Effective Multiple Choice Questions

Posted on by
Reply

Writing test questions is a daunting task for many instructors. It can be challenging to come up with questions that correctly assess students on the comprehension of course objectives. Multiple choice questions are no exception; despite being very popular, instructors often struggle to create well-constructed questions.

Piece of notebook paper with Questions at the top, followed by numbers and ABCD for each of the six numbers. Answers are circled in red.Multiple choice questions have several advantages. They lend themselves to covering a broad range of content and assessing a wide variety of learning objectives. They are very useful when testing a student’s lower level knowledge of a topic, such as factual recall and definitions, but if written correctly, they can be used to assess at the higher levels of analysis, evaluation, and critical thinking skills. Multiple choice questions are scored efficiently (even automatically, if an electronic test is used), therefore, they are frequently the evaluation method preferred by instructors of large courses.

There are some disadvantages, including the fact that this type of question can be time-consuming to construct. Multiple choice questions are made up of two parts: the stem, which identifies the question, and the alternative responses which include the correct answer as well as incorrect alternatives, known as distractors. Coming up with plausible distractors for each question can be a difficult task. And, while some higher level thinking skills can be addressed, multiple choice questions cannot measure a student’s ability to organize and express ideas.  Another thing to consider is that student success when answering multiple choice questions can be influenced by factors unrelated to the subject matter, such as reading ability, deductive reasoning, and the use of context clues.

The following guidelines are offered to help streamline the process of creating multiple choice questions as well as minimize the disadvantages of using them.

General guidelines for writing stems:

  1. When possible, prepare the stem as a clearly written question rather than an incomplete statement.

Poor Example: Psychoanalysis is….

Better example: What is the definition of psychoanalysis? 

  1. Eliminate excessive or irrelevant information from the stem.

Poor example: Jane recently started a new job and can finally afford her own car, a Honda Civic, but is surprised at the high cost of gasoline. Gasoline prices are affected by:

Better example: Which of the following are factors that affect the consumer price of gasoline? 

  1. Include words/phrases in the stem that would otherwise be repeated in the alternatives.

Poor example: Which of the following statements are true?
1. Slowing population growth can prevent global warming
2. Halting deforestation can prevent global warming
3.  Increasing beef production on viable land can prevent global warming
4.  Improving energy efficiency can prevent global warming

Better example: Which of the following techniques can be used to prevent global warming?
1. Slowing population growth
2. Halting deforestation
3. Increasing beef production on viable land
4. Improving energy efficiency 

  1. Avoid using negatively stated stems. If you must use them, highlight the negative word so that it is obvious to students.

Poor example: Which of the following is not a mandatory qualification to be the president of the United States?

Better example: Which of the following is NOT a mandatory qualification to be the president of the United States?

General guidelines for writing alternative responses:

  1. Make sure there is only one correct answer.
  1. Create distractors that are plausible to avoid students guessing the correct answer.

Poor example:
Who was the third president of the United States?
1. George Washington
2. Bugs Bunny
3. Thomas Jefferson
4. Daffy Duck

Better example: Who was the third president of the United States?
1. George Washington
2. Benjamin Franklin
3. Thomas Jefferson
4. John Adams 

  1. Make sure alternative responses are grammatically parallel to each other.

Poor example: Which of the following is the best way to build muscle?
1. Sign up to run a marathon
2. Drinking lots of water
3. Exercise classes
4. Eat protein

Better example: Which of the following is the best way to build muscle?
1. Running on a treadmill
2. Drinking lots of water
3. Lifting weights
4. Eating lots of protein 

  1. When possible, list the alternative responses in a logical order (numerical, alphabetical, etc.)

Poor example: How many ounces are in a gallon?
1. 16
2. 148
3. 4
4. 128

Better example: How many ounces are in a gallon?
1. 4
2. 16
3. 128
4. 148

  1. Avoid using ‘All of the above’ or ‘None of the above’ to prevent students from using partial knowledge to arrive at the correct answer.
  2. Use at least four alternative responses to enhance the reliability of the test.

References:

Brame, C., (2013) Writing good multiple choice test questions. Retrieved December 14, 2016 from https://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/

Burton, S. J., Sudweeks, R. R., Merrill, P.F., and Wood, B. (1991). How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty. Provo, Utah: Brigham Young University Testing Services and The Department of Instructional Science.

“Multiple Choice Questions.” The University of Texas at Austin Faculty Innovation Center, 14 Dec. 2016, https://facultyinnovate.utexas.edu/teaching/check-learning/question-types/multiple-choice.

Amy Brusini, Blackboard Training Specialist
Center for Educational Resources

Image Source: Pixabay.com

The Dead Grandmother Syndrome and How to Treat It

Posted on by
Reply

Gravemarker with angel lying face down in grief, holding a wreath.If you are a woman of a certain age, with grandchildren attending college, please watch out for yourself over the next couple of weeks. Your mortality rate is about to increase dramatically.

This is a well-documented phenomenon, first described in a scholarly journal, the Annals of Improbable Research in the November/December 1999 Special Education Issue with The Dead Grandmother/Exam Syndrome by Mike Adams, Department of Biology at Eastern Connecticut State University. Although Adams’ article reports on the results of serious data collection, you should take the conclusions with a small amount of salt. The Annals of Improbable Research — also known as AIR — is a science humor magazine that publishes “…research that makes people laugh and then think.” But even as you may smile reading Adam’s research on the dead grandmother syndrome, it is likely because you recognize it from personal experience. As the end of the semester approaches and exams and papers are due, students who fall behind may resort to excuses for extensions or make-up dates. Your syllabus makes it clear that you don’t offer exceptions. Enter the death of a beloved grandmother.

While some faculty take a hard line on these excuses [Dear Student: Should Your Granny Die Before The Midterm … Chronicle Vitae, January 29, 2015], others have learned from personal experience that sometimes students’ grandmothers actually do die. Brian Thill writes of the conundrums faculty face in dealing with students excuses in The Time of Dead Grandmothers [Inside Higher Ed, March 14, 2006]. He writes: “As teachers, it seems to me we finally have a choice with respect to student excuses: to become cynics or fools. Cynics disbelieve all excuses. (It’s as if they all dissolve into dead grandmothers.) Fools believe them all. … How rightly to regard a student who is lying to you? No question about teaching is harder to answer because no question is less attractive.”

Karen Eifler, associate professor in the School of Education at the University of Portland, Oregon, offers a practical solution in Dealing with Student Deceptions: What to do with ‘Death in the Family’ Excuses [Faculty Focus, March 12, 2009]. Eifler, while well aware of the dead grandmother syndrome, also recognized that students do have deaths in the family and these events demand a sympathetic and courteous response, even while not wanting to encourage students to practice deception. This was her answer to the problem: “[W]hen a student informs me that a close relative has died, I immediately send a condolence card to the whole family, expressing my sympathy for their loss. If the student has been explicit (“It was my grandmother”), I am too. I can also match their vagueness. If the loss was authentic, the family is touched at the gesture, and I am truly glad to have extended that civility. However, if the story was a fabrication, the student finds he or she has some uncomfortable explaining to do to the family, which usually curbs that behavior.” It only took a couple of semesters before word got around about her practice, and it worked to her advantage to be seen as compassionate. Students quickly realized that it was better to speak with her honestly about their need for an extension as “…they figured anyone willing to call their bluff by sending condolences to the whole family would probably treat them with reasonable due process anyway.”

Ultimately it may be most useful to take a look at the underlying cause of the syndrome and address the stress that students are experiencing.

In his article, ‘Tis the Season of Dead Grandmothers [Chronicle Vitae, November 2, 2016], David Gooblar, lecturer in the Rhetoric department at the University of Iowa, questions  “…the assumption that strict discipline is the same thing as demanding a lot from our students.” He states that it is possible to care about your students without being a pushover. Strict policies with no exceptions may “…signal to students that adherence to the rules is more important than any other learning goal we have for them.” The end result may be detrimental to long-term learning. Gooblar prefers to create a “cohesive and supportive” learning community for his students. For example, he allows his students to come up with policies on device use in the classroom, having learned that students are more likely to adhere to policies when they have had a voice in the decision. He writes: “We should strive to create courses in which students want to do the work on time — because we’ve successfully made the case that doing the work on time will benefit them. We should also look to make students trust us enough that if tragedy does strike — sometimes family members do die, you know — they feel comfortable coming to us and explaining why they need some extra time.”

How do you handle student excuses and/or requests for extensions or makeup exams? Please share your policies and solutions in the comments section.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com