Learning from Student Evaluations

The end of the semester is looming and with it the specter of student course evaluations.There are three face emoticons--yellow neutral face on the left, green smilely face in the center, and red frown face on the right. A hand with the index finger extended is pointing in the direction of the viewer at the yellow smiley face. On the whole, instructors dread dealing with these. Many question the value of institutionally administered evaluations.  Some ignore them, others read them and weep. It’s hard to consider student evaluations and not feel personally affronted by negative comments. You’ve worked hard, done your best, and the students are ungrateful. But maybe there is a better approach. Rather than going on the defensive, how can faculty use these evaluations to improve their teaching, and ultimately, improve student feedback?

Maryellen Weimer, What Can We Learn from End-of-Course Evaluations? (Faculty Focus, March 8th, 2017) suggests developing an “improvement mindset.” “Rather than offering answers, [student evaluations] can be used to raise questions. ‘What am I doing that’s causing students to view my teaching this way?’ Such questions need to lead us to specific, concrete behaviors—things teachers are or aren’t doing.” Weimer suggests looking at the Teaching Practices Inventory developed by Carl Wieman, Professor of Physics at Stanford,  and Sarah Gilbert, Department of Physics and Graduate School of Education at Stanford, as “…a great place to start acquiring this very detailed, nuts and bolts understanding of one’s instructional practice.” Although the inventory was developed with the STEM disciplines in mind, it is easily adaptable for teaching in other fields. An article by Weiman and Gilbert, The Teaching Practices Inventory: A New Tool for Characterizing College and University Teaching in Mathematics and Science, [CBE Life Science Education. 2014 Fall; 13(3):552-69. doi: 10.1187/cbe.14-02-0023], provides additional background and context.

There are other ways to improve teaching and thus evaluations. An underused resource to improve one’s teaching is classroom observation.  Weimer mentions peer observations in her article, but teaching and learning centers such as ours, the Center for Educational Resources, often provide this service for instructors. She also notes the value of formative assessment—seeking timely feedback from students during the course “…when there’s still time to make changes and students feel they have a stake in the action.”

For some solid, common sense advice that’s easy to put into practice, see David D. Perlmutter’s How to Read a Student Evaluation (The Chronicle of Higher Education, October 30, 2011). Perlmutter recommends “scanning for red flags”—the comments/complaints that a number of students make that, by paying attention and correcting, “…can help you head off longer-term troubles.”

He also recommends thinking ahead. Prepare yourself for the evaluations by taking the questions asked into account when designing your courses, and using them as a checklist for your teaching. Perlmutter also suggests how to tease out the qualitative data in open comments, and advises balancing negative comments against positive quantitative ratings. His take is to “…pay attention to student evaluations, try to understand them, and, equally important, communicate that you do not dismiss them as irrelevant.”

Finally, the Vanderbilt Center for Teaching offers a full set of resources, articles, and advice on their website page: Student Evaluations of Teaching. From talking with students, to making sense of evaluation feedback, to research on student evaluations, it’s all here in one convenient location.

Now there is no excuse. Prepare yourself and vow to use student evaluations as a means to improve your teaching skills. Better evaluations will await you in the future.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Scaffolding Part 2: Build Your Students’ Notetaking Skills

A few weeks back The Innovative Instructor posted on teaching your students how to read a journal article, essentially how to provide a scaffold for your students to effectively read scholarly writings. Another place where faculty can provide a framework for students is in the area of notetaking. Students who have grown up using laptops in class may not understand either the value of or the means to taking effective notes. Recognizing that schedules are already jam-packed, I am not suggesting that you spend a lot of class time to cover this. But, taking a few minutes on the first day of class to let students know why using laptops to take notes may not be a good idea, and providing them with some resources for several notetaking methods, may go a long way to improving their learning outcomes.

The image shows a green ball point pen resting on a blank page of graphing paper in a blue covered, open spiral notebook.Why not just let your students use their laptops to take notes? As I wrote in a previous post [May 21, 2014 Summer Reading: Three Articles for Your Consideration] in The Pen is Mightier Than the Keyboard Advantages of Longhand Over Laptop Note Taking  Pam A. Mueller and Daniel M. Oppenheimer [Psychological Science, April 23, 2014, doi: 10.1177/ 0956797614524581], reported on the benefits students gain by taking lecture notes longhand rather than on a laptop. Although using laptops in class is common (and instructors complain about the distractions laptops present), this study “…suggests that even when laptops are used solely to take notes, they may still be impairing learning because their use results in shallower processing.” “In three studies, [the researchers] found that students who took notes on laptops performed worse on conceptual questions than students who took notes longhand.” The authors conclude “…that whereas taking more notes can be beneficial, laptop note takers’ tendency to transcribe lectures verbatim rather than processing information and reframing it in their own words is detrimental to learning.”

As for resources on notetaking, the James Madison University Special Education Program offers the JMU Learning Toolbox, developed with a U.S. Department of Education grant on Steppingstones in Technology Innovation for Students with Disabilities. It features “tools and resources to enable students with learning difficulties to become better learners.” A section on notetaking outlines several different strategies for taking notes, based on common problems students may experience.

Students who have trouble keeping up with the fast pace of a lecture or discussion may benefit from the I SWAM method. If a student needs a better strategy for organizing notetaking, Cornell Notes may be the answer. For taking notes from a recorded talk PP 123 may be helpful. SCROL is beneficial for notetaking while reading course materials. TASSEL is a method offered for those who are easily distracted.

Although TASSEL is designed to help students not to doodle when they get distracted, Sketch Notes encourages drawing as a means to enhance notetaking. The webpage referenced mentions visual learners. In fact, learning styles such as visual, verbal, or kinetic have been debunked (for an overview or the research study see: Learning Styles Debunked: There is No Evidence Supporting Auditory and Visual Learning, Psychologists Say; for the full article see: Learning Styles: Concepts and Evidence by Harold Pashler, Mark McDaniel, Doug Rohrer, and Robert Bjork, Psychological Science in the Public Interest, Volume 9 Number 3 December 2008). Nevertheless, Sketch Notes will likely have the most appeal for those with artistic/creative leanings who like to doodle and draw.

In introducing notetaking strategies to your students, you will want to be sensitive to those who may have learning disabilities that make laptop notetaking a necessity. However, sharing the research on laptop notetaking with students will give them an understanding of why hand notetaking strategies may improve their learning. Providing them with resources to investigate will give them choices based on their needs.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Scaffolding: Teach your students how to read a journal article

Recently I have had conversations with faculty and librarians about students and journal articles, specifically, that students don’t come to college knowing how to find or how to read a journal article. It may seem tedious to have to take time out of your already packed class schedule, but it will be valuable (for you and them) to provide some scaffolding and introduce them to these practices.

A pair of glasses and a highlighter are shown on top of an open text book.Here at Johns Hopkins, our Academic Liaison librarians will be happy to come to your class and discuss with students how to search for and locate appropriate materials for their research. Those at other institutions may have similar resources available. But you may also find it worthwhile to give some guidance on the reading aspect.

The Consortium for Political and Social Research (ICPSR) at the University of Michigan has a great three page guide, How to Read (and Understand) a Social Science Journal Article (pdf) that breaks down the parts of a journal article (e.g., title, abstract, introduction, literature review, etc.) and describes what each is and what it tells the reader. It’s aimed at social science students, but is broad enough to be useful for any discipline. After introducing the parts, the guide describes how to read an article by first determining your purpose, then devising a reading strategy.

The ICPSR guide references an article in Inside Higher Ed, It’s Not Harry Potter (Rob Weir, March 9, 2011), which starts off by asking the question “We tell them, but do we show them how?” referencing reading journal articles. Weir recommends starting by introducing students to the concept of audience and have them consider “…for whom and for what purpose a journal article is written.” He lists things students should consider when reading an article, but emphasizes, as is picked up in the ICPSR article, that having students identify their purpose for reading the article is a critical first step. Strategies such as determining the writer’s method, examining the footnotes to evaluate evidence, and skimming are described.

I usually avoid linking to commercial sites/resources, with the exception of apps and software references, however a blog post from ProfHacker (Chronicle of Higher Education) on another subject [Switching from Evernote to OneNote, part 1 by Amy Cavender, August 11, 2016] alerted me to an article by Michael Hyatt: How to Make Your Non-Fiction Reading More Productive, that I thought was worth citing.

Although Hyatt’s advice may be geared towards those in corporate environments, he offers a succinct guide to reading non-fiction books that will be useful for your students. He suggests starting with reviewing and recording the basic bibliographic information, then summarizing the author’s main premise and argument. “Think of this section like an elevator pitch. If you had to tell someone what the book is about in less than a minute, what would you say?” Then Hyatt advises readers to note the insight they gained before identifying their disagreements with the argument. What was missing from the book? What were the main takeaways? Are they quotes that are notable? Having this kind of template for analyzing a book (or article) will give students a concrete platform for tackling scholarly reading.

If you have tips for scaffolding reading or other assignments for students, please share them in the comments section.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Lunch and Learn: Constructing a Comprehensive Syllabus

Logo for Lunch and Learn program showing the words Lunch and Learn in orange with a fork above and a pen below the lettering. Faculty Conversations on Teaching at the bottom.On Thursday, February 16, the Center for Educational Resources (CER) hosted the third Lunch and Learn—Faculty Conversations on Teaching—for the 2016-1017 academic year. Katie Tifft, Lecturer Biology, and Jane Greco, Associate Teaching Professor Chemistry, shared best practices for creating a comprehensive syllabus.

Tifft and Greco presented as a team, reflecting their commitment to collaboration, and gave an impressive overview of the process they follow. Here are their slides for review. They started by sharing a quote by Gary Gutting “Why Do I Teach?” [New York Times 5/22/2013]: “College education is a proliferation of . . . possibilities: the beauty of mathematical discovery, the thrill of scientific understanding, the fascination of historical narrative, the mystery of theological speculation. We should judge teaching not by the amount of knowledge it passes on, but by the enduring excitement it generates. Knowledge, when it comes, is a later arrival, flaring up, when the time is right, from the sparks good teachers have implanted in their students’ souls.”

This represents an ideal, but in real world practice your experience may differ. One way to ensure that students leave your classroom with the knowledge you hope they will gain is to think about how to construct your course so that the desired learning outcomes align with your pedagogical approaches.

Tifft and Greco noted that standard course planning path is to choose a textbook/readings, produce a syllabus, write or revise lectures and prepare slides, and then create assessments (exams and assignments). This is a teacher-centric approach as it revolves around the content that you as the instructor plan to disseminate.

But what if you wanted to develop a course that was student-centric? Then you might take an approach known as backward design. With backward design you start the course planning process by formulating broad learning goals, then defining specific, measurable learning objectives. To clarify, learning goals express what you want students to get out of the course, while learning objectives detail the specific skills and level of understanding you want students to obtain. Next you design the assessments that will be used to evaluate the students’ mastery of the learning objectives. Finally, you develop the course content and activities and choose supporting texts and readings. This process will help you to create a syllabus that informs the students what you expect them to be able to do at the end of the course, as you will share both the broad learning goals for the course and the learning objectives for each course section on the syllabus.

Tifft and Greco reported that research has shown that the longer and more detailed a syllabus is, the more comfortable students will be, because they can see ahead to what will be coming in the class. They suggest keeping a positive tone, focusing on rewards rather than consequences. They both emphasize collaborative work in their courses, and on the syllabi, which fosters a student-centric environment.

What should the syllabus include? The course schedule in some detail, along with the A sign with an orange background reading "Keep calm and read the syllabus."detailed learning objectives for each unit. The course content will be a major part of your syllabus. Policies for absences and missed work should be included and should be transparent, fair, and set an easily achievable bar by accommodating situations that are bound to occur, such as illness, sports team events, etc. One way to do this is to drop the lowest score if you give multiple quizzes, exams, or homework assignments. Tifft and Greco noted that well thought out and clearly written policies are essential in a large enrollment course, and will help reduce the number of emails from students.

The syllabus should give information about assessments and assignments including due dates, descriptions, the link to learning objectives. Setting the test and assignment dates in stone, so to speak, on the syllabus will help your students know what to expect when. Having a variety of assignments is a good practice as it speaks to the diversity of student learning styles. This isn’t always practical in a large lecture class, but should be considered.

If you are using clickers (classroom polling devices) you will want to include policies for use, credit given for participation, credit for correctness, and contribution to grade. Tifft and Greco do not give credit for correctness as they see that getting something wrong contributes to the student’s learning process.

Grades are a major concern for students at Johns Hopkins; Tifft and Greco said that it is important to be as specific and transparent as possible when describing grading criteria and distribution on the syllabus. Doing so will reduce student complaints and misunderstandings. Some practices to consider in creating a grading scheme include the concept of revision/redemption—giving students a chance to drop a low score or revise a paper. They recommend against grading on a curve to reduce competition and facilitate student collaboration.

Don’t forget to list sources of help for students: office hours, names and contact information for teaching assistants, dates and times for recitations/review sessions, and information about the Learning Den tutoring program or PILOT (peer led team learning) program if applicable.

Finally, Tifft and Greco mentioned the required and recommended statements of policy, such as those on ethics, accommodations for students with disabilities, and copyright compliance. And in closing, they recommend adding a line in your syllabus that reads: “The information on this syllabus is subject to change at any time for any reason.”

Discussion by the faculty in attendance followed. One question asked was “How do you get students to read the syllabus? Should you go over the syllabus in class?” Greco stated that since she is teaching first semester freshman, she spends about 20 minutes on the first day of class going over key points, especially the learning goals and her teaching philosophy. Tifft, who teaches upperclassmen does give a brief summary of key points.

Faculty also shared experiences with grading schemes. Many like the idea of dropping the lowest scores on tests and/or assignments and the concept of redemption, especially when based on how the student has done on other parts of the course work. Some faculty give several section-based exams followed by a comprehensive final. Students who have aced the section exams, are not required to take the final.

The use of extra-credit and make-up work to improve grades was debated. It was agreed that it was important to be transparent in these cases, and to make sure that all students are offered the same opportunities. Greco recommended that faculty not allow students to wait until the end of semester to do make up or extra-credit work as it puts too much burden on you as a grader.

The session ended with Tifft and Greco sharing this cartoon from PhD (Piled Higher and Deeper) by Jorge Cham, something anyone who has ever created a syllabus will relate to.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Image generated by http://www.keepcalm-o-matic.co.uk/

Tips for Writing Effective Multiple Choice Questions

Writing test questions is a daunting task for many instructors. It can be challenging to come up with questions that correctly assess students on the comprehension of course objectives. Multiple choice questions are no exception; despite being very popular, instructors often struggle to create well-constructed questions.

Piece of notebook paper with Questions at the top, followed by numbers and ABCD for each of the six numbers. Answers are circled in red.Multiple choice questions have several advantages. They lend themselves to covering a broad range of content and assessing a wide variety of learning objectives. They are very useful when testing a student’s lower level knowledge of a topic, such as factual recall and definitions, but if written correctly, they can be used to assess at the higher levels of analysis, evaluation, and critical thinking skills. Multiple choice questions are scored efficiently (even automatically, if an electronic test is used), therefore, they are frequently the evaluation method preferred by instructors of large courses.

There are some disadvantages, including the fact that this type of question can be time-consuming to construct. Multiple choice questions are made up of two parts: the stem, which identifies the question, and the alternative responses which include the correct answer as well as incorrect alternatives, known as distractors. Coming up with plausible distractors for each question can be a difficult task. And, while some higher level thinking skills can be addressed, multiple choice questions cannot measure a student’s ability to organize and express ideas.  Another thing to consider is that student success when answering multiple choice questions can be influenced by factors unrelated to the subject matter, such as reading ability, deductive reasoning, and the use of context clues.

The following guidelines are offered to help streamline the process of creating multiple choice questions as well as minimize the disadvantages of using them.

General guidelines for writing stems:

  1. When possible, prepare the stem as a clearly written question rather than an incomplete statement.

Poor Example: Psychoanalysis is….

Better example: What is the definition of psychoanalysis? 

  1. Eliminate excessive or irrelevant information from the stem.

Poor example: Jane recently started a new job and can finally afford her own car, a Honda Civic, but is surprised at the high cost of gasoline. Gasoline prices are affected by:

Better example: Which of the following are factors that affect the consumer price of gasoline? 

  1. Include words/phrases in the stem that would otherwise be repeated in the alternatives.

Poor example: Which of the following statements are true?
1. Slowing population growth can prevent global warming
2. Halting deforestation can prevent global warming
3.  Increasing beef production on viable land can prevent global warming
4.  Improving energy efficiency can prevent global warming

Better example: Which of the following techniques can be used to prevent global warming?
1. Slowing population growth
2. Halting deforestation
3. Increasing beef production on viable land
4. Improving energy efficiency 

  1. Avoid using negatively stated stems. If you must use them, highlight the negative word so that it is obvious to students.

Poor example: Which of the following is not a mandatory qualification to be the president of the United States?

Better example: Which of the following is NOT a mandatory qualification to be the president of the United States?

General guidelines for writing alternative responses:

  1. Make sure there is only one correct answer.
  1. Create distractors that are plausible to avoid students guessing the correct answer.

Poor example:
Who was the third president of the United States?
1. George Washington
2. Bugs Bunny
3. Thomas Jefferson
4. Daffy Duck

Better example: Who was the third president of the United States?
1. George Washington
2. Benjamin Franklin
3. Thomas Jefferson
4. John Adams 

  1. Make sure alternative responses are grammatically parallel to each other.

Poor example: Which of the following is the best way to build muscle?
1. Sign up to run a marathon
2. Drinking lots of water
3. Exercise classes
4. Eat protein

Better example: Which of the following is the best way to build muscle?
1. Running on a treadmill
2. Drinking lots of water
3. Lifting weights
Eating lots of protein 

  1. When possible, list the alternative responses in a logical order (numerical, alphabetical, etc.)

Poor example: How many ounces are in a gallon?
1. 16
2. 148
3. 4
4. 128

Better example: How many ounces are in a gallon?
1. 4
2. 16
3. 128
4. 148

  1. Avoid using ‘All of the above’ or ‘None of the above’ to prevent students from using partial knowledge to arrive at the correct answer.
  2. Use at least four alternative responses to enhance the reliability of the test.


Brame, C., (2013) Writing good multiple choice test questions. Retrieved December 14, 2016 from https://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/

Burton, S. J., Sudweeks, R. R., Merrill, P.F., and Wood, B. (1991). How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty. Provo, Utah: Brigham Young University Testing Services and The Department of Instructional Science.

“Multiple Choice Questions.” The University of Texas at Austin Faculty Innovation Center, 14 Dec. 2016, https://facultyinnovate.utexas.edu/teaching/check-learning/question-types/multiple-choice.

Amy Brusini, Blackboard Training Specialist
Center for Educational Resources

Image Source: Pixabay.com

The Dead Grandmother Syndrome and How to Treat It

Gravemarker with angel lying face down in grief, holding a wreath.If you are a woman of a certain age, with grandchildren attending college, please watch out for yourself over the next couple of weeks. Your mortality rate is about to increase dramatically.

This is a well-documented phenomenon, first described in a scholarly journal, the Annals of Improbable Research in the November/December 1999 Special Education Issue with The Dead Grandmother/Exam Syndrome by Mike Adams, Department of Biology at Eastern Connecticut State University. Although Adams’ article reports on the results of serious data collection, you should take the conclusions with a small amount of salt. The Annals of Improbable Research — also known as AIR — is a science humor magazine that publishes “…research that makes people laugh and then think.” But even as you may smile reading Adam’s research on the dead grandmother syndrome, it is likely because you recognize it from personal experience. As the end of the semester approaches and exams and papers are due, students who fall behind may resort to excuses for extensions or make-up dates. Your syllabus makes it clear that you don’t offer exceptions. Enter the death of a beloved grandmother.

While some faculty take a hard line on these excuses [Dear Student: Should Your Granny Die Before The Midterm … Chronicle Vitae, January 29, 2015], others have learned from personal experience that sometimes students’ grandmothers actually do die. Brian Thill writes of the conundrums faculty face in dealing with students excuses in The Time of Dead Grandmothers [Inside Higher Ed, March 14, 2006]. He writes: “As teachers, it seems to me we finally have a choice with respect to student excuses: to become cynics or fools. Cynics disbelieve all excuses. (It’s as if they all dissolve into dead grandmothers.) Fools believe them all. … How rightly to regard a student who is lying to you? No question about teaching is harder to answer because no question is less attractive.”

Karen Eifler, associate professor in the School of Education at the University of Portland, Oregon, offers a practical solution in Dealing with Student Deceptions: What to do with ‘Death in the Family’ Excuses [Faculty Focus, March 12, 2009]. Eifler, while well aware of the dead grandmother syndrome, also recognized that students do have deaths in the family and these events demand a sympathetic and courteous response, even while not wanting to encourage students to practice deception. This was her answer to the problem: “[W]hen a student informs me that a close relative has died, I immediately send a condolence card to the whole family, expressing my sympathy for their loss. If the student has been explicit (“It was my grandmother”), I am too. I can also match their vagueness. If the loss was authentic, the family is touched at the gesture, and I am truly glad to have extended that civility. However, if the story was a fabrication, the student finds he or she has some uncomfortable explaining to do to the family, which usually curbs that behavior.” It only took a couple of semesters before word got around about her practice, and it worked to her advantage to be seen as compassionate. Students quickly realized that it was better to speak with her honestly about their need for an extension as “…they figured anyone willing to call their bluff by sending condolences to the whole family would probably treat them with reasonable due process anyway.”

Ultimately it may be most useful to take a look at the underlying cause of the syndrome and address the stress that students are experiencing.

In his article, ‘Tis the Season of Dead Grandmothers [Chronicle Vitae, November 2, 2016], David Gooblar, lecturer in the Rhetoric department at the University of Iowa, questions  “…the assumption that strict discipline is the same thing as demanding a lot from our students.” He states that it is possible to care about your students without being a pushover. Strict policies with no exceptions may “…signal to students that adherence to the rules is more important than any other learning goal we have for them.” The end result may be detrimental to long-term learning. Gooblar prefers to create a “cohesive and supportive” learning community for his students. For example, he allows his students to come up with policies on device use in the classroom, having learned that students are more likely to adhere to policies when they have had a voice in the decision. He writes: “We should strive to create courses in which students want to do the work on time — because we’ve successfully made the case that doing the work on time will benefit them. We should also look to make students trust us enough that if tragedy does strike — sometimes family members do die, you know — they feel comfortable coming to us and explaining why they need some extra time.”

How do you handle student excuses and/or requests for extensions or makeup exams? Please share your policies and solutions in the comments section.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Quick Tips: Teaching in Challenging Times and Facilitating Difficult Discussions

In the days following the election faculty and students across the country were faced with Image of a stylized human figure peering into the opening of a large circular maze.teaching and learning in a climate that made both activities difficult. The issues that divided our nation could not be ignored in the classroom. The Center for Teaching at Vanderbilt University published a thoughtful guide for faculty: Teaching in Response to the Election, by Joe Bandy, CFT Assistant Director. The suggestions are practical, reference additional resources, and are useful not just today, but in thinking about supporting students in general. Three other CFT guides are referenced: Teaching in Times of Crisis for when “communities are united in grief or trauma,” Difficult Dialogues will be useful whenever topics of discussion in the classroom touch on “hot button” issues, and the guide for Increasing Inclusivity in the Classroom is relevant at all times.

We welcome your suggestions in the comments for facilitating difficult discussions and teaching in challenging times.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image source: Pixabay.com

Lunch and Learn: Flipped courses: What is the purpose? What are the strategies?

Logo for Lunch and Learn program showing the words Lunch and Learn in orange with a fork above and a pen below the lettering. Faculty Conversations on Teaching at the bottom.On Thursday, October 20, the Center for Educational Resources (CER) hosted the first Lunch and Learn—Faculty Conversations on Teaching for the 2016-1017 academic year. A panel of faculty including Avanti Athreya, Assistant Research Professor Applied Mathematics & Statistics; Michael Falk, Professor Materials Science & Engineering; Bob Leheny, Professor Physics & Astronomy; and Soojin Park, Assistant Professor Cognitive Science; spoke briefly on their experiences and engaged in a lively discussion with attendees on Flipped courses: What is the purpose?  What are the strategies?

Avanti Athreya described flipping a large lecture course in Fall 2015 with her colleague, Dan Naiman, Professor, Applied Math & Statistics. The 4 credit course, Statistical Analysis I had previously met four times a week for 50 minutes – three lectures by faculty and one small-group meeting led by a TA.  Starting in Fall 2015, students watched several short videos (5-15 minutes each) before the week started.  The videos were created by Athreya and Naiman using Camtasia. Students then met once for a 75-minute lecture with the instructor and twice in small-groups with a TA.  During these sessions students, working in teams of three, solved problems with a TA available for help as needed.  Clicker quizzes were given at the beginning of each lecture to motivate students to watch the videos. Athreya noted that clear learning objectives were listed at the beginning of each video. Challenges included initial resistance from the students (she stated that there had been less of that this semester, the second iteration of the flipped course), and that students often need alternative explanation for concepts. Typically, the videos cover an idea in one way. In a lecture, the instructor noting confusion may offer another explanation for clarification.

Soojin Park co-teaches Cognitive Neuroscience: Exploring the Living Brain with Brenda Rapp, Professor, Cognitive Science. This 3 credit course has an enrollment on average of 250 students. Park and Rapp flipped their course in Spring 2016, with a goal of putting more emphasis on student exploration. They videotaped scripted lectures (these videos were shorter and more focused than the lectures in the traditional course) and posted them on Blackboard. Students took quizzes on the video content. Students met twice a week in sections of about 25. One section was structured as a review section, the other as an active learning section. The challenge was to create the active learning activities. They decided to emphasize practical skills, such as exercises to learn spatial areas of the brain using 3-D software. These activities were all group based. There were worksheets for each session. For the final project, students developed a mock NIH proposal. Park and Rapp found a 5% learning improvement on the final exam (the questions were reused from the previous year to allow comparison) as well as higher course evaluations.

Bob Leheny reported that he is in the fourth year of teaching an active-learning version of Introduction to Physical Sciences, which incorporates a flipped classroom model. The course serves 700 students each semester. Before class, students watch videos that were developed at the University of Illinois. Leheny noted that there is a great deal of video content already developed for teaching introductory physics, so the faculty developing the course here were spared having to create their own. Faculty are able to track how much time students spend watching the videos. The course was developed with funding from a JHU Gateway Sciences Initiative grant, which included the design and implementation of an active learning classroom that seats 80 students. In the classroom, students review the video content, then work collaboratively in groups of three on exercises and experiments that explore the topic for the day. The course is supported by three graduate student TAs and four undergraduate TAs. Leheny said that one of the challenges was time management in the active learning setting. He compared the instructor and TAs to “waiters working the tables” where students were doing the activities and exercises. There is a constant monitoring of where students are and what they need.

Michael Falk was an early adopter of flipping the course. He now flips two courses: his undergraduate Computation and Programming for Materials Scientists and Engineers, with an enrollment of 35, and a graduate course, Thermodynamics of Materials. For the undergraduate class he created his own videos using Screen Flow. Students take quizzes on the video content before class. In class students work through exercises collaboratively. Falk uses Class Spot to facilitate this work. Class Spot allows screen sharing; students can see how their classmates worked out solutions to problems. For his graduate course in thermodynamics, Falk made short, Khan Academy-style videos using Quick Time. The students watch the videos before class and use class time for problem solving. He also made use of an application called Perusall for annotation exercises. His found in general that his students like it better if there is a short recap of the video material at the beginning of class. Falk feels that the biggest challenge with flipping is finding meaningful activities for class time.

Some key points covered during discussion included:

  1. Making sure that students aren’t assigned too much to do outside of class–videos should replace some of the reading or other homework assignments.
  2. It may be necessary to incentivize students to watch the videos. This can be in the form of quizzes.
  3. If group or collaborative work is done in class, follow best practices for creating groups. Groups of three are ideal. It is best not to have two males and one female in a group as has been shown in research on gender construction of teams. Group work presents valuable experiences for students. For those going into STEM fields, collaboration will be the norm, thus is a good skill to acquire. Group work can help minimize the negative aspects of competition in a classroom.
  4. Base in-class activities on the student learning goals for the course.
  5. Keep videos short, even, or especially when using a lecture-style delivery of the content. Scripting of lecture delivery was advised, as well as adopting a modular concept. Each lecture video should focus on one idea.
  6. Faculty who had flipped their courses noted that preparation for the initial offering of the course took a tremendous investment of time, but that the results had been worth the effort involved.
  7. Several faculty from the humanities discussed whether a flipped model could be used in their class situations, and specifically whether video delivery offered any advantage over reading a text. Certainly offering a variety of learning modalities can be valuable for students coming to a course with different backgrounds and understanding. A humanities course might not benefit from being flipped in total, but having students work together in class to develop specific skills, such as close reading, could prove valuable.

In all, the session was interesting and informative. If you are an instructor on the Homewood campus, staff in the Center for Educational Resources will be happy to talk with you about flipping a course.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image source: Lunch and Learn logo by Reid Sczerba, Center for Educational Resources.

PowerPoint in the Classroom

Do you use PowerPoint (or Keynote, Prezi or other presentation software) as part of your teaching? If yes, why? This is not meant to be a question that puts you on the defensive, rather to ask you to reflect on how the use of a presentation application enhances your teaching and fits in with other strategies to meet your learning objectives for the class.

Cartoon-like drawing of a presenter showing a slide to a sleeping/snoring audience.It’s been almost three years since The Innovative Instructor wrote on using PowerPoint in the classroom. See Polishing your PowerPoints, a post that covered some tips for creating more effective slides, citing a book by Nancy Duarte called Slide:ology [Nancy Duarte, Slide:ology,  O’Reilly Media, Inc., 2008].

A key point from that post to reiterate: “Duarte reports on research showing that listening and reading are conflicting cognitive processes, meaning that your audience can either read your slides or listen to you; they cannot do both at the same time. However, our brains can handle simultaneous listening to a speaker and seeing relevant visual material.”

It’s important to keep this in mind, particularly if your slides are text heavy. Your students will be scrambling to copy the text verbatim without actually processing what is being said. On the other hand, if your slides are used as prompts (presenting questions or key points with minimal text) or if you don’t use slides at all, students will have to listen to what you are saying, and summarize those concepts in their notes. This process will enhance their understanding of the material.

An article in Focus on Teaching from August 1, 2012 by Maryellen Weimer, PhD asks us to consider Does PowerPoint Help or Hinder Learning? Weimer references a survey of students on the use of PowerPoint by their instructors. A majority of students reported that all or most of their instructors used PowerPoint. Weimer’s expresses the concern that “Eighty-two percent [of students surveyed] said they “always,” “almost always,” or “usually” copy the information on the slides.” She asks, “Does copying down content word-for-word develop the skills needed to organize material on your own? Does it expedite understanding the relationships between ideas? Does it set students up to master the material or to simply memorize it?” Further, she notes that PowerPoint slides that serve as an outline or use bulleted lists may “oversimplify” complex content, encourage passivity, and limit critical thinking.

Four journal articles from Cell Biology Education on PowerPoint in the Classroom (2004 Fall) present different points of view (POV) on the use of PowerPoint. Although written over a decade ago, most of the concepts are still relevant. Be aware that some of the links are no longer working. From the introduction to the series:

Four POVs are presented: 1) David Keefe and James Willett provide their case why PowerPoint is an ideal teaching software. Keefe is an educational researcher at the Center for Technology in Learning at SRI International. Willett is a professor at George Mason University in the Departments of Microbial and Molecular Bioscience; as well as Bioinformatics and Computational Biology. 2) Kim McDonald highlights the causes of PowerPointlessness, a term which indicates the frequent use of PowerPoint as a crutch rather than a tool. She is a Bioscience Educator at the Shodor Education Foundation, Inc. 3) Diana Voss asks readers if PowerPoint is really necessary to present the material effectively or not. Voss is a Instructional Computing Support Specialist at SUNY Stony Brook. 4) Cynthia Lanius takes a light-hearted approach to ask whether PowerPoint is a technological improvement or just a change of pace for teacher and student presentations. Lanius is a Technology Integration Specialist in the Sinton (Texas) Independent School District.

These are short, op-ed style, pieces that will further stimulate your thinking on using presentation software in your teaching.

For more humorous, but none-the-less thought provoking approach, see Rebecca Shuman’s anti-PowerPoint tirade featured in Slate (March 7, 2014): PowerPointless. With the tagline, “Digital slideshows are the scourge of higher education,” Shuman reminds us that “A presentation, believe it or not, is the opening move of a conversation—not the entire conversation.”

Shuman offers a practical guide for those, like her, who do use presentation software, but seek to avoid abusing it. “It is with a few techniques and a little attention, possible to ensure that your presentations rest in the slim minority that are truly interactive and actually help your audience learn.” Speaking.io, the website Shuman references, discusses the use of presentation software broadly, not just for academics, but has many useful ideas and tips. 

For a resource specific to academic use, see the University of Central Florida’s Faculty Center for Teaching & Learning’s Effective Use of PowerPoint. The experts at the Center examine the advantages and challenges of using presentation software in the classroom, suggest approaches to take, and discuss in detail using PowerPoint for case studies, with clickers, as worksheets, for online (think flipped classes as well) teaching, the of use presenter view, and demonstrate best practices for delivery and content construction.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: CC Oliver Tacke https://www.flickr.com/photos/otacke/12635014673/

To Curve or Not to Curve Revisited

Yellow traffic signs showing a bell curve and a stylized graph referencing criterion-referenced grading.The practice of normalizing grades, more popularly known as curving, was a subject of an Innovative Instructor post, To Curve or Not to Curve on May 13, 2013. That article discussed both norm-referenced grading (curving) and criterion-referenced grading (not curving). As the practice of curving has become more controversial in recent years, an op-ed piece in this past Sunday’s New York Times caught my eye. In Why We Should Stop Grading Students on a Curve (The New York Times Sunday Review, September 10, 2016), Adam Grant argues that grade deflation, which occurs when teachers use a curve, is more worrisome than grade inflation. First, by limiting the number of students who can excel, other students who may have mastered the course content are unfairly punished. Second, curving creates a “toxic” environment, a “hypercompetitive culture” where one student’s success means another’s failure.

Grant, a professor of psychology at the Wharton School at the University of Pennsylvania, cites evidence that curving is a “disincentive to study.” Taking observations from his work as an organizational psychologist and applying those in his classroom, Grant has found he could both disrupt the culture of cutthroat competition and get students to work together as a team to prepare for exams. Teamwork has numerous advantages in both the classroom and the workplace as Grant details. Another important aspect is “…that one of the best ways to learn something is to teach it.” When students study together for an exam they benefit from each other’s strengths and expertise. Grant details the methods he used in constructing the exams and how his students have leveraged teamwork to improve their scores on course assessments. One device he uses is a Who Wants to Be a Millionaire-type “lifeline” for students taking the final exam. While his particular approaches may not be suitable for your teaching, the article provides food for thought.

Because I am not advocating for one way of grading over another, but rather encouraging instructors to think about why they are taking a particular approach and whether it is the best solution, I’d like to present a counter argument. In praise of grading on a curve by Eugene Volokh appeared in The Washington Post on February 9, 2015. “Eugene Volokh teaches free speech law, religious freedom law, church-state relations law, a First Amendment Amicus Brief Clinic, and tort law, at UCLA School of Law, where he has also often taught copyright law, criminal law, and a seminar on firearms regulation policy.” He counters some of the standard arguments against curving by pointing out that students and exams will vary from year to year making it difficult to draw consistent lines between, say an A- and B+ exam. This may be even more difficult for a less experienced teacher. Volokh also believes in the value of the curve for reducing the pressure to inflate grades. He points out that competing law schools tend to align their curves, making it an accepted practice for law school faculty to curve. As well, he suggests some tweaks to curving that strengthen its application.

As was pointed out in the earlier post, curving is often used in large lecture or lab courses that may have multiple sections and graders, as it provides a way to standardize grades. However, that issue may be resolved by instructing multiple graders how to assign grades based on a rubric. See The Innovative Instructor on creating rubrics and calibrating multiple graders.

Designing effective assessments is another important skill for instructors to learn, and one that can eliminate the need to use curving to adjust grades on a poorly conceived test. A good place to start is Brown University’s Harriet W. Sheridan Center for Teaching and Learning webpages on designing assessments where you will find resources compiled from a number of Teaching and Learning Centers on designing “assessments that promote and measure student learning.”  The topics include: Classroom Assessment and Feedback, Quizzes, Tests and Exams, Homework Assignments and Problem Sets, Writing Assignments, Student Presentations, Group Projects and Presentations, Labs, and Field Work.

Macie Hall, Instructional Designer
Center for Educational Resources

Image Source: © Reid Sczerba, 2013.