Notice and Wonder

Sometimes you discover a great blog, only to learn that the authors have just posted their final article. At least, in this case, the posts are archived by a professional association, and in searching through said archive, you find a gem. Such is the story of The Teaching Tidbits Blog, hosted by the Mathematical Association of America.

Black and white line drawing of the upper torso of a young male in a thinking pose. Two question marks are on either side of his head.The gem of a post is The Exercise with No Wrong Answer: Notice and Wonder (April 18, 2018) written by May Mei, Assistant Professor of Computer Science and Mathematics at Denison University. Mei asks: “How often have your students said nothing rather than risk saying something wrong? And how often in our own writing are we so paralyzed by the fear of imperfection that we end up writing nothing at all?” The solution is an exercise called Notice and Wonder, which “has no wrong answers.” The concept is to present students with an idea, which could be in the form of a drawing, diagram, graphical presentation, image, video clip, a sentence written on the blackboard, etc., and ask students what they notice about it. After students have explored the possibilities, they are asked what they wonder about it. Mei states: “This provides opportunities to discuss what is still unknown and puts all students on equal ground-everyone can wonder about something. Because there are no wrong answers for either of these questions, all students can participate in the activity.

Notice and Wonder exercises allow students to brainstorm ideas and explore a problem before attempting to find a solution or uncover meaning. These activities can be used in-class, but the concept is flexible enough to be adapted to online discussion board threads or outside-of-class group work. Mei uses it as a way to help her students review for exams. “If my students have an upcoming exam, I will use the class period before as review, and sometime before that review period I ask each of my students to email me one thing they noticed and one thing they’re still wondering about. Just before class on the review day, I put all the students’ Notices and Wonders into one document and distribute it to the class. As I’m making this document I’m able to recognize themes and repeated observations or questions and can focus on those during the in-class review.”

Many of the examples of these activities revolve around mathematics, and/or are directed at the K-12 population. But as creative thinkers, it is easy to see that this is an idea that can be expanded to any discipline or age group. A video by Annie Fetter, on staff at Math Forum (which is now part of National Council of Teachers of Mathematics), is often cited in articles on Notice and Wonder. It is directed at K-12 teachers, but demonstrates the concept in a way that will be useful to anyone considering using Notice and Wonder exercises. See Ever Wonder What They’d Notice (5 minutes).

Notice and Wonder activities involving data visualizations and images can help students develop interpretive, critical thinking, analytical, and visual literacy skills. The New York Times offers two resources: What is Going on in This Picture and the more recently launched What is Going on in This Graph. Of the latter, the Times says: “The content and statistical concepts will be suitable for most middle and high school students. Often, we’ll strip it of some key information, then ask students three question — inspired by Visual Thinking Strategies, but anchored in math and statistics thinking: • What do you notice? • What do you wonder? • What’s going on in this graph?” Again, these platforms are directed towards a K-12 audience, but looking at the examples may inspire you to conceive examples that will be relevant for your college-age students.

Have you used Notice and Wonder exercises in your class? If so, please share your experience in the comments.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Image from Pixabay.com

 

Lunch and Learn: Teaching Discussion-based Classes

Logo for Lunch and Learn program showing the words Lunch and Learn in orange with a fork above and a pen below the lettering. Faculty Conversations on Teaching at the bottom.On Friday, April 20, the Center for Educational Resources (CER) hosted the final Lunch and Learn—Faculty Conversations on Teaching, for the 2017-2018 academic year.

Sahan Karatasli, Assistant Research Scientist and Lecturer, Arrighi Center for Global Studies, Sociology, and Bill Egginton, Professor, German and Romance Languages and Literatures, presented on Teaching Discussion-based Classes.

Karatasli led off with a presentation that looked at various strategies for leading discussion (see slides). He noted that lectures are a straightforward means of disseminating information to students, and we have a pretty good understanding of what constitutes success and failure in this format. Seminars are more complicated. Nobody is sure what good ones look like. We don’t have a sense of how to determine success or failure—in general, leading a seminar is a more complex form of the teaching/learning practice.

What is good discussion? Karatasli noted that it is possible to have discussion within the lecture format, in the form of the “The Usual”: Q and A. The lecturer throws out a question and students volunteer answers or risk being called upon. Moving into the seminar style class or discussion section, Karatasli identified four strategies.

“Free-for-all” describes staging a debate where students actively participate with little interference from the instructor. The strategy favors students who are well-prepared, understand the course content, and aren’t afraid to voice their opinions. Students who feel less confident may hide under the table (see slides to appreciate the analogy) and won’t learn.

With the “Kids’ Participation Trophy” strategy, the instructor encourages all students to participate without risk of criticism for their opinions. This can be useful in some classes, especially at the beginning of the term, to start discussion, as students will feel safe about contributing. To be sure that the class isn’t operating under misconceptions, it is important that the instructor finds a way to gently correct misinformation without shutting down the student.

“Socratic House Tour” involves asking one student (or several students) to come to class prepared to present or explain an idea or concept. The other students then visit the idea with questions, counter-ideas, further explanations. At the end, the instructor summarizes and synthesizes the discussion. These sessions should be designed so that all students take a turn as presenters during the semester. While some students may feel anxious in the role of presenter, the strategy tends to ensure that students will participate in the discussion, knowing that in the future they will be the ones leading the class.

“Barn Raising” builds on the “Socratic House Tour” in that one student introduces an idea or project and other students work to make it more substantive. This is the most creative and collective strategy and one where the instructor helps with the process.

Which strategy is the most helpful? That depends on your objective, the subject matter 13 students and a professor in discussion around a seminar table.you are teaching, and the audience. Karatasli suggested that most instructors will use a combination of methods in their teaching.  Lecturing with Q&A is an efficient way to disseminate a lot of information, the “free-for-all” is good when there is not necessarily a right or wrong answer and you want to explore ways of thinking about a topic, “kids’ participation trophy” helps in situations where you want to raise students’ self-esteem. Subject matter also comes into play—if you are teaching statistics, there is a right and wrong answer, so the “barn raising” approach might be most effective at getting the students where you want them to be.

Karatasli ended by talking about mental preparation and offering some tips. Preparation is key, perhaps even more so for discussion classes than lectures. Make sure you know what you want to achieve from the discussion. Prepare alternative sets of plans, questions, and activities for discussion. Know the level and expectations of students. And it is important to learn the names of all of your students and to use them.

The physical environment matters. Arrange the room to facilitate discussion—ideally in a circle or around a table. Give students time to respond. Don’t answer your own questions. Instead, turn to other students and ask what they think.

Bill Egginton offered a different approach by giving us a case study of a course he is currently teaching in the Department of German and Romance Languages and Literatures:  Cervantes: Don Quixote and The Exemplary Novels. This is a new course, and in designing it Egginton faced several challenges.

In the upper-level literature seminar, the students would be reading a total of 12 novels and novellas. He wanted students who wished to get credit towards the Spanish major to be able to read the works in the original language and attend a class conducted in Spanish. At the same time, he wanted to make the course available to other students, who could read the works in English translation and discuss them in English. As there were over 1600 pages of primary literature assigned, Egginton did not feel he could assign any secondary material, and would have to cover background material and context in class. He did, however, want the class to be primarily discussion-based, so needed to limit lecturing. He also wanted to be sure the students were keeping up with the reading and to have assessment be writing-based, not quizzes, test, or exams.

How did Egginton meet these challenges? He taught the course on Tuesdays and Thursdays in 90 minute sessions. Tuesdays were lecture days, but Egginton incorporated discussion as well by asking students to give examples from the week’s reading in response to the background he presented. He made sure to include lots of visual materials in his lectures. On Thursdays, the class was split into two sections, one English language and one Spanish language, meeting at the same time. He divided the class time in half. In one section, he would start the students with a challenge—for instance, a collaborative writing assignment. He would then move to the other classroom and spend some time in discussion on the readings before giving that section the day’s challenge. He would then return to the first section and have discussion with those students. Egginton says that he got lots of exercise on Thursdays going between the two sections’ classrooms.

Another important component of the course was a class discussion board in Blackboard. Egginton was relatively new at using Blackboard’s features, but felt that the discussion board added an important component to the course. Each week he posted prompts and all students were expected to respond. He did not grade the responses, just checked that students had completed the assignment. This work ensured that students kept current with the reading. Egginton would read the responses before class each week and use the responses to facilitate class discussion. In addition to the weekly board writing, students write and revise a one-page paper mid-semester, and a three-page paper at the end of the semester.

Egginton has been pleased with how the course is going and feels that the design has allowed him to meet all of the challenges.

The discussion following Karatasli and Egginton’s presentations was appropriately stimulating and thought-provoking. Following are some of the questions asked and the responses given by Egginton and Karatasli as well as participants.

Q: How do you deal with the wall of silence?

Egginton suggests asking a question or making a statement and then calling on students in a conversational way: “Emma, what do you think about that?” After one student answers, he follows with a comment and then, “Carlos, what are your thoughts?” In the course he is teaching now, he refers to the specific discussion board posts that students have made and asks them to elaborate or clarify, or asks other students what their ideas are.

Karatasli noted that there is a study that shows that students don’t mind being called on, but do object to the instructor allowing one or two students to dominate the conversation. This problem can be dealt with by having the dominating student take notes on the board, or by pointedly asking that student to allow others to answer.

Others noted that it can be helpful to start with questions or topics that are accessible to students—personal, concrete, and specific—and then, once students are contributing, move to more challenging concepts. Visuals can be useful in stimulating discussion, as well as having students do prep work in advance of class, such as posting on a discussion board, or writing out questions in response to a reading assignment to bring to class.

Students who feel they are part of a community will be more likely to participate in discussion. Egginton suggested that collaborative work is a good way to build that feeling. Karatasli offered some specific solutions: Students can collaborate on a research project where some work is done in class. Or, in advance of class, assign a group of students to be responsible for asking questions for the discussion or preparing a short in-class presentation.

Q. How do you grade participation in discussion?

Egginton does not find this to be a problem because he makes sure to call on all students. If they are attending class, they are participating. Someone else noted that there is a concept in discourse analysis called uptake. Uptake is adding to the discussion and building on it, elevating it. That is what to look for when thinking about grading participation—that students are doing more than just answering responsively.

Q. How do you handle sensitive topics?

Egginton commented that sometimes attempts at empathy can go astray. It’s not necessarily appropriate to say that you can relate. Someone offered that the instructor should lead the discussion by stating what the expectations are and asking students to keep comments on topic and relevant.

Q. How do you handle a situation where one student has polarizing political or religious views and makes comments that become disruptive to class discussion?

Eggington suggested that the instructor attempt to seek a balance. Can you encourage other students to express opinions? Ask the student to give evidence or present arguments rather than simply make comments. Perhaps the class could have a short debate and then return to the topic at hand.

Karatasli proposed saying: “I hear you.” Then put the comment or question on the board or state or rephrase it. Then ask what others think. Another thought was to ask students to switch sides and argue the opposite viewpoint. In this way you can unlink the person from the idea.

Egginton added that knowing the student’s perspective, the instructor can be prepared with facts and counter-questions to ask exactly what the student is asserting.

Q. How do you bring a discussion back when it veers off topic?

Karatasli referred to the different discussion strategies he had presented, which may mean that in each case the instructor has a different role. But, generally the instructor acts as a referee and when the ball goes off the field, the referee must bring it back into play.  Egginton sees this as an occupational hazard. He will say something like: “That’s an interesting insight—how can we relate that to the reading?” Someone else suggested that it may be appropriate at times to go off topic. The purpose of college is to explore ideas that are challenging and sometimes off-topic discussions can elevate your class.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Wikimedia Commons

 

 

Texts of Engagement: Reading and Writing for Inclusivity in Any Discipline

[Guest post by Anne-Elizabeth Brodsky, Senior Lecturer, Expository Writing, JHU]

As part of the Lunch and Learn series, Karen Fleming (biophysics professor) and I gave presentations on fostering an inclusive classroom environment, summarized in the previous post.

In this follow up, I’ll offer some resources that seem to me versatile enough to use in different kinds of classrooms, and toward different ends. I’ve divided them into the categories below; the sources in the first three sections, in particular, can nearly all live quite comfortably in most disciplines.

What is college for?
Why bother with stories
How textual analysis works
Ideas for writing assignments

What is college for?

Cartoon rendering of an illuminated light bulb.Plenty of college students, from visibly diverse backgrounds and otherwise, feel mystified by the academy, at best —and overwhelmed by it, at worst. One way I try to help them find their footing is to engage them in examining university culture.

In my courses we use the sources below, and others, for informal “response writing.” I’m not using these sources to teach academic argument, although plenty of them have elegant arguments built into their narratives, and my students do notice that. I use them in hopes of activating their engagement and imagination about what their undergraduate experience will mean for them—to give them glimpses from the skybox.

Here’s a sampling:

Anna Deveare Smith raises the stakes of higher ed in her Chronicle interview “Are Students “Learning Anything About Love Here?’”

In “How to Live Wisely,”  Richard J. Light describes the Harvard course “Reflecting on Your Life.” Students explore: What does it mean to live a good life? What about a productive life? How about a happy life? How might I think about these ideas if the answers conflict with one another? And how do I use my time here at college to build on the answers to these tough questions?”

Mark Edmundson, first in his family to go to college, takes on undergraduate experience at research institutions in “Who Are You and What Are You Doing Here?”

“The Cosmos and You,” just a few paragraphs long, tells the story of an astronomy student who asks her professor the big question: “How do you keep from despairing at the immensity of space and the smallness of us?”

Julio M. Ottino and Gary Saul Morson write about empathy, engineering, and more in “Building a Bridge Between Engineering and the Humanities.”

Neil Koblitz advocates for clear, engaging storytelling in the sciences in “Why STEM Needs the Humanities.”

Lyell Asher’s essay “Low Definition in Higher Education” takes on Anna Karenina, safe spaces, and more: “Rightly understood, the campus beyond the classroom is the laboratory component of college itself. It’s where ideas and experience should meet and refine one another, where things should get more complicated, not less.”

Nicholas Lemann’s “The Case for a New Kind of Core” proposes “a methods-based, rather than a canon-based, curriculum,” with skills categories: information acquisition, cause & effect, interpretation, numeracy, perspective, language of form, thinking in time, and argument.

“The Astrophysicist Who Wants to Help Solve Baltimore’s Urban Blight” tells the story of a Hopkins professor and the Baltimore Housing Commissioner teaming up to tackle our vacant housing problem.

Finally, a dated but still very popular article among my students is Thomas Friedman’s “How to Get a Job at Google.” To a person, every student who has read this has been surprised. At JHU as at many other schools, it’s common for students to see college as a means to an end—professional school, a big job. Living in the future tense, focused almost entirely on the retroactive value of college they plan to enjoy after graduation, they default to a safe, well-worn path of coursework and GPA management. Friedman’s article, like many of those above, nudges them a bit on this.

Why bother with stories

Sea turtle rendered as a multi-colored mosaic.A central tenet of the Expository Writing Program at Hopkins is that our students engage in meaningful writing. Our students write to contribute to an ongoing intellectual conversation about a genuine problem or question. We expect students to think not only about their argument, based on the evidence at hand, but also about the implications of their argument.

Thus if I’m asking students (in my classes, mostly STEM majors) to enter a critical conversation about a short story, I have to be sure we are explicit about how it could possibly matter whether or not they make a compelling argument about a piece of fiction—you know, something that never really happened.

So, in class we ask the question: Why bother with stories? Here are some resources that have helped answer this:

Short fiction warrants a column in the New York Times op-ed pages in David Brooks’ “The Child in the Basement,” which nicely models a summary (of Ursula Le Guin’s “The Ones Who Walk Away from Omelas”) and then offers three socially relevant readings of the text.

Diaa Hadid’s interview with Mahlia Lone and Laaleen Sukhera, “2 Sisters in Pakistan Find They Have a Lot in Common with Jane Austen” illustrates connections between upper-class contemporary Pakistan and the England of Austen’s novels.

Two TED talks: The well-known “The Danger of a Single Story” by Chimamanda Ngozi Adichie and Raghava KK’s also wonderful 4½ -minute “Shake Up Your Story.”

Here’s an idea I haven’t tried yet: put Adichie’s talk next to John W. Krakauer and David Poeppel’s “Neuroscience Needs Behavior: Correcting a Reductionist Bias,” which is also about the “danger of a single story.”

Most recently: Lindy West writes in her op-ed “We Got Rid of Some Bad Men. Now Let’s Get Rid of Bad Movies”: “Art didn’t invent oppressive gender roles, racial stereotyping or rape culture, but it reflects, polishes and sells them back to us every moment of our waking lives.”

All of this depends on analysis, of course, and especially textual analysis…

How textual analysis works

Budding tree growing out of the center of an opened book.There are lots of ways to teach students how to do textual analysis (aka close reading). A few favorite resources:

“Twenty Titles: One Poem,” by poet Douglas Kearney, helps students see different interpretations (titles) of the same data (the sculpture).

Here, students analyze the poem “Since You Are Mortal” (Simonides) and then turn over the page to see how the translator, Danielle S. Allen, interprets it.

Danielle S. Allen, Our Declaration: A Reading of the Declaration of Independence in Defense of Equality (excerpts)

John Oliver, “Migrants and Refugees” (1:12-1:54)

NPR Code Switch Podcast: Camila Domonoske, “When ‘Miss’ Meant So Much More: How One Woman Fought Alabama — And Won”

Two more TED talks: John McWhorter’s “Txting is Killing Language. JK!!!” and Jean-Baptiste Michel and Erez Lieberman Aiden’s “What We Learned from 5 Million Books”

Ideas for writing assignments

Colored pencils arranged in a row.Again, just a quick sampling of texts that have worked well with first- and second-year students. My colleagues have loads of their own, as you likely do as well.

Summarizing an academic article:

Lin Bian et al, “Messages About Brilliance Undermine Women’s Interest in Educational and Professional Opportunities”

Samantha P. Fan et al, “The Exposure Advantage: Early Exposure to a Multilingual Environment Promotes Effective Communication”

Martha C. Nussbaum, “Teaching Patriotism: Love and Critical Freedom”

Finding an interpretive question in a text and reasoning through it:

James Baldwin, “Stranger in the Village”
Octavia Butler, “Bloodchild”
Ursula K. Le Guin, “She Unnames Them”
Toni Morrison, “Recitatif”
ZZ Packer, “Brownies”
David Sedaris, “Repeat After Me”
Alice Walker, “The Revenge of Hannah Kemhuff”
Patricia Williams, “The Emperor’s New Clothes”
EB White, “The Ring of Time” (the whole essay, including the section on Florida and segregation)

Entering a critical conversation about a text:

Build an argument about EB White’s “The Ring of Time” by responding to Craig Seligman’s review “Mr. Normal”

Build an argument about Octavia Butler’s “Bloodchild” by responding to the Butler’s comments in the Afterword about love and slavery

Build an argument about Toni Morrison’s “Recitatif” by responding to two of the following: Morrison’s prologue to Playing in the Dark; Elizabeth Abel’s “Black Writing, White Reading: Race and the Politics of Feminist Interpretation”; David Goldstein-Shirley’s “Race and Response: Toni Morison’s ‘Recitatif’”; and Danielle S. Allen’s prologue, chapter 1, and chapter 9 of Talking to Strangers: Anxieties of Citizenship Since Brown v. Board of Education.

 

Anne-Elizabeth Brodsky, PhD, has been teaching at Johns Hopkins since 2007 and co-chairs, with Professor Fleming, JHU’s Committee on the Status of Women.

Images source: Pixabay.com

Lunch and Learn: Fostering an Inclusive Classroom

Logo for Lunch and Learn program showing the words Lunch and Learn in orange with a fork above and a pen below the lettering. Faculty Conversations on Teaching at the bottom.On Thursday, February 15, the Center for Educational Resources (CER) hosted the third Lunch and Learn—Faculty Conversations on Teaching, for the 2017-2018 academic year.

Anne-Elizabeth Brodsky, Senior Lecturer, Expository Writing and Karen Fleming, Professor, Biophysics, presented on Fostering an Inclusive Classroom.

Anne-Elizabeth Brodsky led off by describing the Expository Writing program. [See presentation slides.] Students learn in small class settings (10-15 students) how to write and academic argument using evidence. The academic argument is framed as an ongoing conversation—they say (identifies the problem or viewpoint), I say (recognizes a flaw in the argument and establishes a thesis to correct the flaw), so what? (what’s at stake or what’s next?)—to which the student is contributing. The vast majority of her students are in the STEM fields. This is the background for Brodsky’s introduction of diversity and inclusion in the classroom.

Students bring their complex identities and a sense of whether or not they feel they belong on campus to the classroom. Brodsky seeks to make sure that her curriculum includes reading diverse authors. Students are given a number of choices for each major writing assignment so that they can find something that interests and speaks to them. Additionally, students do informal writing on topics fincluding What is College For? Learning (science of learning, recent research, advice), School (campus life, student realities, ideas for redesign), Stories—why they matter, Words (language change, texting, big data). For each topic area Brodsky identifies a number of articles that students can choose from to read and write a response.  She tries to make topics current and relevant, and to be flexible, for example, after the death of Freddie Gray in April 2015, “I offered my students supplemental reading. I hoped to show how the JHU community engaged in local, ‘real-life’ struggles with rigorous, historically responsible, and creative thinking.” She encourages interdisciplinary conversations so that students don’t shut themselves off from possibilities.

Brodsky works with students who are afraid of writing and finds that different approaches work for different students. She is an advocate of Universal Design for Learning (UDL). In the classroom she practices eight ways of teaching.

  1. Giving students permission to fail successfully. The writing process is all about Slide from Anne-Elizabeth Brodsky's presentation listing 8 ways of learning.drafts and revisions; students learn that their first efforts may not work well, but they can learn from mistakes and rewrite.
  2. Using color highlighting while reading, by Brodsky and the students, to understand how an academic argument works.
  3. Sorting through key words used in an academic argument to understand how language works. For example, when identifying a flaw in an argument, key words are repudiate, contest, reject. Next, Brodsky has students perform textual analysis, by looking at a text and deciding whether it is either a conversation over coffee or actual textual analysis. For example, the text being analyzed is a comment on this quote (taken from a John Oliver segment): “A former British prime minister once described refugees as a “swarm of migrants.” Answer A is: “That made me crazy.” [conversation] Answer B is: “The word ‘swarm’ suggests invasion, threat, as well as equating humans to insects.” [textual analysis]
  4. Using Lego building blocks with statement taped to the sides, students “build their argument.”
  5. Learning to analyze and interpret by practicing describing things in different ways.
  6. Employing different ways of talking and discussing in the classroom. Students prepare elevator talks, engage in think/pair/share activities, anonymously share worries and wishes on index cards, and write questions to establish knowledge.
  7. Crafting a succinct “elevator pitch” to streamline their own academic argument.
  8. “What I mean here is to help and expect students to engage fully in their work on campus—not just their science brains for science, or even just their intellectual skills for their courses. Rather, they come here with richness of experience far beyond the academic classroom.”

In engaging students, Brodsky helps them to see frustration as unrealized potential and uncertainty as curiosity. She drew on a 2013 interview with British psychoanalyst and essayist Adam Phillips: “A student who’d been frustrated for weeks by my suggestion that thesis statements were probably not good things to have about works of literature began to recognize that her frustration was partly a fear of freedom. ‘You’re really just increasing my allowance,’ she said.”

By engaging in inclusive teaching practices, Brodsky has given her students the freedom to learn.

Karen Fleming started off by stating her goals for an inclusive classroom: To create a learning environment where everyone feels safe to express ideas and questions, where discussion brings multiple diverse questions and all opinions are considered, and where student experiences of marginalization are minimized. Two of the many challenges that instructors face are unconscious bias and student stereotype threat. [See presentation slides.]

Fleming noted that 99% of our cognitive processing uses unconscious reasoning. Unconscious bias stems from our experiential expectations; bias being an error in decision making. Expectation bias is grounded in stereotypes such as women are not as good at math or spatial learning; Hispanics and African Americans are lower achieving than Whites; Hispanics, African Americans, poor, and obese people are lazy; male athlete are jocks (muscular but not smart); Asians have higher math abilities; and blondes are less intelligent than brunettes. Stereotypes are cultural and both faculty and students have biases.

Fleming described a research study on science faculty and unconscious bias [Moss-Racusin et al. (2012) “Science faculty’s subtle gender biases favor male studentsPNAS 109: 16474-79.] The study was headed by Jo Handlesman, Professor, Department of Molecular, Cellular & Developmental Biology, Yale University. Fleming walked through the study details, noting that the institutions selected—research intensive universities—were geographically diverse, the demographics on faculty participants agree with national averages, and it was a randomized double-blind study (n = 127). Science faculty in biology, physics, and chemistry were asked to rate the application of a student for a lab manager position in terms of competency and hireability. Faculty participants were randomly assigned an application with a male or female name—the applications were otherwise identical and designed to reflect high but not outstanding qualifications.

Both male and female faculty participants showed gender bias by rating the male applicant as significantly more competent and hireable than the female applicant. They were willing to offer a significantly higher starting salary (and more career mentoring) to the male applicant. And, perhaps surprisingly to women in the audience, both female and male faculty were equally likely to show gender bias favoring male applicants and to offer the female candidates a lower salary. One possible explanation? Male stereotypes meet the expectation for the lab manager position.

Slide from Karen Fleming presentation showing male and female stereotypes.Expectation bias occurs when we hold different groups accountable to different standards. When a man performs well in a traditional male-type task, this performance is expected. When a woman performs well in a traditional male-type task, this conflicts with stereotypic expectations. As a result, her performance is closely scrutinized, and she is required to repeatedly prove her competence.

We can check our biases using tests set up by Project Implicit. “Project Implicit is a non-profit organization and international collaboration between researchers who are interested in implicit social cognition – thoughts and feelings outside of conscious awareness and control. The goal of the organization is to educate the public about hidden biases and to provide a ‘virtual laboratory’ for collecting data on the Internet.” Look for tests on bias concerning gender, race, religion, disability, age, sexuality, weight, skin tone, and more  in the Social Attitudes section.

Fleming then discussed the issue of stereotype threat. For students, performance in academic contexts can be harmed by the awareness that one’s behavior might be viewed through the lens of stereotypes. And, culturally-shared stereotypes suggesting poor performance of certain groups can disrupt performance of an individual who identifies with that group. She referenced a study done by Steele and Aronson in 1995 [CM Steele and J Aronson (1995) Journal of Personality and Social Psychology 69: 797-811.] At Stanford, 114 undergraduate students volunteered for the study. They were given a test composed of Verbal GRE questions. First they took a test that was presented as diagnostic, a measure of intrinsic ability, and then a test presented as non-diagnostic, a laboratory tool for studying problem solving. The results showed that invoking stereotype threat (diagnostic test) affects performance of students who identified as African-Americans in an evaluation setting. Further, having to identify their race before taking the test, “race-priming,” increases the results of stereotype effect on performance in an evaluation setting.

How can instructors be more inclusive? Fleming offers these suggestions.

  • Establish a Growth Mindset
  • Set clear, equal and high but achievable expectations for all students
  • Include material created by people of different backgrounds
  • Pay attention to group dynamics (especially compositions of small groups).

As both Fleming and Brodsky reminded the audience, instructors can change the way they teach to be more inclusive, identify their biases, and work to establish a positive classroom climate.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Sources: Lunch and Learn Logo (© CER), slides from Brodsky and Fleming presentations

 

Fail (in order) to Succeed

Last week I attended the CUE2 Symposium held here on the Homewood Campus of Johns Hopkins University. CUE2 stands for the Second Commission on Undergraduate Education. The first CUE was formed in 2002 “…to identify the core values that should characterize the undergraduate experience of our students.” The charge of the second commission “… is to interpret the mission of an undergraduate education in the 21st century and develop a new model that will serve us for the next decade or more.” In fulfilling that charge, CUE2 leaders have invited noted thinkers and innovators in undergraduate education to campus to address these issues. Three speakers addressed us last week, Edward B. Burger, President, Southwestern University, Randy Bass, Vice Provost for Education and Professor of English at Georgetown University, and Steven Mintz, Professor of History at University of Texas, Austin and founding director of The University of Texas System’s Institute for Transformational Learning.

All three had interesting things to say about transforming undergraduate education, but I found Edward Burger’s remarks to be particularly intriguing.

Burger is the co-author of a book, The Five Elements of Effective Thinking, with Michael Starbird [Princeton University Press, 2012]. Burger and Starbird describe five elements that characterize effective thinking: understanding deeply, failing in order to succeed, raising questions, seeing the flow of ideas, and engaging change. A review of the book on Farnam Street Blog summarizes the elements. The book is a quick read and one that I think instructors will find valuable in thinking at a high level about learning and teaching. The chapter on how mistakes can ignite insight, along with Burger’s remarks at the symposium on effective failure, struck a chord.

Last summer The Innovative Instructor featured two posts on successful failure, When Failure is a Good Thing (July 14, 2017) and How Pretesting Can Help Your Students Fail Well (July 18, 2017. The first post discussed a program at Smith College that encourages students to understand the value of failure as a learning tool, the second offered a practical means, pre-testing, for instructors to use failure as an instructional strategy.

Burger maintains that effective failure is key to successful learning. It’s what you do after you make a mistake that informs your understanding. If a student receives an 80% on a test, does s/he think that is an adequate grade and move on or does the student take a careful look at what questions were answered incorrectly and seek to understand the topic? Burger characterizes acceptance of the 80% grade without further refelction as “ineffective failure.”

In Burger’s math courses, 5% of a student’s grade is based on effective failure; in order to get an A, students must first fail. Taking this approach opens students to taking risks and learning from their mistakes. When a student answers a question in class incorrectly, the instructor should ask, “Why is this wrong?” Let the students be the teachers and transform the failure into moving thinking and understanding forward. Burger maintains that “every mistake is a teacher and holds a lesson.” [The Five Elements of Effective Thinking, p. 50].

Burger summarizes the concept of making mistakes: “Fail to succeed. Intentionally get it wrong to inevitably get it even more right. Mistakes are great teachers — they highlight unforeseen opportunities and holes in your understanding. They also show you which way to turn next, and they ignite your imagination.” [p. 6].

These are valuable strategies for students and instructors alike.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

The Open Faculty Patchbook: An OER on Pedagogy

We are seeing an increased interest in Open Educational Resources (OER) in our library and among faculty. [See Consider the OER an Innovative Instructor blog post by Marian Feldman, November 7, 2016.] What is an OER? Wikipedia defines open educational resources as being “freely accessible, openly licensed text, media, and other digital assets that are useful for teaching, learning, and assessing as well as for research purposes.”

With the skyrocketing costs of textbooks used in higher education, colleges have responded to student demand by instituting programs to support the production of OER texts, especially for discipline-standard introductory courses. There is a listing of OER initiatives, resources, and projects on the SPARC (the Scholarly Publishing and Academic Resources Coalition) website. The Open Education Consortium also lists OER initiatives, including a list of open textbooks.

Image resembles a patchwork quilt with 26 hexagonal patches, each with a stylized image of a person representing the authors of the articles making up the book.By happenstance, I came across an OER written by faculty for faculty on pedagogy. The Open Faculty Patchbook was created by faculty at Fleming College in Peterborough, Ontario.  The Learning Design and Support Team at Fleming was tasked in 2016 with revising the faculty development model. Inspired by a presentation given by Robin DeRosa at the 2016 Open Education Conference in Richmond, Virginia, the team decided to create a how-to teach manual. They came up with the metaphor of a quilt, with each contributor creating a “patch” to add to the “community quilt of pedagogy.” Currently, there are 21 pieces describing how instructors do their work. Topics include cohort-based-learning, teaching within the Universal Design for Learning (UDL) framework, activating students’ background knowledge, formative assessments, facilitating deep learning, laboratory assessments, co-teaching, group work, and more.

The Patchbook was initially designed to cover the University of Michigan’s School of Education High Leverage Practices, which are described as “…the basic fundamentals of teaching. These practices are used constantly and are critical to helping students learn important content. The high-leverage practices are also central to supporting students’ social and emotional development. These high-leverage practices are used across subject areas, grade levels, and contexts. They are “high-leverage” not only because they matter to student learning but because they are basic for advancing skill in teaching.”

The patches in the Open Faculty Patchbook are relatively short, making this an easy to digest guidebook. It is also open to contributors for additional patches. At the end of the current book is the statement: “Future versions planned include ones focused on professional learning, digital pedagogy (online learning) and course design. If you would like to contribute, email ldsteam@flemingcollege.ca with the subject heading ‘I’m awesome. I want to add a Patch’. Your patch, when complete, would immediately appear on our WordPress site facultypatchbook.wordpress.com and be added to a Pressbook publication.”

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Open Faculty Patchbook

Teaching Transparently

Back in September 2015, The Innovative Instructor posted Do Your Students Understand the Assignment?, an article that examined the concept of transparent Semi-spherical transparent soap bubble on a grey wood surface.teaching. Transparent teaching helps students understand the why and how of their learning. Research from the University of Nevada, Las Vegas (UNLV) Transparency in Learning and Teaching Project (TILT) has shown that when students understand the task, its purpose, and the criteria for evaluating their work, they are more motivated and feel the work is more relevant. The TILT website has some excellent suggestions and resources for instructors, including examples of assignments from various disciplines presented in two versions, less transparent and more transparent, for comparison.

A recent post on Teaching Tidbits,  a blog sponsored by the Mathematical Association of America to keep higher ed math faculty up on advances in educational research and pedagogical practices by providing “…quality, evidence-based ideas with high impact and low time commitment that can be used by a wide audience,” examined teaching with transparency. [October 24, 2017, How Transparency Improves Learning by Darryl Yong] Although the focus is on teaching college mathematics, the key points are applicable to a range of subjects.

Yong starts by citing the work done at UNLV, noting in particular the finding that underrepresented students experienced the greatest improvement in learning outcomes when transparent teaching methods were used. Yong speculates that transparent teaching helps to level the playing field for these students.

A key to teaching more transparently is to see things from your students’ vantage points. What would they find “bewildering, frustrating or alienating?” Being transparent does not mean that you don’t expect the work to be challenging rather that you will “engage your students in a productive struggle.”

Providing instructions in more than one format is helpful. For example, you should include information on assignments in writing on your syllabus, verbally in class, and again in written form in handouts to be sure that students aren’t missing important details.

Yong says, “The amount of transparency that you provide to students depends on their maturity and the level of the course. There are times when you don’t want to be explicit about everything. For example, you don’t want to constrain their creativity by priming them with examples, you want them to struggle with figuring out what the first step should be, or you want them to be more independent in their learning.”

He concludes the post with some suggestions on transparency for mathematics courses, but even these can be translated to more general use. The concepts are:

  • Be sure that students understand discipline-specific terminology.
  • Be clear about the tools, applications, and resources are students allowed to use for assignments and exams.
  • Explain why you have chosen a particular assignment, project, or type of exam. Connect these choices to their learning outcomes. Share strategies that successful students have used in the past for assignments and evaluations. Share rubrics when used for grading. Share examples of successful projects.
  • Tell students why you have chosen the pedagogical strategies you use to teach.
  • Start each class by highlighting a relevant current area of research and the people doing it. If your field has not been inclusive in the past, acknowledge that and “showcase women and people of color in these highlights to engage in counter-stereotyping.”

Teaching transparently will involve more planning and preparation for your course. It also means teaching intentionally. Improved learning outcomes and greater student satisfaction will make it worth your effort.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

 

Lunch and Learn: Creating and Implementing Authentic Assignments

Logo for Lunch and Learn program showing the words Lunch and Learn in orange with a fork above and a pen below the lettering. Faculty Conversations on Teaching at the bottom.On Tuesday, October 15, the Center for Educational Resources (CER) hosted the first Lunch and Learn—Faculty Conversations on Teaching—for the 201-2018 academic year.  Sanchita Balachandran, Associate Director, the Johns Hopkins Archaeological Museum and Senior Lecturer, Department of Near Eastern Studies; and Sauleh Siddiqui, Assistant Professor, Civil Engineering presented on their experiences using authentic assignments.

As a preface, students often ask why they need to learn something, and wonder when, if ever, they will use course information. Authentic assignments give students “real-world” experience and context, and involve hands-on, active learning.

Students building a kiln for Sanchita Balachandran's Greek Vases course.Sanchita Balachandran is Associate Director and conservator of the JHU Archaeological Museum as well as a lecturer in the Department of Near Eastern Studies. The collection was started in 1882, just six years after the founding of the University, and now occupies a jewel-box of a space in the renovated Gilman Hall, where its collection is at long last appropriately displayed. Balachandran uses the museum collection and “teaches courses related to the identification and analysis of ancient manufacturing techniques of objects, as well as the history, ethics and practice of museum conservation and curation.” She’s long been interested in authentic learning, and has recently taught two courses that exemplify this method: Recreating Ancient Greek Ceramics and Roman Egyptian Mummy Portraits.  [See presentation slides.]

When designing authentic learning assignments Balachandran asks herself a series of questions.

  1. Is this a question I am genuinely curious about and don’t know the answer to? With the course on recreating Greek ceramics she had long wondered how these objects were made (a subject of speculation and debate but no definitive answers). For both Balachandran and her students, it was both “exhilarating and terrifying” to not know what the end results would be. They would be discovering the answers together and this was motivating for the students.
  2. Is the question big enough, and are the stakes high? For her course on Roman Egyptian mummy portraits (Freshman Seminar: Technical Research on Archaeological Objects in the Johns Hopkins Archaeological Museum) the primary goal was to generate and collect technical data on these ancient portraits for contribution to an international data base. Other collaborators included the J. Paul Getty Museum, the British Museum, the Boston Museum of Fine Arts, the Walters Art Museum and the Art Institute of Chicago. The students were working with “big players” in the museum world.
  3. Do I have a physical thing that can be the focus of sustained and weekly examination and research? In both courses museum artifacts provided a focus point for the students.
  4. What methodology am I trying to teach? Balachandran’s methodology involved working hands-on with museum objects, consulting with experts and specialists in the field, documenting through writing, photography, and film the processes, and sharing observations and reflections with a broad audience. She noted that it was important that students experience moments of confusion during the process as it teaches them to think critically about, for example, past research, and what applies and doesn’t.
  5. What kind of expertise is need and who has it and will help? Balachandran spends a great deal of time in advance of her courses identifying relevant resources. She noted the value of Skype for bringing subject matter experts and specialists into the classroom from around the world.
  6. Is my class of students disciplinarily diverse? Balachandran advertises her courses broadly. Museum work often involves material scientists, for example. Her Greek vase course had students from materials science, applied mathematics, and biomedical engineering as well as the humanities and social sciences.
  7. Is the class work challenging and is there a hands on component? In each of the courses, Balachandran had students working with the materials that were used in the creation of the original art objects. The students made vases from clay using the techniques known to have been used in Ancient Greece; in the portrait course, they painted with encaustic, the material used by Roman Egyptians. She stressed that this was more than an arts and crafts session. Students studied the material science behind the techniques that were used and gained an appreciation for how the works were created.
  8. Is there an enduring “deliverable” or a regular public component to the class? Students contributed to the international data base in the mummy portraits class and blogged regularly as a part of the Greek vases class. Balachandran used social media (Facebook) to publicize student work. There was also a documentary film—Mysteries of the Kylix—made during the class that has been viewed over 4000 times. She arranged for radio spots on WYPR (Baltimore’s NPR station) and gained exposure through Johns Hopkins publications and the Baltimore Sun newspaper.
  9. Do I see my students as collaborators? Balachandran makes sure that students are given credit in the public components of the course and regularly acknowledges their participation. She sees herself as in the trenches with the students, finding answers to problems together.
  10. Am I ready not to be in control of what we find out? This is perhaps the most difficult step for an instructor to take with authentic assignments, but the one that will allow for the real learning gains. We learn from our failures as well as successes, and that is important for students to experience firsthand.

In conclusion, Balachandran summarized what students learned during her courses:

  • Everything is more complicated than we think and merits repeated examination/re-examination
  • Our work in the classroom produces unique specialized knowledge
  • We can participate in and contribute to scholarly conversations
  • We should broaden our own knowledge base and collaborate beyond our usual networks
  • We must provide access to the knowledge we produce
  • The process of trying to answer a question is more important than answering the question—and will lead to more interesting questions
  • We can/must ask more daring questions.

Siddiqui discussed the main components of authentic learning assignments as he uses them in his courses with the most important being that students should be doing rather than listening. [See presentation slides.] These are:

  • The judgment to distinguish reliable from unreliable information.
  • The patience to follow longer arguments.
  • The synthetic ability to recognize relevant patterns in unfamiliar contexts.
  • The flexibility to work across disciplinary and cultural boundaries to generate innovative solutions.

Example of problem involving transportation networks by Sauleh Siddiqui.In his course, Equilibrium Models in Systems Engineering, students work on real-life examples such as designing transportation networks. To demonstrate an exercise that Siddiqui uses in his course, he passed out clickers to the audience, as his students would use. He then set up a problem involving getting from Washington, DC to Baltimore, MD using a combination of driving and taking a train, with two possible routes. Driving time on each route will vary depending on the number of cars on the road. The model is set for the number of participants/students in the group—if there are 28 participants driving on the same route, the driving part of the trip will take 28 minutes. If there are 5 participants driving on the route, it will take 5 minutes. The train trip is static and takes 30 minutes on each route. Using their clickers, participants vote on a route, A or B. Siddiqui then show the histogram of the vote, and participants can change their vote based on the road time component. As participants change votes, the driving time will increase or decrease on each choice. Voting continues until eventually a state of equilibrium is reached and the driving time on the two routes is equal.

Siddiqui then throws in another component. What happens if you add another variable, a new road? Participants can now vote for three options. Ultimately his students will see (as did the participants at the Lunch and Learn) that sometimes a third option can worsen the situation rather than improve it.

In his classes, students work with actual examples taken from New York City, Germany, South Korea, and other places, to examine the factors that went into the design process, and analyze what went wrong. Siddiqui feels that engineers are not necessarily taught to work with real-life situations and this can lead to poor design. Engineers need to understand the factors that impact actual human decision making in order to build successful solutions.

In the discussion period that followed the presentation, Balachandran and Siddiqui agreed that students are motivated by working with real-life problems. Siddiqui noted that his students still had to “slog through” doing the mathematics behind the exercises, but valued understanding both sides.

In discussing how to gauge whether an assignment or project was too big or too small, it was agreed that it is important to scaffold larger projects, build support structures, and allow for flexibility. It was acknowledged that students will struggle with ambiguity. It is important with authentic assignments to be clear that the goal is not so much to find an answer as to go through a process.

Both presenters agreed that setting up these authentic learning experiences—assignments, projects, and courses, can be time consuming and challenging. But, for both, the benefits for students have been substantial and they will continue to explore the possibilities for future classes.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Sources: Lunch and Learn Logo, slides from Balachandran and Siddiqui presentations

Facilitating and Evaluating Student Writing

Over the summer I worked on revising a manual for teaching assistants that we hand out each year at our annual TA Orientation. One of the sections deals with writing intensive courses across disciplines and how TAs can facilitate and evaluate writing assignments. The information, advice, and resources in the manual speak to an audience beyond graduate student teaching assistants. Even seasoned instructors may struggle with teaching writing skills and evaluating written assignments.

View from above and to the right of a woman's hands at a desk writing in a journal next to a lap top computer.Two mistakes that teachers may make are assuming that students in their courses know how to write a scholarly paper and not providing appropriate directions for assignments. These assumptions are likely to guarantee that the resulting student writing will disappoint.

As a quick aside, faculty often complain about the poor quality of student writing, claiming that students today don’t write as well as students in some vaguely imagined past, perhaps when the faculty member was a college freshman. However, the results of an interesting longitudinal study suggest otherwise. A report in JSTOR Daily, Student Writing in the Digital Age by Anne Trubek (October 19, 2016), summarizes the findings of the  2006 study by Andrea A. Lunsford and Karen J. Lunsford, Mistakes Are a Fact of Life: A National Comparative Study. “Lunsford and Lunsford, decided, in reaction to government studies worrying that students’ literacy levels were declining, to crunch the numbers and determine if students were making more errors in the digital age.” Their conclusion? “College students are making mistakes, of course, and they have much to learn about writing. But they are not making more mistakes than did their parents, grandparents, and great-grandparents.” Regardless of your take on the writing of current students, it is worth giving thoughtful consideration to your part in improving your students’ writing.

Good writing comes as a result of practice and it is the role of the instructor to facilitate that practice. Students may arrive at university knowing how to compose a decent five-paragraph essay, but no one has taught them how to write a scholarly paper. They must learn to read critically, summarize what they have read, identify an issue, problem, flaw, or new development that challenges what they have read. They must then construct an argument, back it with evidence (and understand what constitutes acceptable evidence), identify and address counter-arguments, and reach a conclusion. Along the way they should learn how to locate appropriate source materials, assemble a bibliography, and properly cite their sources. As an instructor, you must show them the way.

Students will benefit from having the task of writing a term paper broken into smaller components or assignments. Have students start with researching a topic and creating a bibliography. Librarians are often available to come to your class to instruct students in the art of finding sources and citing them correctly. Next, assign students to producing a summary of the materials they’ve read and identifying the issue they will tackle in their paper. Have them outline their argument. Ask for a draft. Considering using peer review for some of these steps to distribute the burden of commenting and grading. Evaluating other’s work will improve their own. [See the May 29, 2015 Innovative Instructor post Using the Critique Method for Peer Assessment.] And the opportunity exists to have students meet with you in office hours to discuss some of these assignments so that you may provide direct guidance and mentoring. Their writing skills will not develop in a vacuum.

Your guidance is critical to their success. This starts with clear directions for each assignment. For an essay you will be writing a prompt that should specify the topic choices, genre, length, formal requirements (whether outside sources should be used, your expectations on thesis and argument, etc.), and formatting, including margins, font size, spacing, titling, and student identification. Directions for research papers, fiction pieces, technical reports, and other writing assignments should include the elements that you expect to find in student submissions. Do not assume students know what to include or how to format their work.

As part of the direction you give, consider sharing with your students the rubric by which you will evaluate their work. See the June 26, 2014 Innovative Instructor post Sharing Assignment Rubrics with Your Students for more detail. Not sure how to create a rubric? See previous posts: from October 8, 2012 Using a Rubric for Grading Assignments, November 21, 2014 Creating Rubrics (by Louise Pasternak), and June 14, 2017 Quick Tips: Tools for Creating Rubrics. Rubrics will save you time grading, ensure that your grading is equitable, and provide you with a tangible defense against students complaining about their grades.

Giving feedback on writing assignments can be time consuming so focus on what is most important. This means, for example, noting spelling and grammar errors but not fixing them. That should be the student’s job. For a short assignment, writing a few comments in the margins and on the last page may be doable, but for a longer paper consider typing up your comments on a separate page. Remember to start with something positive, then offer a constructive critique.

As well, bring writing into your class in concrete ways. For example, at the beginning of class, have students write for three to five minutes on the topic to be discussed that day, drawing from the assigned readings. Discuss the assigned readings in terms of the authors’ writing skills. Make students’ writing the subject of class activities through peer review. Incorporate contributions to a class blog as part of the course work. Remember, good writing is a result of practice.

Finally, there are some great resources out there to help you help your students improve their writing. Purdue University’s Online Writing Lab—OWL—website is all encompassing with sections for instructors (K-12 and Higher Ed) and students. For a quick start go to the section Non-Purdue College Level Instructors and Students. The University of Michigan Center for Research on Learning and Teaching offers a page on Evaluating Student Writing that includes Designing Rubrics and Grading Standards, Rubric Examples, Written Comments on Student Writing, and tips on managing your time grading writing.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image source: Photo by: Matthew Henry. CC License via Burst.com.

 

 

How Pretesting Can Help Your Students Fail Well

Our previous post When Failure is a Good Thing, looked at an initiative at Smith College called Failing Well, a set of programs that helps student understand that failing can lead to better learning. Today, The Innovative Instructor offers a concrete way in which you can introduce students to that concept.

Piece of lined, loose leaf notebook paper with six multiple choice questions. ABC or D is circled in red for each question.In Why Flunking Exams Is Actually a Good Thing (New York Times Magazine, September 4, 2104), Benedict Carey discusses the benefits of pretesting. He asks us to imagine that on the first day of a course we illicitly got a copy of the final exam. Would it help us to study more effectively and better attend to course readings, lecture materials, and class discussions? Undoubtedly it would. He then asks, “But what if, instead, you took a test on Day 1 that was just as comprehensive as the final but not a replica? You would bomb the thing, for sure. You might not understand a single question. And yet as disorienting as that experience might feel, it would alter how you subsequently tuned into the course itself — and could sharply improve your overall performance.” This is the concept of pretesting.

Carey calls it one of the most exciting developments in the science of learning field. “Across a variety of experiments, psychologists have found that, in some circumstances, wrong answers on a pretest aren’t merely useless guesses. Rather, the attempts themselves change how we think about and store the information contained in the questions. On some kinds of tests, particularly multiple-choice, we benefit from answering incorrectly by, in effect, priming our brain for what’s coming later.” The failure on the pretest is an example of failing well. It sets students up for better learning during the course. A study by U.C.L.A. psychologist Elizabeth Ligon Bjork found that “…pretesting raised performance on final-exam questions by an average of 10 percent compared with a control group.”

Carey cites additional studies of pretesting with the insight that “testing might be the key to studying” and a way of “enriching and altering memory.” More traditional ways of studying do not seem to produce the same depth of learning that frequent testing, including the kind of self-examination that includes recitation, appears to yield. Other studies have shown that immediacy of feedback—getting the correct answers soon after the pretest—led to the greatest learning gains.

Why does pretesting work? There are several theories. First it gives students a preview of the material and helps them “prime the brain” to absorb what is most important. A pretest sets up a hierarchy and adjusts student thinking. Secondly, it exposes false impressions, things students think they know but don’t, by conveying multiple possible answers that they may not have considered as possibilities. Biological factors may come into play as well. Guessing at an answer on a pretest works differently from the memory functions at play in remembering and studying. Guessing embeds an unfamiliar concept into the brain that will be recognized when come across again, particularly if that happens within a short timeframe.

There are limitations. For example, a pretest for an intro course in a foreign language using unfamiliar characters (Russian, Chinese, Arabic) wouldn’t work because students have no “scaffolding of familiar language to work with.” In fact, “[t]he research thus far suggests that prefinals will be much more useful in humanities courses and social-science disciplines in which unfamiliar concepts are at least embedded in language we can parse.”

What can we take away from Carey’s article? Because pretests don’t need to be graded, this can be an easy innovation to implement in your courses. A short multiple choice quiz given before your lecture or class discussion asking questions pertaining to the key points you will cover could make a big difference in your students’ learning of the material. To be sure that they leave with the right information, review the quiz and the correct answers at the end of class.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image source: Pixabay.com