Learning from Student Evaluations

The end of the semester is looming and with it the specter of student course evaluations.There are three face emoticons--yellow neutral face on the left, green smilely face in the center, and red frown face on the right. A hand with the index finger extended is pointing in the direction of the viewer at the yellow smiley face. On the whole, instructors dread dealing with these. Many question the value of institutionally administered evaluations.  Some ignore them, others read them and weep. It’s hard to consider student evaluations and not feel personally affronted by negative comments. You’ve worked hard, done your best, and the students are ungrateful. But maybe there is a better approach. Rather than going on the defensive, how can faculty use these evaluations to improve their teaching, and ultimately, improve student feedback?

Maryellen Weimer, What Can We Learn from End-of-Course Evaluations? (Faculty Focus, March 8th, 2017) suggests developing an “improvement mindset.” “Rather than offering answers, [student evaluations] can be used to raise questions. ‘What am I doing that’s causing students to view my teaching this way?’ Such questions need to lead us to specific, concrete behaviors—things teachers are or aren’t doing.” Weimer suggests looking at the Teaching Practices Inventory developed by Carl Wieman, Professor of Physics at Stanford,  and Sarah Gilbert, Department of Physics and Graduate School of Education at Stanford, as “…a great place to start acquiring this very detailed, nuts and bolts understanding of one’s instructional practice.” Although the inventory was developed with the STEM disciplines in mind, it is easily adaptable for teaching in other fields. An article by Weiman and Gilbert, The Teaching Practices Inventory: A New Tool for Characterizing College and University Teaching in Mathematics and Science, [CBE Life Science Education. 2014 Fall; 13(3):552-69. doi: 10.1187/cbe.14-02-0023], provides additional background and context.

There are other ways to improve teaching and thus evaluations. An underused resource to improve one’s teaching is classroom observation.  Weimer mentions peer observations in her article, but teaching and learning centers such as ours, the Center for Educational Resources, often provide this service for instructors. She also notes the value of formative assessment—seeking timely feedback from students during the course “…when there’s still time to make changes and students feel they have a stake in the action.”

For some solid, common sense advice that’s easy to put into practice, see David D. Perlmutter’s How to Read a Student Evaluation (The Chronicle of Higher Education, October 30, 2011). Perlmutter recommends “scanning for red flags”—the comments/complaints that a number of students make that, by paying attention and correcting, “…can help you head off longer-term troubles.”

He also recommends thinking ahead. Prepare yourself for the evaluations by taking the questions asked into account when designing your courses, and using them as a checklist for your teaching. Perlmutter also suggests how to tease out the qualitative data in open comments, and advises balancing negative comments against positive quantitative ratings. His take is to “…pay attention to student evaluations, try to understand them, and, equally important, communicate that you do not dismiss them as irrelevant.”

Finally, the Vanderbilt Center for Teaching offers a full set of resources, articles, and advice on their website page: Student Evaluations of Teaching. From talking with students, to making sense of evaluation feedback, to research on student evaluations, it’s all here in one convenient location.

Now there is no excuse. Prepare yourself and vow to use student evaluations as a means to improve your teaching skills. Better evaluations will await you in the future.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Scaffolding Part 2: Build Your Students’ Notetaking Skills

A few weeks back The Innovative Instructor posted on teaching your students how to read a journal article, essentially how to provide a scaffold for your students to effectively read scholarly writings. Another place where faculty can provide a framework for students is in the area of notetaking. Students who have grown up using laptops in class may not understand either the value of or the means to taking effective notes. Recognizing that schedules are already jam-packed, I am not suggesting that you spend a lot of class time to cover this. But, taking a few minutes on the first day of class to let students know why using laptops to take notes may not be a good idea, and providing them with some resources for several notetaking methods, may go a long way to improving their learning outcomes.

The image shows a green ball point pen resting on a blank page of graphing paper in a blue covered, open spiral notebook.Why not just let your students use their laptops to take notes? As I wrote in a previous post [May 21, 2014 Summer Reading: Three Articles for Your Consideration] in The Pen is Mightier Than the Keyboard Advantages of Longhand Over Laptop Note Taking  Pam A. Mueller and Daniel M. Oppenheimer [Psychological Science, April 23, 2014, doi: 10.1177/ 0956797614524581], reported on the benefits students gain by taking lecture notes longhand rather than on a laptop. Although using laptops in class is common (and instructors complain about the distractions laptops present), this study “…suggests that even when laptops are used solely to take notes, they may still be impairing learning because their use results in shallower processing.” “In three studies, [the researchers] found that students who took notes on laptops performed worse on conceptual questions than students who took notes longhand.” The authors conclude “…that whereas taking more notes can be beneficial, laptop note takers’ tendency to transcribe lectures verbatim rather than processing information and reframing it in their own words is detrimental to learning.”

As for resources on notetaking, the James Madison University Special Education Program offers the JMU Learning Toolbox, developed with a U.S. Department of Education grant on Steppingstones in Technology Innovation for Students with Disabilities. It features “tools and resources to enable students with learning difficulties to become better learners.” A section on notetaking outlines several different strategies for taking notes, based on common problems students may experience.

Students who have trouble keeping up with the fast pace of a lecture or discussion may benefit from the I SWAM method. If a student needs a better strategy for organizing notetaking, Cornell Notes may be the answer. For taking notes from a recorded talk PP 123 may be helpful. SCROL is beneficial for notetaking while reading course materials. TASSEL is a method offered for those who are easily distracted.

Although TASSEL is designed to help students not to doodle when they get distracted, Sketch Notes encourages drawing as a means to enhance notetaking. The webpage referenced mentions visual learners. In fact, learning styles such as visual, verbal, or kinetic have been debunked (for an overview or the research study see: Learning Styles Debunked: There is No Evidence Supporting Auditory and Visual Learning, Psychologists Say; for the full article see: Learning Styles: Concepts and Evidence by Harold Pashler, Mark McDaniel, Doug Rohrer, and Robert Bjork, Psychological Science in the Public Interest, Volume 9 Number 3 December 2008). Nevertheless, Sketch Notes will likely have the most appeal for those with artistic/creative leanings who like to doodle and draw.

In introducing notetaking strategies to your students, you will want to be sensitive to those who may have learning disabilities that make laptop notetaking a necessity. However, sharing the research on laptop notetaking with students will give them an understanding of why hand notetaking strategies may improve their learning. Providing them with resources to investigate will give them choices based on their needs.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Does Active Learning Disadvantage the Learning Disabled?

Black and white line drawing of the upper torso of a young male in a thinking pose. Two question marks are on either side of his head.Active Learning is a good thing, right? As an instructional designer, I’ve read a great deal of research compiling evidence for teaching practices that promote active learning as a way to engage students and secure better learning outcomes. In my role consulting with faculty on curriculum design, I often suggest ways to increase student participation in their learning that match the learning goals and objectives articulated by the instructor. So it was a surprise to read a dissenting view in a Tomorrow’s Professor post by Fernando Gonzalez, an assistant professor of software engineering at Florida Gulf Coast University, titled For Some, Active Learning Can Be a Nightmare. [Full citation for original publication: Gonzalez, Fernando. “For Some, Active Learning Can Be a Nightmare.” ASEE Prism 26, no. 4 (December 2016): 52.]

To be clear at the outset, this is an opinion piece, based on anecdotal evidence and personal experience. There is no research backing Gonzalez’s claims, at least not yet. The article is short, and I encourage you to read it for yourself. In summary, Gonzalez provides a short overview of active learning, then states that “…[active learning] can be a nightmare for students with learning disabilities (LD). While learning disabled students – including those with dyslexia, dyscalculia, dysgraphia, visual and auditory processing deficits, ADHD, nonverbal learning disabilities, and many others – vary in how they learn and on the type of accommodation they require, a common characteristic found in most LD students is needing more time to assimilate information from a lecture.” This he contends, makes it difficult for the learning disabled student “…who may not be able to learn the material in time to participate in the active learning activity immediately following the lecture or may have problems with the activity itself.” He notes that he has severe dyslexia and states he would not have “survived” an undergraduate education heavily based on active learning, and certainly would not have then been able to go on to get a PhD.

There are weaknesses in Gonzalez’s argument, starting with his construct of active learning as mostly being “…strategies [that] consist of a lecture where the student listens passively, followed by an activity that serves to clarify and reinforce what the student has learned.” There are many active learning strategies, and it is misleading to characterize them in total as being difficult for those with learning disabilities, which also are many and varied.

He cites only one concrete example of a strategy, the minute paper, which, although it can be considered an example of active learning, is typically used to obtain formative assessment from students. These exercises are not typically graded and therefore pose little pressure for students.

That said, I do not want to dismiss Gonzalez’s concerns. I was unable to find any published research on the benefits or disadvantages of active learning strategies for learning disabled students. Indeed, it would be valuable for these students and their instructors to have evidence of teaching and learning strategies that are inclusive. If you are aware of research in this area, please share the information in the comments.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Scaffolding: Teach your students how to read a journal article

Recently I have had conversations with faculty and librarians about students and journal articles, specifically, that students don’t come to college knowing how to find or how to read a journal article. It may seem tedious to have to take time out of your already packed class schedule, but it will be valuable (for you and them) to provide some scaffolding and introduce them to these practices.

A pair of glasses and a highlighter are shown on top of an open text book.Here at Johns Hopkins, our Academic Liaison librarians will be happy to come to your class and discuss with students how to search for and locate appropriate materials for their research. Those at other institutions may have similar resources available. But you may also find it worthwhile to give some guidance on the reading aspect.

The Consortium for Political and Social Research (ICPSR) at the University of Michigan has a great three page guide, How to Read (and Understand) a Social Science Journal Article (pdf) that breaks down the parts of a journal article (e.g., title, abstract, introduction, literature review, etc.) and describes what each is and what it tells the reader. It’s aimed at social science students, but is broad enough to be useful for any discipline. After introducing the parts, the guide describes how to read an article by first determining your purpose, then devising a reading strategy.

The ICPSR guide references an article in Inside Higher Ed, It’s Not Harry Potter (Rob Weir, March 9, 2011), which starts off by asking the question “We tell them, but do we show them how?” referencing reading journal articles. Weir recommends starting by introducing students to the concept of audience and have them consider “…for whom and for what purpose a journal article is written.” He lists things students should consider when reading an article, but emphasizes, as is picked up in the ICPSR article, that having students identify their purpose for reading the article is a critical first step. Strategies such as determining the writer’s method, examining the footnotes to evaluate evidence, and skimming are described.

I usually avoid linking to commercial sites/resources, with the exception of apps and software references, however a blog post from ProfHacker (Chronicle of Higher Education) on another subject [Switching from Evernote to OneNote, part 1 by Amy Cavender, August 11, 2016] alerted me to an article by Michael Hyatt: How to Make Your Non-Fiction Reading More Productive, that I thought was worth citing.

Although Hyatt’s advice may be geared towards those in corporate environments, he offers a succinct guide to reading non-fiction books that will be useful for your students. He suggests starting with reviewing and recording the basic bibliographic information, then summarizing the author’s main premise and argument. “Think of this section like an elevator pitch. If you had to tell someone what the book is about in less than a minute, what would you say?” Then Hyatt advises readers to note the insight they gained before identifying their disagreements with the argument. What was missing from the book? What were the main takeaways? Are they quotes that are notable? Having this kind of template for analyzing a book (or article) will give students a concrete platform for tackling scholarly reading.

If you have tips for scaffolding reading or other assignments for students, please share them in the comments section.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Lunch and Learn: Constructing a Comprehensive Syllabus

Logo for Lunch and Learn program showing the words Lunch and Learn in orange with a fork above and a pen below the lettering. Faculty Conversations on Teaching at the bottom.On Thursday, February 16, the Center for Educational Resources (CER) hosted the third Lunch and Learn—Faculty Conversations on Teaching—for the 2016-1017 academic year. Katie Tifft, Lecturer Biology, and Jane Greco, Associate Teaching Professor Chemistry, shared best practices for creating a comprehensive syllabus.

Tifft and Greco presented as a team, reflecting their commitment to collaboration, and gave an impressive overview of the process they follow. Here are their slides for review. They started by sharing a quote by Gary Gutting “Why Do I Teach?” [New York Times 5/22/2013]: “College education is a proliferation of . . . possibilities: the beauty of mathematical discovery, the thrill of scientific understanding, the fascination of historical narrative, the mystery of theological speculation. We should judge teaching not by the amount of knowledge it passes on, but by the enduring excitement it generates. Knowledge, when it comes, is a later arrival, flaring up, when the time is right, from the sparks good teachers have implanted in their students’ souls.”

This represents an ideal, but in real world practice your experience may differ. One way to ensure that students leave your classroom with the knowledge you hope they will gain is to think about how to construct your course so that the desired learning outcomes align with your pedagogical approaches.

Tifft and Greco noted that standard course planning path is to choose a textbook/readings, produce a syllabus, write or revise lectures and prepare slides, and then create assessments (exams and assignments). This is a teacher-centric approach as it revolves around the content that you as the instructor plan to disseminate.

But what if you wanted to develop a course that was student-centric? Then you might take an approach known as backward design. With backward design you start the course planning process by formulating broad learning goals, then defining specific, measurable learning objectives. To clarify, learning goals express what you want students to get out of the course, while learning objectives detail the specific skills and level of understanding you want students to obtain. Next you design the assessments that will be used to evaluate the students’ mastery of the learning objectives. Finally, you develop the course content and activities and choose supporting texts and readings. This process will help you to create a syllabus that informs the students what you expect them to be able to do at the end of the course, as you will share both the broad learning goals for the course and the learning objectives for each course section on the syllabus.

Tifft and Greco reported that research has shown that the longer and more detailed a syllabus is, the more comfortable students will be, because they can see ahead to what will be coming in the class. They suggest keeping a positive tone, focusing on rewards rather than consequences. They both emphasize collaborative work in their courses, and on the syllabi, which fosters a student-centric environment.

What should the syllabus include? The course schedule in some detail, along with the A sign with an orange background reading "Keep calm and read the syllabus."detailed learning objectives for each unit. The course content will be a major part of your syllabus. Policies for absences and missed work should be included and should be transparent, fair, and set an easily achievable bar by accommodating situations that are bound to occur, such as illness, sports team events, etc. One way to do this is to drop the lowest score if you give multiple quizzes, exams, or homework assignments. Tifft and Greco noted that well thought out and clearly written policies are essential in a large enrollment course, and will help reduce the number of emails from students.

The syllabus should give information about assessments and assignments including due dates, descriptions, the link to learning objectives. Setting the test and assignment dates in stone, so to speak, on the syllabus will help your students know what to expect when. Having a variety of assignments is a good practice as it speaks to the diversity of student learning styles. This isn’t always practical in a large lecture class, but should be considered.

If you are using clickers (classroom polling devices) you will want to include policies for use, credit given for participation, credit for correctness, and contribution to grade. Tifft and Greco do not give credit for correctness as they see that getting something wrong contributes to the student’s learning process.

Grades are a major concern for students at Johns Hopkins; Tifft and Greco said that it is important to be as specific and transparent as possible when describing grading criteria and distribution on the syllabus. Doing so will reduce student complaints and misunderstandings. Some practices to consider in creating a grading scheme include the concept of revision/redemption—giving students a chance to drop a low score or revise a paper. They recommend against grading on a curve to reduce competition and facilitate student collaboration.

Don’t forget to list sources of help for students: office hours, names and contact information for teaching assistants, dates and times for recitations/review sessions, and information about the Learning Den tutoring program or PILOT (peer led team learning) program if applicable.

Finally, Tifft and Greco mentioned the required and recommended statements of policy, such as those on ethics, accommodations for students with disabilities, and copyright compliance. And in closing, they recommend adding a line in your syllabus that reads: “The information on this syllabus is subject to change at any time for any reason.”

Discussion by the faculty in attendance followed. One question asked was “How do you get students to read the syllabus? Should you go over the syllabus in class?” Greco stated that since she is teaching first semester freshman, she spends about 20 minutes on the first day of class going over key points, especially the learning goals and her teaching philosophy. Tifft, who teaches upperclassmen does give a brief summary of key points.

Faculty also shared experiences with grading schemes. Many like the idea of dropping the lowest scores on tests and/or assignments and the concept of redemption, especially when based on how the student has done on other parts of the course work. Some faculty give several section-based exams followed by a comprehensive final. Students who have aced the section exams, are not required to take the final.

The use of extra-credit and make-up work to improve grades was debated. It was agreed that it was important to be transparent in these cases, and to make sure that all students are offered the same opportunities. Greco recommended that faculty not allow students to wait until the end of semester to do make up or extra-credit work as it puts too much burden on you as a grader.

The session ended with Tifft and Greco sharing this cartoon from PhD (Piled Higher and Deeper) by Jorge Cham, something anyone who has ever created a syllabus will relate to.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Image generated by http://www.keepcalm-o-matic.co.uk/

Dealing with Difficult Students

Reading a recent post from Stanford University’s Tomorrow’s Professor on dealing with difficult students made me realize that The Innovative Instructor had not dealt with that topic specifically in four plus years of posting. Time to remedy that situation.

Four students in a lecture hall setting, three female, one male. The male and one female appear bored and inattentive.The excerpt from a book [How to Teach Adults: Plan Your Class. Teach Your Students. Change the World, by Dan Spalding, 2014 Jossey-Bass, San Francisco] is from a chapter on managing a class. The short and concise How to Deal With Difficult Students lists eight pieces of advice, all useful considerations, from “Never attack the student,” to “Listen and validate,” to “Draw a line,” with the advice to “focus on the behavior, not on the student.”

Two articles from the archives of Faculty Focus Higher Ed Teaching Strategies offer different approaches to the problem. In Four Tips for Dealing with Difficult Students, March 26, 2010, Jason Ebbeling and Brian Van Brunt suggest taking a collaborative approach using four principles—Express Empathy, Develop Discrepancy, Avoid Argumentation, and Roll with Resistance—to acknowledge a problem and work with a student to correct it. The second article, Dealing with Difficult Students and Other Classroom Disruptions by Mary Bart, June 4, 2012, suggests a proactive approach. Setting expectations on the first day of class and clearly communicating those expectations both verbally and on the syllabus can set the tone for the learning environment you want to create. Still, the author acknowledges, even when an instructor does everything right, there is a potential for a student to be disruptive. Suggestions are provided for having a conversation with a difficult student from a point of concern for the student’s success.

In searching for material on this topic, I discovered a site that instructors might find useful in general for teaching: 4Faculty.org. Unfortunately, the site is no longer being supported and updated, but there are great resources available in the archive—just be aware that the material may be dated. In most instances, that is not an issue. The Classroom Management guide by Linda Rodriguez is a case in point, offering solutions to specific issues under the heading of dealing with difficult students. These include undermining the instructor’s authority, leaving class frequently, verbal or physical threats, cell phone disruption, monopolizing discussions, and disrespectful behavior among others. A handy PDF version of the suggestions can be found here. Keep scrolling down the page to find more tips on setting classroom atmosphere, managing class time, facilitating connections in the classroom, and helping students to be successful college students.

The Innovative Instructor welcomes your comments and suggestions on dealing with difficult students and other classroom management topics.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Microsoft Clip Art

Flipping a Statistical Analysis Course

I wanted to share my reflections on flipping a course in Fall 2015 with my colleague, Dan Naiman, Professor of Applied Mathematics and Statistics at Johns Hopkins University. The course is 550.111: Statistical Analysis I. Previously, this 4-credit course met four time per week for 50 minutes – three lectures by faculty and one small-group meeting led by a Teaching Assistant (TA).

Text reading flipping the classroom with the classroom upside downStarting in Fall 2015, students watched several short videos (anywhere from 5 to around 20 minutes each) before the week started. Students then met once for a 75-minute lecture with the instructor and twice in small-groups with a TA. During these sessions students solved problems in teams of three with a TA available for help as needed.

In Fall 2016, we amended the format slightly: students met in a large lecture twice a week, on Mondays and Fridays, and met in discussion sections twice a week, on Tuesdays and Thursdays. This was in response to feedback from students indicating that they preferred a bit of additional face-to-face meeting time with the instructor. The Monday-Friday lecture times also made homework submission and certain aspects of course planning (such as exams) easier to handle.

We made this change because we wanted students to spend more time in small groups solving problems and engaged in activities, as opposed to simply listening to a lecture.

What did we learn? I would strongly advise those interested in flipping a class to keep the videos short. They should be about five minutes each. This allows each video to cover a discrete topic, and it’s about as long as students will watch in one session. Recording shorter videos is easier on the instructor as well. The video production took longer than I expected. For each video, Dan and I would first construct a slideshow, and then we would record it using the software program Camtasia. My colleague, Dan, did an excellent job with video production, and we generated significant video content before the start of the semester. I would also advise instructors to complete all video production before the start of the semester; we still had a few videos to produce during the semester, and this was a challenge. I found I was pressed to finish those additional videos in time. We plan to revisit, edit, and potentially add more videos before the next course offering. Specifically, we are considering animations and possible hand-written solutions.

We conducted clicker quizzes at the beginning of each lecture to motivate students to watch the videos. However, based on the video logs, quiz results, and the questions they asked, I found a number of students were not fully prepared. Their questions were on topics covered in the videos. I would estimate that in Fall 2015, until the first exam, a number of students did not pay sufficient attention to the videos. However, after the first exam, students began watching the videos more diligently.

One reason we flipped the course was to restructure class time so that students could spend more time in mentored environments working in small groups solving problems. As it turned out, though, students requested more lecture than the once-weekly format. Students struggled to grasp some concepts from the videos. While students can review these topics multiple times, I believe they sometimes needed an alternative explanation. In a lecture, when students ask questions, I try to respond with a different perspective or explanation. With the flipped model in Fall 2015, students had only one class meeting each week to ask me questions about the homework. The second time we ran the course, in Fall 2016, we had two lectures each week, and I think students appreciated the additional lecture time.

I really enjoy teaching this course. It’s a lot of fun and a great privilege. Many non-majors enroll, and humanities undergrads have shared that this was the first math course they enjoyed and they were impressed with the applicability and universality of statistics. The class typically enrolls about 100 students.  Even with this large number I am able to learn most of their names by the end of the semester when we met three times per week. I did feel, though, that I was not able to get to know students as well when we met once per week. More important, I think the once-weekly lecture deterred students from coming to see me during office hours: I noticed a sharp decrease in the number of students who consulted me during office hours in Fall 2015. In Fall 2016, under the twice-weekly lecture model, I had better office hour attendance and was better able to get to know students.

While we were happy with the increase in the number of lectures, I think it’s important that we not decrease the number of small group meetings. The worksheet activities were important for their learning. Students were not always as enthusiastic about the small group problem solving, but they adjusted to the format and things improved as the semester moved forward. Furthermore, we still found it better than a TA solving demo problems for the class, especially in terms of class engagement and in terms of fostering independent problem-solving.

We used two types of problems in the course. The first required more synthesis-based understanding of previous topics. We began to develop more basic, conceptual worksheets once we saw students were not always able to keep up with the videos.

We did not give students the solutions to the worksheets. We worried that if we provided full solutions, they might be less motivated to work through challenging problems and/or skip discussion section altogether, and participation in section was important. Students did get feedback from the TA when they presented their solutions in class, and we did provide solutions to most assigned homework problems.

Overall, we did not see a dramatic change in student learning. We did not conduct a controlled study of learning gains, but exam scores were not much different from year-to-year. Course evaluations for the one-lecture-per-week format were slightly lower. (Again, the main complaint was that students wanted more time with faculty member in lecture.) Students were happier with the two-lecture-per-week format we implemented in Fall 2016. Therefore, we plan to stick with this format, meeting four times per week so students attend two lectures and two small-group sessions per week. We have also been more explicit about the role of each component of the course – videos, lecture, clicker quizzes, small group meetings – and what students are responsible for completing and when.

Most of all, we were very lucky to experiment with this approach with many terrific TAs—we owe them a real debt of gratitude for their assistance. We gratefully acknowledge support from the Office of the Provost and President for a PILOT grant that assisted us in implementing the flipped course.

 

Avanti Athreya is an Assistant Research Professor in Applied Mathematics and Statistics (AMS) at Johns Hopkins University. Prior to flipping the statistics course, she and Professors Naiman, Fishkind, Torcaso, and Jedynak (all AMS faculty) implemented a case-study based approach to introductory statistics as a part of the JHU Gateway Sciences Initiative. Her research interests are in probability and statistical inference on random graphs.

Dan Naiman has been on the faculty in Applied Mathematics and Statistics since 1982. Upon arrival at JHU, he taught Statistical Analysis I for 3 consecutive years, and has continued to teach the course occasionally, as well as a host of other statistics courses at all levels, since then.

Image Source: CC Macie Hall 2013

 

Using Classroom Simulations as an Active Learning Technique

College educators have many goals for students; we want them to acquire more knowledge and be better critical thinkers, but also to feel empowered and energized about their future contribution to society. Students that are motivated and ambitious are more likely to pursue personal opportunities and inventive ideas. This type of energy and focus also contributes to the problem-solving capacity of society as a whole. Although a positive attitude often comes from within the student or outside the classroom, the structure of learning also has an impact.

For the global environmental politics classroom, the problem of student attitudes is especially acute: students of global environmental governance are particularly prone to negative emotional reactions, including feelings of helplessness and hopelessness, which can engender apathy and cynicism.  Students come to believe that the complexity and depth of problems like climate change make effective action impossible. Students who do not believe a problem can be solved are unlikely to seek solutions to that problem in their post-college careers. Using active learning techniques like Simulations can combat these attitudes, by giving students the opportunity to collectively investigate and tackle barriers to international action.

I designed a Simulation for the last week of my fall 2017 “Politics of the Ocean” class, because I noticed that the students often left class in despair. Solutions to over-fishing, Model United Nations simulation with students sitting at tables with flags of the represented countries.plastic pollution, dead zones, ocean acidification, coral bleaching, and other ocean issues seemed out of reach because of political and economic barriers. The number and complexity of ocean issues seemed overwhelming. And yet, we knew that the United Nations was gearing up to negotiate a new treaty to govern the high seas. This provided me with the opportunity to design a politics Simulation that hewed as close to the real world as possible, where students could practice negotiating a treaty that addressed many of the problems they had learned about in class.

The basic features of the course dictated the options for Simulation design – I had 15 students, and we met twice a week for a total of 2.5 hours. I started by assigning students to polity teams in the week before the Simulation began. I choose countries that have had the most influence on ocean governance historically, and groups that would likely have influence in the upcoming negotiations: The United States, China, Russia, the G77 coalition, Singapore, and NGOs. I asked students to do the assigned readings for the next week – each of which contained a specific proposal for ocean governance – with their team in mind.

The Simulation was divided into two days. On day one, students worked within their teams to answer a series of questions like “Who are the primary ocean interest groups in your country?” “What are your priorities for ocean governance?” and “What treaty design best serves your interests?” Students were instructed to work with their teammates, and to do supplementary in-class research to help flesh out their positions. Some teams had specific questions: the NGOs had to decide which NGOs to represent, and the China team had to decide whether to negotiate with the G77, or on its own. The Singapore team had additional questions about how the negotiations ought to be run, because of Singapore’s historic role as a leader in organizing past Law of the Sea negotiations.

On day two, students entered the classroom to discover groups of tables designated with small flags. Singapore ran the negotiations while I took notes, with some minor interventions. Each team started with an opening statement about their key interests and main concerns, with short rebuttals following. Then Singapore asked each team to submit a list of priority topics, and chose the top four. While the original plan was to address each in turn through speeches and open discussion, the students ended up deciding to address all the issues simultaneously. In the last ten minutes, Singapore collected specific treaty language proposals. Each of six new rules was voted on individually, and those that with a majority of teams affirming became the agreed upon treaty.

I designed this Simulation to achieve attitudinal goals in three ways. First, role playing required students to formulate prescriptions from the descriptions of ocean problems and governance models they had learned about in class. The idea is that practicing advocacy will help students recognize that they have informed opinions about ocean issues, and see themselves as agents of change. Second, the format shows students that complexity is not the same as intractability. The two-day design allows group work to break down the structure of a collective action problem, construct a policy agenda and negotiation strategy, and consider various policy models described in the literature. Third, the negotiations allow students to directly encounter barriers to consensus formation, instead of speculating about everything that could hold up an agreement. Confronting obstacles to agreement this way may illustrate the utility of issue-linkages, and demonstrate that there are coalitions willing to move forward.

I assessed the achievement of attitudinal learning outcomes using a short pre- and post-Simulation survey, which asked students to rate their level of agreement with statements like “All relevant parties can get what they want from the oceans” and “The situation in the high seas is too complicated for effective management.” The survey also asked students to rank the importance of different barriers to an international treaty, like “political will” and “public education.” The final questions were open-ended, and asked students to use one word to describe the situation in the ocean, and also how they feel about it. While the survey results showed a slight improvement in optimism, I was surprised by the fact that students started out more optimistic than I expected.

The biggest mistake I made in the design of this Simulation was asking the Singapore team to take a leadership role by designing the basic structure of the negotiations, and leading the class on day two. Although I chose two students with obvious leadership qualities, they found it difficult to command authority among the teams, and to push for efficiency in negotiations. They also seemed displeased that they had a “special” role, and more interested in participating as a regular team. Most of the students reported wanting to start the Simulation earlier in the semester, so they could have more time getting into the details of constructing a workable solution to collective problems in the ocean.

This type of Simulation is relatively easy to design and implement, and there exists a broad literature relating game design to specific cognitive and attitudinal goals. Even though this Simulation was imperfect, students reported on their course evaluations that they appreciated doing something different, and having the chance to work through obstacles to consensus as a group. And because this type of Simulation can be used with a larger class size (just add more teams), I know that the lessons from this class can be used to improve the Simulation for the future.

Elizabeth Mendenall, PhD candidate, Johns Hopkins University

Elizabeth Mendenhall is a PhD candidate in International Relations. Her dissertation concerns obstacles to effective governance in the global commons, specifically the ocean, atmosphere, and outer space. She will be starting as an assistant professor at the University of Rhode Island in the Fall of 2017.

Image source: Wikimedia Commons

 

 

 

 

 

 

 

Snow Day? How to Keep Your Classes Going Even When Life Doesn’t Cooperate

Dog sled shown in a snow-covered landscape with mountains in the background.Winter here in Baltimore brings the specter of freak snowstorms dropping two feet of the white stuff on our campus and shutting the city down for a week. Missing two or three class sessions can push your course syllabus into the realm of unrecoverable. Even if you live in more tropical climes, there is always a critical conference, a virulent virus, or other unplanned absence-causer lurking. The good news is that with a little thinking ahead, you can keep your classes going virtually whether or not you are present in reality.

Staff in the Center for Educational Resources prepared a handy guide for weather-related emergencies: Options for Continuing Instruction. While the guide is specific to Johns Hopkins tools, resources and applications, it is adaptable to other circumstances. The suggestions will be even easier to implement if you take some time to plan ahead.

Some of the suggestions recommend the use of Blackboard, the JHU learning management system. Readers from outside of Hopkins can substitute your institution’s LMS. Even if you don’t use the LMS regularly it is a good idea to have a course shell ready to go for an emergency situation. At JHU all courses have a Blackboard shell ready to be activated by the instructor. Here is general help with Blackboard if you are a new user or need a refresher.

First and foremost, it is important to have a way to contact all of your students. JHU Faculty can do this through Blackboard or our Student Information System (SIS). In any case, letting your students know how to proceed in an unplanned absence will be critical to your success. It’s also crucial to let your students know your expectations for assignments and other course modifications made during the closure or your absence.

You can share course materials with students using your LMS, or through a file sharing system such as DropBox. JHU faculty have JHBox freely available for their use. Students can submit assignments by email or through the LMS.

Replacing actual time in the lecture hall or classroom can be more challenging, but is doable. For a smaller class or seminar where discussion is the norm, you can conduct asynchronous discussions using a threaded discussion application. Blackboard has one as a built in feature (see here for help setting this up and here for tips on implementation). Voicethread (here for JHU, here for others), which at JHU is integrated with Blackboard, is another option. A wiki application, such as Google Sites, could be adapted for use as an online discussion tool. Teleconferencing is also an option for smaller classes. IT@JH provides instructions on live teleconferencing options. Skype could also be used for live discussion.

 There are applications, such as Adobe Connect (available here for use by JHU faculty) that will allow you to conduct a live, synchronous lecture and record it for students to watch later. Panopto is another JHU resource for recording a video lecture that can be posted to your Blackboard course site for students to watch on their own schedule. If you don’t have access to these applications, it is possible to create a PowerPoint presentation and do a voice recording over the slides to send to your students. Even lower-tech and easier, put your lecture script in the notes section of the slides instead of voice recording.

The purpose of these solutions is to keep your students and course content delivery from falling irretrievably behind. Having a plan in place ahead of time, figuring out the options that will work best for your course, learning how to use the relevant applications, and alerting your students to the possibilities, will save you time and headaches when the snow starts falling.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Lunch and Learn: Team-Based Learning

Logo for Lunch and Learn program showing the words Lunch and Learn in orange with a fork above and a pen below the lettering. Faculty Conversations on Teaching at the bottom.On Friday, December 16, the Center for Educational Resources (CER) hosted the second Lunch and Learn—Faculty Conversations on Teaching, for the 2016-1017 academic year. Eileen Haase, Senior Lecturer in Biomedical Engineering, and Mike Reese, Director, Center for Educational Resources, and Instructor in Sociology, discussed their approaches to team-based learning (TBL).

Eileen Haase teaches a number of core courses in Biomedical Engineering at the Whiting School of Engineering, including Freshmen Modeling and Design, BME Teaching Practicum, Molecules and Cells, and System Bioengineering Lab I and II, as well as being course director for Cell and Tissue Engineering and assisting with System Bioengineering II. She has long been a proponent of team work in the classroom.

In her presentation, Haase focused on the Molecules and Cells course, required for BME majors in the sophomore year, which she co-teaches with Harry Goldberg, Assistant Dean at the School of Medicine, Director of Academic Computing and faculty member, Department of Biomedical Engineering. The slides from Haase’s presentation are available here.

In the first class, Haase has the students do a short exercise that demonstrates the value of teamwork. Then the students take the VARK Questionnaire. VARK stands for Visual Aural Read/Write Kinesthetic and is a guide to learning styles. The questionnaire helps students and instructors by suggesting strategies for teaching and learning that align with these different styles. Haase and Goldberg found that 62% of their students were “multimodal” learners who will benefit from having the same material presented in several modes in order to learn it. In Haase’s class, in addition to group work, students work at the blackboard, use clickers, have access to online materials, participate in think-pair-share exercises, and get some content explained in lecture form.

Team work takes place in sections most FridSlide from Eileen Haase's presentation on Team-based Learning showing a scratch card test.ays. At the start of class, students take an individual, 10 question quiz called the iRAT, Individual Readiness Assurance Test, which consists of multiple-choice questions based on pre-class assigned materials. The students then take the test as a group (gRAT). Haase uses IF-AT scratch cards for these quizzes. Both tests count towards the students’ grades.

To provide evidence for the efficacy of team-based learning, Haase and Goldberg retested students from their course five months after the original final exam (99 of the 137 students enrolled in the course were retested). The data showed that students scored significantly better on the final exam on material that had been taught using team-based learning strategies and on the retest, retained significantly more of the TBL taught material. [See Haase’s presentation slides for details.]

Slide from Mike Reese's presentation on Team-based Learning showing four students doing data collection at a Baltimore neighborhood market.Mike Reese, Director of the Center for Educational Resources and instructor in the Department of Sociology, presented on his experiences with team-based learning in courses that included community-based learning in Baltimore City neighborhoods [presentation slides]. His courses are typically small and discussion oriented. Students read papers on urban issues and, in class, discuss these and develop research methodologies for gathering data in the field. Students are divided into teams, and Reese accompanies each team as they go out into neighborhoods to gather data by talking to people on the street and making observations on their surroundings. The students then do group presentations on their field work and write individual papers. Reese says that team work is hard, but students realize that they could not collect and analyze data in such a short time-frame without a group effort.

Reese noted that learning is a social process. We are social beings, and while many students dislike group projects, they will learn and retain more (as Haase and Goldberg demonstrated). This is not automatic. Instructors need to be thoughtful about structuring team work in their courses. The emotional climate created by the teacher is important. Reese shared a list of things to consider when designing a course that will incorporate team-based learning.

  1. Purpose: Why are you doing it? For Reese, teamwork is a skill that students should acquire, but primarily it serves his learning objectives.  If students are going to conduct a mini-research project in a short amount of time, they need multiple people working collectively to help with data collection and analysis.
  2. Group Size: This depends on the context and the course, but experts agree that having three to five students in a group is best to prevent slacking by team members.
  3. Roles: Reese finds that assigning roles works well as students don’t necessarily come into the course with strong project management skills, and projects typically require a division of labor. It was suggested that assigning roles is essential to the concept of true team-based learning as opposed to group work.
  4. Formation: One key to teamwork success is having the instructor assign students to groups rather than allowing them to self-select. [Research supports this. See Fiechtner, S. B., & Davis, E. A. (1985). Why some groups fail: A survey of students’ experiences with learning groups. The Organizational Behavior Teaching Review, 9(4), 75-88.] In Reese’s experience assigning students to groups helps them to build social capital and relationships at the institution beyond their current group of friends.
  5. Diversity: It is important not to isolate at-risk minorities. See: Heller, P. and Hollabaugh, M. (1992). Teaching problem solving through cooperative grouping. American Journal of Physics, 60 (7), 637-644.
  6. Ice Breakers: The use of ice breakers can help establish healthy team relationships. Have students create a team name, for example, to promote an identity within the group.
  7. Contracts: Having a contract for teamwork is a good idea. In the contract, students agree to support each other and commit to doing their share of the work. Students can create contracts themselves, but it is best if the instructor provides structured questions to guide them.
  8. Persistence: Consider the purpose of having groups and how long they will last. Depending on learning goals, teams may work together over an entire semester, or reform after each course module is completed.
  9. Check-ins: It is important to check in with teams on a regular basis, especially if the team is working together over an entire semester, to make sure that the group hasn’t developed problems and become dysfunctional.
  10. Peer Evaluation: Using peer evaluation keeps a check on the students to ensure that everyone is doing a fair share of the work. The instructor can develop a rubric, or have students work together to create one. Evaluation should be on specific tasks. Ratings should be anonymous (to the students, not the instructor) to ensure honest evaluation, and students should also self-evaluate.

In the discussion that followed the presentation, mentoring of teams and peer assessment were key topics. Several faculty with experience working with team-based learning recommended providing support systems in the form of mentors and or coaches who are assigned to the groups. These could be teaching assistants or undergraduate assistants who have previously taken the course. Resources for team-based learning were mentioned. CATME, “which stands for ‘Comprehensive Assessment of Team Member Effectiveness,’ is a free set of tools designed to help instructors manage group work and team assignments more effectively.”

Doodle was suggested as another tool for scheduling collaborative work. Many are familiar with the Doodle poll concept, but there are also free tools such as Connect Calendars and Meet Me that can be used by students.

An Innovative Instructor print article, Making Group Projects Work by Pam Sheff and Leslie Kendrick, Center for Leadership Education,  August 2012, covers many aspects of successful teamwork.

Another resource of interest is a scholarly article by Barbara Oakley and Richard Felder, Turning Student Groups into Effective Teams [Oakley, B., Felder, R.M., Brent, R., Elhajj, I. Journal of student centered learning, 2004]. “This paper is a guide to the effective design and management of team assignments in a college classroom where little class time is available for instruction on teaming skills. Topics discussed include forming teams, helping them become effective, and using peer ratings to adjust team grades for individual performance. A Frequently Asked Questions section offers suggestions for dealing with several problems that commonly arise with student teams, and forms and handouts are provided to assist in team formation and management.

If you are an instructor on the Homewood campus, staff in the Centerfor Educational Resources will be happy to talk with you about team-based learning and your courses.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Sources: Lunch and Learn logo by Reid Sczerba, presentation slides by Eileen Haase and Mike Reese