Flipping a Statistical Analysis Course

I wanted to share my reflections on flipping a course in Fall 2015 with my colleague, Dan Naiman, Professor of Applied Mathematics and Statistics at Johns Hopkins University. The course is 550.111: Statistical Analysis I. Previously, this 4-credit course met four time per week for 50 minutes – three lectures by faculty and one small-group meeting led by a Teaching Assistant (TA).

Text reading flipping the classroom with the classroom upside downStarting in Fall 2015, students watched several short videos (anywhere from 5 to around 20 minutes each) before the week started. Students then met once for a 75-minute lecture with the instructor and twice in small-groups with a TA. During these sessions students solved problems in teams of three with a TA available for help as needed.

In Fall 2016, we amended the format slightly: students met in a large lecture twice a week, on Mondays and Fridays, and met in discussion sections twice a week, on Tuesdays and Thursdays. This was in response to feedback from students indicating that they preferred a bit of additional face-to-face meeting time with the instructor. The Monday-Friday lecture times also made homework submission and certain aspects of course planning (such as exams) easier to handle.

We made this change because we wanted students to spend more time in small groups solving problems and engaged in activities, as opposed to simply listening to a lecture.

What did we learn? I would strongly advise those interested in flipping a class to keep the videos short. They should be about five minutes each. This allows each video to cover a discrete topic, and it’s about as long as students will watch in one session. Recording shorter videos is easier on the instructor as well. The video production took longer than I expected. For each video, Dan and I would first construct a slideshow, and then we would record it using the software program Camtasia. My colleague, Dan, did an excellent job with video production, and we generated significant video content before the start of the semester. I would also advise instructors to complete all video production before the start of the semester; we still had a few videos to produce during the semester, and this was a challenge. I found I was pressed to finish those additional videos in time. We plan to revisit, edit, and potentially add more videos before the next course offering. Specifically, we are considering animations and possible hand-written solutions.

We conducted clicker quizzes at the beginning of each lecture to motivate students to watch the videos. However, based on the video logs, quiz results, and the questions they asked, I found a number of students were not fully prepared. Their questions were on topics covered in the videos. I would estimate that in Fall 2015, until the first exam, a number of students did not pay sufficient attention to the videos. However, after the first exam, students began watching the videos more diligently.

One reason we flipped the course was to restructure class time so that students could spend more time in mentored environments working in small groups solving problems. As it turned out, though, students requested more lecture than the once-weekly format. Students struggled to grasp some concepts from the videos. While students can review these topics multiple times, I believe they sometimes needed an alternative explanation. In a lecture, when students ask questions, I try to respond with a different perspective or explanation. With the flipped model in Fall 2015, students had only one class meeting each week to ask me questions about the homework. The second time we ran the course, in Fall 2016, we had two lectures each week, and I think students appreciated the additional lecture time.

I really enjoy teaching this course. It’s a lot of fun and a great privilege. Many non-majors enroll, and humanities undergrads have shared that this was the first math course they enjoyed and they were impressed with the applicability and universality of statistics. The class typically enrolls about 100 students.  Even with this large number I am able to learn most of their names by the end of the semester when we met three times per week. I did feel, though, that I was not able to get to know students as well when we met once per week. More important, I think the once-weekly lecture deterred students from coming to see me during office hours: I noticed a sharp decrease in the number of students who consulted me during office hours in Fall 2015. In Fall 2016, under the twice-weekly lecture model, I had better office hour attendance and was better able to get to know students.

While we were happy with the increase in the number of lectures, I think it’s important that we not decrease the number of small group meetings. The worksheet activities were important for their learning. Students were not always as enthusiastic about the small group problem solving, but they adjusted to the format and things improved as the semester moved forward. Furthermore, we still found it better than a TA solving demo problems for the class, especially in terms of class engagement and in terms of fostering independent problem-solving.

We used two types of problems in the course. The first required more synthesis-based understanding of previous topics. We began to develop more basic, conceptual worksheets once we saw students were not always able to keep up with the videos.

We did not give students the solutions to the worksheets. We worried that if we provided full solutions, they might be less motivated to work through challenging problems and/or skip discussion section altogether, and participation in section was important. Students did get feedback from the TA when they presented their solutions in class, and we did provide solutions to most assigned homework problems.

Overall, we did not see a dramatic change in student learning. We did not conduct a controlled study of learning gains, but exam scores were not much different from year-to-year. Course evaluations for the one-lecture-per-week format were slightly lower. (Again, the main complaint was that students wanted more time with faculty member in lecture.) Students were happier with the two-lecture-per-week format we implemented in Fall 2016. Therefore, we plan to stick with this format, meeting four times per week so students attend two lectures and two small-group sessions per week. We have also been more explicit about the role of each component of the course – videos, lecture, clicker quizzes, small group meetings – and what students are responsible for completing and when.

Most of all, we were very lucky to experiment with this approach with many terrific TAs—we owe them a real debt of gratitude for their assistance. We gratefully acknowledge support from the Office of the Provost and President for a PILOT grant that assisted us in implementing the flipped course.


Avanti Athreya is an Assistant Research Professor in Applied Mathematics and Statistics (AMS) at Johns Hopkins University. Prior to flipping the statistics course, she and Professors Naiman, Fishkind, Torcaso, and Jedynak (all AMS faculty) implemented a case-study based approach to introductory statistics as a part of the JHU Gateway Sciences Initiative. Her research interests are in probability and statistical inference on random graphs.

Dan Naiman has been on the faculty in Applied Mathematics and Statistics since 1982. Upon arrival at JHU, he taught Statistical Analysis I for 3 consecutive years, and has continued to teach the course occasionally, as well as a host of other statistics courses at all levels, since then.

Image Source: CC Macie Hall 2013


Using Classroom Simulations as an Active Learning Technique

College educators have many goals for students; we want them to acquire more knowledge and be better critical thinkers, but also to feel empowered and energized about their future contribution to society. Students that are motivated and ambitious are more likely to pursue personal opportunities and inventive ideas. This type of energy and focus also contributes to the problem-solving capacity of society as a whole. Although a positive attitude often comes from within the student or outside the classroom, the structure of learning also has an impact.

For the global environmental politics classroom, the problem of student attitudes is especially acute: students of global environmental governance are particularly prone to negative emotional reactions, including feelings of helplessness and hopelessness, which can engender apathy and cynicism.  Students come to believe that the complexity and depth of problems like climate change make effective action impossible. Students who do not believe a problem can be solved are unlikely to seek solutions to that problem in their post-college careers. Using active learning techniques like Simulations can combat these attitudes, by giving students the opportunity to collectively investigate and tackle barriers to international action.

I designed a Simulation for the last week of my fall 2017 “Politics of the Ocean” class, because I noticed that the students often left class in despair. Solutions to over-fishing, Model United Nations simulation with students sitting at tables with flags of the represented countries.plastic pollution, dead zones, ocean acidification, coral bleaching, and other ocean issues seemed out of reach because of political and economic barriers. The number and complexity of ocean issues seemed overwhelming. And yet, we knew that the United Nations was gearing up to negotiate a new treaty to govern the high seas. This provided me with the opportunity to design a politics Simulation that hewed as close to the real world as possible, where students could practice negotiating a treaty that addressed many of the problems they had learned about in class.

The basic features of the course dictated the options for Simulation design – I had 15 students, and we met twice a week for a total of 2.5 hours. I started by assigning students to polity teams in the week before the Simulation began. I choose countries that have had the most influence on ocean governance historically, and groups that would likely have influence in the upcoming negotiations: The United States, China, Russia, the G77 coalition, Singapore, and NGOs. I asked students to do the assigned readings for the next week – each of which contained a specific proposal for ocean governance – with their team in mind.

The Simulation was divided into two days. On day one, students worked within their teams to answer a series of questions like “Who are the primary ocean interest groups in your country?” “What are your priorities for ocean governance?” and “What treaty design best serves your interests?” Students were instructed to work with their teammates, and to do supplementary in-class research to help flesh out their positions. Some teams had specific questions: the NGOs had to decide which NGOs to represent, and the China team had to decide whether to negotiate with the G77, or on its own. The Singapore team had additional questions about how the negotiations ought to be run, because of Singapore’s historic role as a leader in organizing past Law of the Sea negotiations.

On day two, students entered the classroom to discover groups of tables designated with small flags. Singapore ran the negotiations while I took notes, with some minor interventions. Each team started with an opening statement about their key interests and main concerns, with short rebuttals following. Then Singapore asked each team to submit a list of priority topics, and chose the top four. While the original plan was to address each in turn through speeches and open discussion, the students ended up deciding to address all the issues simultaneously. In the last ten minutes, Singapore collected specific treaty language proposals. Each of six new rules was voted on individually, and those that with a majority of teams affirming became the agreed upon treaty.

I designed this Simulation to achieve attitudinal goals in three ways. First, role playing required students to formulate prescriptions from the descriptions of ocean problems and governance models they had learned about in class. The idea is that practicing advocacy will help students recognize that they have informed opinions about ocean issues, and see themselves as agents of change. Second, the format shows students that complexity is not the same as intractability. The two-day design allows group work to break down the structure of a collective action problem, construct a policy agenda and negotiation strategy, and consider various policy models described in the literature. Third, the negotiations allow students to directly encounter barriers to consensus formation, instead of speculating about everything that could hold up an agreement. Confronting obstacles to agreement this way may illustrate the utility of issue-linkages, and demonstrate that there are coalitions willing to move forward.

I assessed the achievement of attitudinal learning outcomes using a short pre- and post-Simulation survey, which asked students to rate their level of agreement with statements like “All relevant parties can get what they want from the oceans” and “The situation in the high seas is too complicated for effective management.” The survey also asked students to rank the importance of different barriers to an international treaty, like “political will” and “public education.” The final questions were open-ended, and asked students to use one word to describe the situation in the ocean, and also how they feel about it. While the survey results showed a slight improvement in optimism, I was surprised by the fact that students started out more optimistic than I expected.

The biggest mistake I made in the design of this Simulation was asking the Singapore team to take a leadership role by designing the basic structure of the negotiations, and leading the class on day two. Although I chose two students with obvious leadership qualities, they found it difficult to command authority among the teams, and to push for efficiency in negotiations. They also seemed displeased that they had a “special” role, and more interested in participating as a regular team. Most of the students reported wanting to start the Simulation earlier in the semester, so they could have more time getting into the details of constructing a workable solution to collective problems in the ocean.

This type of Simulation is relatively easy to design and implement, and there exists a broad literature relating game design to specific cognitive and attitudinal goals. Even though this Simulation was imperfect, students reported on their course evaluations that they appreciated doing something different, and having the chance to work through obstacles to consensus as a group. And because this type of Simulation can be used with a larger class size (just add more teams), I know that the lessons from this class can be used to improve the Simulation for the future.

Elizabeth Mendenall, PhD candidate, Johns Hopkins University

Elizabeth Mendenhall is a PhD candidate in International Relations. Her dissertation concerns obstacles to effective governance in the global commons, specifically the ocean, atmosphere, and outer space. She will be starting as an assistant professor at the University of Rhode Island in the Fall of 2017.

Image source: Wikimedia Commons








Snow Day? How to Keep Your Classes Going Even When Life Doesn’t Cooperate

Dog sled shown in a snow-covered landscape with mountains in the background.Winter here in Baltimore brings the specter of freak snowstorms dropping two feet of the white stuff on our campus and shutting the city down for a week. Missing two or three class sessions can push your course syllabus into the realm of unrecoverable. Even if you live in more tropical climes, there is always a critical conference, a virulent virus, or other unplanned absence-causer lurking. The good news is that with a little thinking ahead, you can keep your classes going virtually whether or not you are present in reality.

Staff in the Center for Educational Resources prepared a handy guide for weather-related emergencies: Options for Continuing Instruction. While the guide is specific to Johns Hopkins tools, resources and applications, it is adaptable to other circumstances. The suggestions will be even easier to implement if you take some time to plan ahead.

Some of the suggestions recommend the use of Blackboard, the JHU learning management system. Readers from outside of Hopkins can substitute your institution’s LMS. Even if you don’t use the LMS regularly it is a good idea to have a course shell ready to go for an emergency situation. At JHU all courses have a Blackboard shell ready to be activated by the instructor. Here is general help with Blackboard if you are a new user or need a refresher.

First and foremost, it is important to have a way to contact all of your students. JHU Faculty can do this through Blackboard or our Student Information System (SIS). In any case, letting your students know how to proceed in an unplanned absence will be critical to your success. It’s also crucial to let your students know your expectations for assignments and other course modifications made during the closure or your absence.

You can share course materials with students using your LMS, or through a file sharing system such as DropBox. JHU faculty have JHBox freely available for their use. Students can submit assignments by email or through the LMS.

Replacing actual time in the lecture hall or classroom can be more challenging, but is doable. For a smaller class or seminar where discussion is the norm, you can conduct asynchronous discussions using a threaded discussion application. Blackboard has one as a built in feature (see here for help setting this up and here for tips on implementation). Voicethread (here for JHU, here for others), which at JHU is integrated with Blackboard, is another option. A wiki application, such as Google Sites, could be adapted for use as an online discussion tool. Teleconferencing is also an option for smaller classes. IT@JH provides instructions on live teleconferencing options. Skype could also be used for live discussion.

 There are applications, such as Adobe Connect (available here for use by JHU faculty) that will allow you to conduct a live, synchronous lecture and record it for students to watch later. Panopto is another JHU resource for recording a video lecture that can be posted to your Blackboard course site for students to watch on their own schedule. If you don’t have access to these applications, it is possible to create a PowerPoint presentation and do a voice recording over the slides to send to your students. Even lower-tech and easier, put your lecture script in the notes section of the slides instead of voice recording.

The purpose of these solutions is to keep your students and course content delivery from falling irretrievably behind. Having a plan in place ahead of time, figuring out the options that will work best for your course, learning how to use the relevant applications, and alerting your students to the possibilities, will save you time and headaches when the snow starts falling.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com