Tag Archives: Mike Reese

ChatGPT: A Brief Introduction and Considerations for Academic Integrity

Posted on by
Reply

I’ve been reading about the potential impact of artificial intelligence (AI) on teaching and learning for some time. A close family friend gave me a book entitled In the Mind of the Machine by Ken Warwick in 1998. The Education Horizon Report Advisory Committee, of which I was a member, first listed artificial intelligence as an emerging technology likely to have an impact on learning, teaching, and creative inquiry in education in 2017. November 2022 brought the long-anticipated arrival of ChatGPT beta with accompanying media attention.

What is OpenAI and ChatGPT?

OpenAI is an artificial intelligence research lab. Open AI developed a chatbot called ChatGPT (GPT = Generative Pre-trained Transformer) and an image generation application DALL-E 2. ChatGPT is a natural language processing model that is trained on hundreds of billions of documents and websites. Users can interact by asking it questions or submitting statements to which it can generate responses.  For example, here is ChatGPT answering a question about itself:

Prompt: What is ChatGPT?
“ChatGPT is a large language model developed by OpenAI. It is trained on a diverse range of internet text and is able to generate human-like text in response to various prompts. The model can be fine-tuned for various natural language processing tasks such as language translation, question answering, and conversation.”

While ChatGPT received most of the media attention in winter 2022-23, there are other chatbots that exist like Jasper and Chincilla.

What are the main concerns?

The main concern for instructors is students asking OpenAI applications to complete assignments for them. This includes writing essays or research papers along with coding assignments for which ChatGPT is trained. Students can also ask ChatGPT to answer test questions.

Things to Consider

While the capabilities of artificial intelligence applications will continue to evolve, there are currently some limitations. For example, current models do not include articles behind paywalls (e.g., subscription journals). This makes it harder for students to generate essays based on peer-reviewed research.  While the models are trained on a large number of documents, the applications’ responses to specific, focused inquiries tend to be vague.  My colleagues and I asked ChatGPT to write a strategic plan for the Center for Teaching Excellence and Innovation. It suggested relevant ideas, but it was generic and too broad to be useful. That said, we could have used it as a starting point for brainstorming a draft.

Some applications, like Turnitin, are claiming they can detect if students used ChatGPT, but like any technology, these applications are not perfect and students can work around them (e.g., editing the essay produced to make it closer to their own writing style).

 Academic Integrity

Use of OpenAI applications can fall under academic integrity policies like plagiarism, but the gray zone between clearly plagiarized work and an academic support tool is large. For example, most instructors would consider it plagiarism for students to ask ChatGPT to write a paper based on a writing prompt from class. But is it OK for students to ask ChatGPT for a summary of research on a topic, which they then use to generate a bibliography as the basis for a research paper they write?  Instructors should learn more about how ChatGPT and other AI technologies work so they can inform students what is considered appropriate use of AI technologies and what is not. Here are additional strategies to consider to help you and your students navigate this new territory:

  • Scaffold the activity by asking students to turn in an outline and iterative drafts that address comments and feedback from the instructor or teaching assistants. This requires students to show progression in a way that is difficult for tools like ChatGPT to produce.
  • Ask students to write papers using a shared Microsoft document through One Drive so you can see the version history.
  • Use writing prompts that are more specific or require students to cite specific texts.
  • Use AI tools to teach students. For example, generate essays in ChatGPT and have students critique them.
  • Discuss with students what is considered acceptable use of AI technologies (e.g., generating a summary of a field) and what is not (e.g., responding to a specific assignment prompt).

A colleague also commented that as we engage with ChatGPT and other AI technologies, we are feeding it data it can use to improve its models. They own the submissions as part of the terms of agreement when accounts are created. Explain to students that they may be giving over their intellectual property if they are using these tools.  If they submit your tests for ChatGPT to answer, they may be violating your intellectual property rights.

Where to Learn More

 Here are some resources to learn more about AI technologies:

We are all orienting ourselves to this new technology and best practices are evolving. The CTEI will continue to share more information and host discussions over the semester.

Mike Reese
Mike Reese is Associate Dean of the Center for Teaching Excellence and Innovation and associate teaching professor in Sociology.

Image Source: OpenAI Logo, Pixabay

A Faculty Follow-up Discussion: Re-engaging Students for the Fall Semester

Posted on by
Reply

On Tuesday, November 8th, the Center for Teaching Excellence and Innovation (CTEI) hosted a discussion on re-engaging students for the fall semester. At faculty request, this discussion was a continuation of one initially held in August, when participants explored the challenges they faced with the return to in-person teaching in Spring semester 2022. During that session, faculty offered potential ways to address disengagement in a student population who reported high levels of “stress, fatigue, and anxiety” in a post-pandemic world.male student staring at his computer This phenomenon has been noted in many media outlets, including The Chronicle of Higher Education, which recently hosted a webinar on addressing student disengagement and summarized it in a follow-up article. Mike Reese, Associate Dean and Director of the CTEI, moderated the conversation.  

The session kicked-off with instructors offering their general sense about how student engagement in their Fall courses compared to their Spring courses. The overall assessment was that problems remained, though there were some bright spots:  

  • One instructor noted that attendance in his course’s Friday session, led by teaching assistants, was down almost 50% in the recent week.  
  • Another noted that Fall was “a little bit” better than Spring, when she was still teaching online via Zoom, but she continued to observe a lot of “struggle” among her students, exacerbated by a lack of knowledge of how to address it.  
  • One participant, who regularly polled his students on their overall well-being on a scale from one to five with five being the highest score, said he was seeing a lot of ones and twos among his students. However, he started this practice during the pandemic so he didn’t have any pre-pandemic data to baseline the response.  
  • A fourth participant had observed that her students’ behavior was better, but they also had large gaps in their subject-matter knowledge due to the instructional disruptions incurred by the pandemic. 

Time management issues quickly became the dominant topic when one faculty member pointed out that this was a particular problem for his students. Other participants also offered examples of students struggling with time management; one faculty member said that she had received a lot of requests for extensions from students who admitted these were due to poor time management, and another said that she observed an all-senior class – usually a population with a good sense of time management –also contending with this issue.group of students socializing The reason for this, attendees speculated, may have to do with the full return to on-site courses and residential campus life. Students may be excited to dive back into campus life, trying to take advantage of opportunities, like lab-based research, not available during the pandemic, and becoming over-committed as a result. Another reason offered was that the time management skills needed to negotiate pandemic life and instruction needed to be re-adjusted for more typical university life.   

The post-pandemic gap in content-specific knowledge, particularly in the STEM disciplines, has prompted some academic programs to start looking at ways to make changes to their large introductory or gateway courses. One participant said her program was looking to make data-based adjustments informed by placement tests, in-person attendance at office hours, and data from Canvas classrooms and learning-support software, such as ALEKS. 

As a group, the participants generated several useful ideas to enhance engagement in both large lecture-style courses and smaller seminar courses:  

  • Increasing structure for small-group discussions in large classrooms: One instructor had added question prompts and a pre-identified spokesperson to her small-group break-out discussions to increase student focus, participation, and output during these sessions.  
  • Flipping one class meeting a week to start homework: Another instructor had flipped one class meeting a week to provide students with a pre-determined timeslot in which to start their homework each week and receive real-time instructional feedback. This helped students with time management and on-time completion of the homework.  
  • Requiring a one-to-one meeting outside class: An attendee required that seminar students meet with him one-on-one at least once outside of class, which helped build relationships and comfort with class participation.  
  • Requiring student socialization outside class: A participant volunteered onegroup of students smiling approach that she heard about via the Chronicle Webinar: to require that students meet and socialize outside of class twice a month to work on “conversation fundamentals” – how to have a balanced conversation, how to use open-ended questions – to build “social sophistication and stamina” in in-person environments post-pandemic. 
  • Mid-semester surveys: Two instructors distributed mid-semester surveys to students that specifically targeted issues of classroom engagement, and one queried participants about their time-on-task for assignments and activities. Though survey participation was low in one course, both instructors were reviewing and integrating appropriate feedback.  
  • Panels of former students: One attendee noted that he had invited a panel of former students to talk about their experiences in the class and what contributed to their success. The credibility of the speakers and the authenticity of the guidance resonated with the current students.  
  • Strategic use of Learning Assistants or Course Assistants: Some instructors in large or introductory courses used Learning Assistants or Course Assistants – undergraduate students successful in the subject area who are trained to provide in-class instructional support – to scale up instructional reach and feedback. These assistants had been particularly crucial in courses that needed more hands-on instructional support, structure, and feedback.    

Many instructors found themselves structuring tasks and activities for students that, pre-pandemic, may not have required direct guidance and direction. Given this need, the importance of student meta-cognition – knowing how to learn something – was raised, which resulted in the following suggestions:       

  • Using learning science data to persuade students: One participant noted that her students were very responsive to research-based arguments. When she offered students evidence-based examples of effective ways to learn (she cited  The Learning Scientists blog as a good source of information), they responded affirmatively to these suggestions. Leveraging learning science research when suggesting better ways to study – retain, recall, and synthesize content – might be one way to help bolster meta-cognition.  
  • Building in self-reflection on effective learning approaches: An attendee recommended integrating opportunities for students to self-reflect on the usefulness of teaching interventions, such as the one-course-meeting-a-week flipped classroom for starting homework. Such reflection on why a certain approach worked (in this case, in-class time dedicated to starting homework with in-person instructional feedback) may help students build (or re-build) their meta-cognitive muscles.  

The conversation turned to tools that could support both targeted in-class instruction and meta-cognition skill development. Brian Cole, Associate Director of the CTEI, said that he had been investigating different technologies that would enable real-time assessment of content comprehension and upvoting of particularly confusing content areas. Melo Yap, the new Sr. Educational Research Consultant at the CTEI, volunteered Kahoot as a tool that could offer such flexibility. 

 A faculty member suggested developing a toolkit with proven meta-cognitive strategies that could be inserted into the Canvas sections of each course. Instructors and students could access this toolkit on-demand and integrate into it their course design for both “just-in-time” support (e.g., before a high-stakes test) and more long-term development. The CTEI offered to collect any already-available guidance to help students learn more effectively in an effort to start collating this information in one place.  

Caroline Egan
Caroline Egan is a Project Manager in the Center for Teaching Excellence and Innovation, supporting instructional training and development for Hopkins faculty, graduate students, post-doctoral fellows, and staff.

Mike Reese
Mike Reese is Associate Dean of the Center for Teaching Excellence and Innovation and associate teaching professor in Sociology.

Image Source: Unsplash

Faculty Discussion: Re-engaging Students for the Fall Semester

Posted on by
Reply

The return to in-person teaching last year brought with it a high degree of uncertainty for students and faculty. Professors reported that stress, fatigue, and anxiety contributed to higher levels of student disengagement, disconnection, and languishing than in pre-pandemic courses. college students in lecture hallFaculty at other schools reported similar trends with several articles and essays published in the NY Times and Chronicle of Higher Education over the past several months. At faculty’s request, the Center for Teaching Excellence and Innovation (CTEI) hosted a discussion for instructors to share their experiences and brainstorm solutions for Fall 2022 and beyond. Over thirty instructors participated. Brian Cole, Associate Director of the CTEI, moderated the discussion.

Faculty attendees reported that students seemed to struggle transitioning from online classes to in-person classes last spring. One person shared that colleagues at other institutions were reporting the same concerns. The behaviors described included students:

  • expressing concerns about being prepared for high-stakes exams
  • regularly being distracted by devices during class
  • skipping class more frequently than before the pandemic
  • requesting more mental health accommodations
  • acting more aggressively toward  instructors (One instructor said this behavior is more likely seen by instructors of certain races and gender.)

Participants felt these behaviors could be traced to several sources. The pandemic was traumatic for many students suffering from social isolation and the stress of living under the threat of COVID. Faculty also felt some students were not as prepared for their classes because pre-requisite college classes or high school courses were online and employed pass-fail grading schemas. Other instructors reported students feeling stressed about other global events (e.g., political polarization, Ukraine war).  While the pandemic may be fading or becoming normalized, faculty shared that other stressors are constants that continue to weigh on students.

This led to a discussion about when it was appropriate for faculty to engage students about their stress and anxiety over global crises. Instructors teaching in the social sciences can more easily integrate discussions of current events into their curriculum. Is it appropriate for faculty in science and engineering to dedicate class time to social stressors that affect students? Most participants said yes. Several shared they provide opportunities for students to talk about current events that may impact them and ask students openly, “How are you doing? What’s going on with your peers?” to give them space to talk. stressed out male student looking at his laptop Several faculty participants asked for advice on how to set boundaries on the amount of time to dedicate to these discussions during class and what types of topics to discuss. Those that dedicate class time to talking about global issues or student mental health concerns felt it was appropriate to occasionally dedicate 15 minutes of class time to connect with students and build relationships with them. As for topics, instructors should choose topics they are comfortable discussing.

Another source of stress mentioned was the number of hours students are working. One professor shared he has seen an increase in students trying to work full time while attending school. It’s not a large number, but definitely an increase that was likely precipitated by the flexible schedules during COVID when classes were more likely to be asynchronous. Another instructor shared that she observed an increase in students working part-time jobs during COVID. She felt it was important to be more explicit with students about expectations for deadlines and the time it takes to complete assignments to help students balance working outside of class. However, many participants felt it was not appropriate for students to work full-time. Another instructor reported an increase in students continuing summer internships into the academic year. Students see this as a pathway to a job after graduation and are motivated to continue working. This may be a good opportunity, but again, faculty are concerned students are overcommitted.

Some faculty record their lectures for students to review later, while others questioned if this would result in students skipping class. Several faculty responded with strategies to encourage students to attend. These included using activities that motivate students to show up and awarding points for participating in those activities.  One instructor said he records the videos, but shares them with students only upon request. If a student is regularly requesting them and not showing up for class, he engages them to learn more about why they are skipping to address any issues. He shares all of the videos with students the week before the exam. Another instructor said she has recorded her lectures for 15 years, and uses activities to encourage student attendance and engagement with each other. She stresses, “I’m not the only person in the room. You also learn from the other students when you are here.”

Another instructor asked if recording the class discourages students from asking questions. Two instructors responded it did not affect behavior in their classes, but that could be because they teach large courses so students are generally less likely to ask questions during class meetings.

While the pandemic was stressful for everyone, faculty reported it provided opportunities to experiment with new teaching strategies. Many are now trying to evaluate which changes to keep or drop. One instructor said their decision is based on the time required to implement the strategy and its impact on student learning.students doing an activity with post-it notes on a whiteboard Another instructor said she offered more low-stakes assessments while teaching online. She felt this helped reduce the stress of high-stakes assessments, but she is now considering if she is requiring too much of students. The value of offering more low-stakes assessments is that students get more regular feedback throughout the semester.

Another instructor is hearing from students that instructors are assigning more applied learning assignments that leverage technology (e.g., arcGIS) which require significantly more time to complete. While it’s good to have interdisciplinary projects and collaborate with other groups, such as data services at the university, we need to remember that this also requires students to work with different groups and learn additional skills, all of which take time.

Overall, faculty felt the goal is not to penalize disengagement, but to encourage students to engage. Faculty participants shared additional strategies to address student disengagement which are summarized below:

  • Eliminate online tools for the course that are not critical.
  • Offer asynchronous work days. This is not an off day, but a chance to work on class assignments.  One instructor shared that students really liked having a day to work on their projects.
  • Clearly communicate expectations for class (e.g., assignment deadlines, pre-class work expectations) on a regular basis and in multiple modalities (e.g., verbally in class, on Canvas, via class email).
  • Build in flexibility to assignment deadlines. One instructor allows students to skip up to two assignments, and their grade is replaced by the average of the existing assignments.
  • Another instructor shared a similar approach: she does not permit any extensions on homework even if a student is sick, however, she replaces missing grades with the average of the other assignments if the student presents a viable excuse for missing the deadline. The instructor uses this approach because it allows her to share the homework solutions immediately after the deadline while students are still intellectually engaged with the assignment.
  • Another instructor builds in quizzes related to the homework to encourage students to look at it early. For example, the day after homework is posted, students are presented with a quiz asking them to describe what the homework is asking them to do. They don’t need to solve it, but it motivates them to look at it early instead of waiting until the last day when they will have less time to get help.
  • Consider using the discussion board in Canvas or encourage students to attend office hours if you don’t want to dedicate class time to check in with them.
  • Share wellness resources from JHU including the Student Well-being blog about talking with students about current events: https://wellbeing.jhu.edu/blog/. They also provide a page for dealing with more acute issues when students are in distress: https://wellbeing.jhu.edu/resources/faculty-staff/
  • Consider limiting the use of devices (phones, laptops, etc.) to minimize distractions during class.
  • Share mental health resources with your students including Mental Telehealth (free counseling via video chat), Calm app (great for sleep, focus, mindfulness), A Place to Talk (peer listening), Stress and Depression Questionnaire (10minute confidential assessment with feedback w/in 48 hours from a Hopkins clinician).
  • Share how you manage your stress to demonstrate that students are not the only ones dealing with these issues. One participant said, “We might feel it, but if we don’t say it, they don’t know it. If you talk about it, then you open up a space for students to talk about their own stressors.”
  • Consult inclusive teaching practices and Hopkins Universal Design for Learning resources which can help instructors build flexibility into their teaching. One instructor said that being flexible includes building in processing time to help students prepare for their assignments. This includes emotional processing time on difficult topics.

female student smiling in classroom with other studentsClare Lochary from the Office of Student Health and Wellbeing shared that the Counseling Center sees two clear spikes in incoming clients each semester. The first is six weeks into the semester when students begin to realize they aren’t doing as well as they want in their courses. The second  is during the last two weeks of the semester as they prepare for final papers and exams. Faculty should be aware of these cycles and pay special attention to student behavior  so they can refer students to help resources or address concerns about academic performance.

Mike Reese
Mike Reese is Associate Dean of the Center for Teaching Excellence and Innovation and associate teaching professor in Sociology.

Image Source: Unsplash

Lunch and Learn: Evaluating Teaching Effectiveness

Posted on by
2

This post summarizes recent Lunch and Learn discussions among Homewood faculty about methods for evaluating teaching effectiveness. This discussion supported the work of the Provost’s ad hoc Committee on Teaching Evaluations. Provost Kumar established this committee in response to the Second Commission on Undergraduate Education report, which included a recommendation to establish a new system for the assessment of teaching and student mentoring by faculty.  This was the first of multiple conversations the committee will hold with faculty, graduate students, and undergraduates.

Fifty faculty joined one of two discussions (February 16th and 22nd) moderated by Vice Deans of Undergraduate Education, Michael Falk and Erin Rowe, along with Mike Reese of the Center for Teaching Excellence and Innovation (CTEI).

The Vice Deans reviewed principles and objectives drafted by the committee to collect faculty feedback and suggestions for improvement. Attendees then discussed methods of improving how we evaluate teaching based on these principles and objectives that go beyond the current system of teaching evaluations.  The following summarizes some of the attendees’ comments.

  • Teaching Evaluations serve multiple purposes: students use them to choose courses, faculty use them to improve their teaching, Homewood Academic Council uses them in promotion and tenure decisions, and schools use them for program assessment and accreditation. One mechanism – course evaluations (or student evaluations of teaching) – should not serve all these purposes.
  • Using multiple methods of evaluating teaching (peer evaluation, review of course materials, etc.) and not just traditional course evaluations, will minimize student bias against underrepresented minorities.
  • An attendee shared that when she worked at a more teaching-focused college, it included a committee of peer evaluators – faculty trained to provide feedback. The instructor would meet with the peer evaluator before class to discuss lesson plans and then debrief after the observation. Junior faculty were reviewed more frequently than senior faculty. They tried to find a peer match based on discipline. This review was used as a formative assessment and also summative assessment when someone came up for promotion.
  • Another professor shared that at West Point, instructors attended formal training on how to teach. Senior faculty came in three times during the semester to observe new instructors in class with a defined rubric that was shared beforehand. As for course evaluations, only the instructor saw them; the department chair did not. Student comments did not play any role in promotion.
  • With more faculty recording their classes, peer evaluators could review those recordings and provide feedback on those videos. This is apparently done at the Harvard Business School.
  • Someone raised the question, what is considered “quality teaching?” and suggested there must be some standard.  We need to consider how much we weight the entertainment value of sitting in class or comfort level of students as opposed to being inspired to pursue a career or digging in deeper [learning more about topic].  Who decides that focus for course evals and how to do it? Another person asked the individual who raised the question for his thoughts on this question. He responded, “In engineering, every class must have an objective and we need to demonstrate we are collecting data to show we are meeting it. Another is to have an expert – maybe in sociology or psychology – to write a question that measures if the class in interesting or stimulating to be in.”
  • One instructor raised the question of who are the experts to conduct evaluations. Attendees mentioned instructional design staff at the Center for Teaching Excellence and Innovation, but also felt that discipline-based education experts were needed. Teaching faculty familiar with discipline-specific teaching strategies (e.g., math, engineering, humanities seminars) should also be considered.
  • For evaluation of teaching effectiveness, the most important thing is measuring what the students have learned and, ideally, retained over long periods. We need concept inventories and tests of student knowledge beyond the end of a class, possibly in future semesters.  Peer evaluations could be part of the process of helping instructors improve, but they don’t really measure learning.
  • An instructor shared, “At my previous institution, we assessed teachers through narratives describing changes they made [in their course] based on new studies that have been published in their areas, in addition to participation in teaching workshops/conferences, and adoption of new practices. I would also suggest we evaluate teaching rigor in some of the same ways we evaluate scientific rigor. Look at whether faculty make their resources open and accessible, use OER, etc.”
  • One professor suggested faculty share with students the purpose of course evaluations and how they will be used. It may discourage complaining. “I tell them I read their comments and my boss reads them. I’ve seen biased comments decline since I shared that with students.”
  • Another instructor said the dean’s offices could provide a script for instructors to read about how course evaluations are used and the importance of students civilly communicating feedback.
  • Course evaluations should be filtered for racist and misogynistic comments so faculty are not subject to them.
  • Course evaluations could include a checklist of comments so student feedback is more specific. Students would select the statements that are relevant to their instructor in addition to entering open comments.
  • It would make the surveys longer, but someone suggested every comment students make should be required to include at least one specific example as evidence.
  • One instructor asked, “How do we tease apart teaching effectiveness so they focus on learning and not grading?”
  • Is it possible to ask students to reflect on gateway or core classes sometime in the future to identify how the course provided foundational skills for future courses or co-curricular activities (e.g., internships)? Someone added, “This week I had a senior tell me, ‘I didn’t realize that those concepts really would come up over and over in my other classes, but they did!’” Another attendee shared, “They also report back 3+ years after graduating to say that they use something they learned in class that during the time they thought would be useless.”
  • Perhaps it would also be helpful to leverage our alumni network as one way to capture the enduring effects of learning from various classes.
  • Another instructor mentioned that it can also help to ask students for feedback during the semester so you can adjust instructional methods.
  • Whatever system is developed should not place undue administrative burden on faculty who are already taking on more administrative burdens.
  • An instructor remarked that once a change is implemented, faculty will need training on the new system of evaluation. It will also help encourage professors to be open to this process. Support for instructors assessing the results of evaluations will be critical.

One faculty member shared two books to inform the committee’s future work:

If you have questions or comments about the teaching evaluation process, feel free to email Mike Reese.

Mike Reese, Associate Dean and Director, CTEI
Mike Reese is Associate Dean of University Libraries and Director of the Center for Teaching Excellence and Innovation. He has a PhD from the Department of Sociology at Johns Hopkins University.

Image Source: Lunch and Learn Logo, Unsplash

 

Lunch and Learn: Inclusive Pedagogy

Posted on by
Reply

On Tuesday, October 19, 2021, the Center for Educational Resources (CER) hosted a virtual Lunch and Learn: Inclusive Pedagogy. Karen Fleming, Professor in Biophysics, and Mike Reese, Associate Teaching Professor in Sociology and CER director, each presented strategies that are important to them in helping to make their classes more inclusive:

  • Recognize that everyone comes from a different place with different experiences. Fleming mentioned The Privileged Poor, a book by Anthony Jack, that addresses the struggles faced by less privileged students after being admitted to elite universities. She explained that taking students’ backgrounds into consideration and embracing differences is vital to their success.
  • Try to instill a growth mindset. In her teaching, Fleming acknowledges to her students that the work is difficult, it can be a struggle at times, and it’s ok if you don’t get it right the first time – this is all part of learning. She tells her students that everyone has unlimited potential and encourages them to keep practicing and they will come away with new skills. Fleming stressed the importance of trying to get students to internalize this way of thinking and offers her students a great deal of positive reinforcement throughout the semester.
  • Show the humanity of science and diversify materials. Fleming described how she makes an effort to showcase diverse scientists who are working in the field – people of color, women, etc. Students need to see role models and images of people that look like them. Like Fleming, Reese explained how he also makes an effort to display photos of diverse experts in the fields as he discusses key findings or theories in sociology..
  • Learn students’ names. Reese acknowledged this can be a challenge if the class is large. He suggested instructors print out the student photo roster from SIS and/or bring tented name cards to the first class for students to display on their desks (if teaching in person) to help learn names. Reese stated that students are more engaged and come to office hours more often when he makes an effort to learn their names.
  • Use non-competitive grading strategies. Reese noted that this was one of the recommendations in JHU’s Second Commission on Undergraduate Education (CUE2) report. Using straight grading, rather than curved, is one example. Another example is to add a standard number of points to every students final score if they overall average is lower than expected, which might suggest the test was more challenging than intended.
  • Conduct a mid-semester survey. Reese described how he administers a brief mid-semester survey that is anonymous. He tells students ahead of time that he may not be able to address every concern, but will do his best to support them. Once submitted, he summarizes the results for students and outlines any changes he plans to make.
  • Explain the purpose of different components in your class. Reese gave the example of something basic like office hours. First generation students may not understand the purpose or value of office hours – it is a chance to ask for help but also discuss career goals with instructors. Taking the time to explain resources that are available to students has proven very useful.
  • Follow the principles of Universal Design for Learning (UDL). UDL is an approach to designing instruction in flexible ways in order to reduce barriers to learning. For example, instructors can provide alternatives when giving an assessment – some students may take an exam, others may submit a paper, etc. Reese acknowledged that this often means more work on his end, as he will be grading different types of assignments. He noted that although the format may be different, he is assessing students according to the same learning objectives.

The presentation continued with faculty attendees offering comments and suggestions of their own:

  • One faculty member explained how she thinks very carefully about what language she uses with students to mitigate her own implicit biases. She thinks about how certain words (i.e. binary language) may unintentionally signal something to students and is careful to avoid this whenever possible. Fleming agreed and stated how important it is that we all regularly examine our own biases; we should be open with students and let them know we are making an effort to communicate without bias. Reese mentioned the Harvard implicit bias test which is a tool that can help all of us discover our own hidden biases.
  • Another faculty member shared how she has found success with specifications grading to help instill a growth mindset. With specifications grading, students have multiple chances to succeed and are given lots of feedback to help them reach their goals. The faculty member also pointed out that using specifications grading is another example of a non-competitive grading strategy as students are only graded on the work they choose to complete.
  • An engineering faculty member has made an effort to proactively host events that feature speakers of varied races, cultures, and identities in order to show students who make up this particular field of study.
  • Another faculty member stressed the importance of using live captions and how beneficial they are to students and how much students appreciate them.

Towards the end, there were a few questions from the audience:

Q:  Regarding growth mindset, what exactly do you say to students?

A: Fleming responded that she tells her students everyone can be successful with whatever task they are working on, that practice is important, and failure is ok. Some students feel that if they don’t succeed quickly, they are a failure and may lose motivation to persist on difficult topics. She described how she explains the process to students like a journey – you will get to a better place than you are now and you will become more confident with time and practice. Fleming gives her students lots of encouragement throughout the semester.

Reese added that he consistently provides feedback to students throughout the semester and continually shows them examples of their success.

Q: What about students’ own biases? This comes up in teaching evaluations.

A: Fleming explained how she includes a discussion about implicit bias with her students at the beginning of the semester. She tries to make her class a positive, inclusive environment and asks that students do the same by honoring and respecting others’ opinions. She also discusses stereotypes of scientists and asks students not to evaluate her in that way.

Q: I find that most of the time, female students do not speak up. What can we do about this?

A: Reese responded that if working in groups,  a best practice is to ensure no group contains a minority of underrepresented minorities. For example, with groups of 3 there should be 0,2, or 3 woman in each group. He also suggested giving students multiple ways to participate, such as sharing questions through different modalities (email, chat, raising their hand to comment).

Q: Can we expand the time on assessments to accommodate everyone?

A: Reese replied yes, giving everyone more time will lower the pressure for everyone on that assessment. However, there are rules that still need to be followed. Reese suggested working with Student Disability Services if there are specific questions about accommodating students. Another option would be to allow students an alternative to a timed assessment.

Amy Brusini, Senior Instructional Designer
Center for Educational Resources

Image Source: Lunch and Learn Logo, Pixabay

 

Lunch and Learn: Working with Teaching Assistants

Posted on by
Reply

Lunch and Learn LogoOn December 16, 2020, the Center for Educational Resources (CER) hosted a Lunch and Learn: Working with Teaching Assistants (TAs). Rebecca Pearlman, Senior Lecturer in Biology, and Reid Mumford, Instructional Resource Advisor in Physics and Astronomy, shared strategies for how to best work with TAs to support student learning. Mike Reese from the CER moderated the discussion.

Pearlman manages a mix of graduate and undergraduate level TAs in the Biology department each semester and has worked with hundreds of TAs during her career at Hopkins. Mumford manages approximately 30 graduate level TAs each semester as well as a group of undergraduate learning assistants who help with classroom tasks but do not take part in grading. The following is a combined list of strategies that the presenters shared during the presentation:

  • Set clear expectations – This could include a contract, job overview, checklist of duties, etc.
  • Provide tips for TAs so they are prepared for the semester – This may include training documents, resources, and tools they will need.
  • Take advantage of technology – Google Forms for identifying availability, Doodle polls for arranging meetings, Blackboard site to store answer keys, Slack/MS Teams to communicate among the instructional team.
  • Build community – Take an interest in the TAs and get to know them. This helps to establish positive relationships between the instructor and TAs and among the TAs themselves.
  • Meet weekly – Review upcoming course content, monitor progress of each section, ensure TAs understand their upcoming tasks. Mumford also uses his weekly meetings to provide feedback and encouragement to the TAs.
  • Designate a head TA, if possible – If you have a large number of TAs, it may be beneficial to designate a head TA to help you manage the rest of the group. With his large number of TAs each semester, Mumford relies heavily on his head TA to make sure weekly instructions and tasks are assigned appropriately.
  • Consider feedback from students – TA evaluations can offer helpful feedback to both TAs and instructors. Pearlman noted that student feedback is especially helpful if she needs to make decisions about rehiring a TA. She can pull quotes directly from the evaluations. Mumford noted that he always meets individually with each of his TAs to review their evaluations.group of people having a discussion in front of bulletin board

There were some questions from other faculty members after the presentation:

Q: To what extent do you use undergraduate TAs?

A: Mumford responded that by policy, all of the TAs in his department are graduate students. He went on to say that he finds that experienced TAs are valuable, but it is harder to change their behavior. If they are first year TAs, he has more of an opportunity to shape them.  He does hire undergraduate learning assistants who assist with day to day tasks in the classroom but do not take part in any grading.

Pearlman stated that it seems to vary by department, but in biology, they have a mix of graduate and undgraduate TAs, and all of them participate in grading. She enjoys working with both populations, noting that the undergraduates tend to be self-starters, while the graduates bring a wealth of knowledge from the lab.

Q: To what extent is it useful for TAs to TA twice?

A: Pearlman responded that the undergraduate TAs often stay with her for years. They help each other and learn a lot by continuing in the role. They enjoy teaching the material more than once.

Q: How do you select your TAs?

A: Pearlman replied that the graduate level TAs are assigned to her department, but the undergraduates apply for the position. In the Fall 2020 semester she had 50 students apply for 18 positions. She uses a Google Form to ask them questions such as “why are you excited to be a TA? – or “what can you bring to the position?” Pearlman says she does review their grades, but they are not necessarily a priority; it often comes down to who is available at the right time.

Mumford responded that graduate TAs are assigned, but for the undergraduate learning assistants, he relies on referrals from the graduate TAs. He specifically asks them to recommend  great students. He then follows up with an interview process. Mumford is able to choose his head TA and strongly considers diversity when choosing this person.

Other faculty shared various characteristics that they use to select TAs: enthusiasm for the subject matter, respect for others, patience with students, familiarity with course material, interest in teaching, and above average grades.

Q: What can I do about TAs who do not prioritize their TA duties, such as grading?

A: Reese suggested making expectations very clear but acknowledged that it is a struggle when the TAs have obligations to research faculty as well as their TA duties.

Mumford responded that he checks in with his TAs every week to try and keep them on task. If they are non-responsive, he escalates the issue to the graduate committee to handle. If it continues, he will also reach out to the student’s research advisor and discuss the situation with that person as well.

Pearlman suggested contacting the people who run the graduate program or the department chair for assistance. She also suggested bringing the issue up at a faculty meeting.

Amy Brusini, Senior Instructional Designer
Center for Educational Resources

Image Source: Lunch and Learn Logo, Pixabay

 

 

 

 

 

 

 

Teaching Online: What Have We Learned?

Posted on by
Reply

On Friday, October 9, the Center for Educational Resources (CER) hosted an online session, “Teaching Online: What Have We Learned?” where faculty were able to share and discuss best practices based on their experiences teaching online.  Mike Reese, director of the CER and faculty member in Sociology, and Allon Brann, teaching support specialist at the CER, moderated the discussion, structuring it with a few guiding questions as outlined below:

What is something you are doing differently this semester online that you will continue to do when you are back in the classroom?

Jamie Young from Chemistry described how he is using tools such as MS Teams and Slack to build classroom community. He anticipated students feeling isolated in an online environment, so he set up spaces for casual conversation for them to communicate and get to know one another. He said it has definitely encouraged and increased conversation among students. When asked how he motivates students to participate in these environments, Young responded that he made their participation a very small part of their grade. Young and his TAs make it a point to respond right away when students post to these spaces so that students know this is an active space and that they are being heard. This level of responsiveness has also helped boost participation. Back in the classroom, Young plans to continue using these tools for informal office hours.

Rachel Sangree from Civil and Systems Engineering shared that she has been holding evening office hours and what a difference it has made in the number of students who attend.  Acknowledging that it is sometimes exhausting, she stated that she sees more students now than ever before. When we’re back in the classroom, Sangree plans to continue to hold evening office hours, but perhaps not quite as late as she offers currently.

Alison Papadakis from Psychological and Brain Sciences described how she has adapted the “think pair share” active learning strategy to an online environment. Students are split into groups and assigned breakout rooms in Zoom, then use Google Sheets to record their ideas and notes while they’re in the rooms. This allows Papadakis to monitor the progress of students without having to manually drop in to each of the rooms. She is also able to add her own comments directly to the sheets in real time as students work on them. It was noted that separate tabs are created for each group in Google Sheets, so each group has its own space to work.  Initially Papadakis was concerned that students would feel like she was ‘spying’ on them and wouldn’t like this method, but so far the feedback has been positive.  Jamie Young shared a tutorial he put together on this topic: https://docs.google.com/document/d/1uvRB38GHIKNaxQL-dN-9vpWgC43Yslssyz_jh2uPtno/edit

Francois Furstenberg from History shared how he is using the e-reader platform Perusall which allows students to collaboratively annotate their online readings. The annotations inform Furstenberg what parts of the readings students are finding interesting and are helping to shape the in-class discussions. He plans to continue using this platform when in-person classes resume. It was noted that instructors need to have copyright permission before uploading reading material to Perusall. If they have questions about obtaining copyright, they are encouraged to contact their university librarian.

David Kraemer from Mechanical Engineering mentioned that he mailed USB oscilloscope boards and a kit of devices to each of his students so they could perform “hands-on” experiments at home. He recognizes the value of these kits whether or not students are learning online, and plans to keep using them when in person classes resume.

Joshua Reiter from the Center for Leadership Education described how he adjusted his approach to assessments by breaking up large exams into more frequent quizzes throughout the semester.  Some of these quizzes were meant to be ‘fun’ quizzes for participation points, but he noticed that students were feeling pressured when taking them, defeating their purpose. Reiter changed them from individual to group quizzes, using the breakout room feature in Zoom.  Since then, he’s noticed a significant reduction in stress among the students.

 

What is something that you are still struggling with?

Several faculty members mentioned that their workload is significantly higher this semester as they do their best to recreate their lessons online.  Some feel like they are putting in three times the normal amount of preparation time and as a result, are experiencing technological overload, having to learn and keep up with so many tools. Many faculty mentioned that their students are feeling this way, too.  It was suggested that adding technology should be done in a purposeful way and that faculty should not feel compelled to use all of the available tools. Some faculty acknowledged that although it has been very challenging, they have learned a great deal about tools that are out there, and which ones seem to work better for students.

Similarly, a faculty member mentioned how difficult it is to multitask within the online environment; for example, keeping up with the chat window in Zoom while lecturing synchronously. Many faculty agreed, commenting that they feel pressured to keep up with everything going on and that it often feels like a performance. Someone commented that sharing concerns with the students helps to humanize the situation, while setting realistic expectations helps to take pressure off of instructors. One instructor mentioned how he purposely builds pauses into his lectures to allow himself time to catch up. Others mentioned that they use their TAs to monitor the chat window; if the instructor does not have a TA, he or she could ask a student.  Another general suggestion is to ask students what works best for them, instead of trying to monitor everything.

Academic integrity was another issue that came up. At least one instructor acknowledged multiple instances of students cheating since moving online, while others shared that they are concerned it may happen to them. A brief discussion followed, with instructors sharing strategies they use to try and mitigate cheating: using online monitoring tools, lowering weights of exams, making all exams open-book/open-note, and placing more emphasis on project-based work.

Some technical questions also came up, such as how to recreate an interactive whiteboard. The responses ranged from configuring Powerpoint in a certain way to using multiple devices. Jamie Young shared a tutorial he put together that uses Open Broadcaster Software (OBS): https://docs.google.com/document/d/1JXptPGjnAOiqbpvrXJPGWDcbqE_l95C6Cm0moYpaelk/edit?usp=sharing  Faculty are welcome to contact the CER for help with this and other specific technological challenges.

Anything you would like to share with others?

A few faculty members shared how they are taking advantage of the online environment. Andrew Cherlin from Sociology mentioned how much easier it is to schedule outside guests, such as authors, since there are no travel arrangements, logistics, etc. to be worked out. He has had several guests this semester already. Cherlin also described how he has taken advantage of Zoom to meet with each student individually for about fifteen minutes to check in with them and make sure they are on track. He acknowledged that this is not practical for large courses, but it has been very beneficial to those in his seminar style course.

Lori Finkelstein from Museum Studies described how being online has reshaped her assignments. She usually has students go out into the field to different museums to conduct research. This semester, students are taking a look at what museums are offering virtually and whether or not they are successful, what seems to be working, what is not working, etc.

Lester Spence from Political Science is teaching a course with instructors from Goucher College and Towson University. Students from all three schools are collaborating together as they work on group projects, something that would not necessarily be possible in a face-to-face environment.

Amy Brusini, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay

Writing Course Learning Goals

Posted on by
5

Today’s post is timely—many instructors are putting together syllabi for fall courses. This year, Johns Hopkins’ faculty who teach undergraduates are being urged to include course learning goals in their syllabi. Mike Reese, Associate Dean and Director of the Center for Educational Resources (CER), and Richard Shingles, a lecturer in Biology and Pedagogy Specialist in the CER, and created an Innovative Instructor print series article as an aid, shared below. If you are looking for other information on creating effective syllabi, type syllabus in the search box for this blog to see previous articles on the topic. Another resource for writing course learning goals is Arizona State University’s free Online Objectives Builder. It runs instructors through a logical process for creating course goals and objectives. Take the short tutorial and you are on your way.

 

Graphic illustration of three lit light bulbs.

What are course learning goals and why do they matter?

Effective teaching starts with thoughtful course planning. The first step in preparing a course is to clearly define your course learning goals. These goals describe the broad, overarching expectations of what students should be able to do by the end of the course, specifically what knowledge students should possess and/or what skills they should be able to demonstrate. Instructors use goals to design course assignments and assessments, and to determine what teaching methods will work best to achieve the desired outcomes.

Course learning goals are important for several reasons. They communicate the instructor’s expectations to students on the syllabus. They guide the instructor’s selection of appropriate teaching approaches, resources, and assignments. Learning goals inform colleagues who are teaching related or dependent courses. Similarly, departments can use them to map the curriculum. Departmental reviews of the learning goals ensure prerequisite courses teach the skills necessary for subsequent courses, and that multiple courses are not unnecessarily teaching redundant skills.

Once defined, the overarching course learning goals should inform the class-specific topics and teaching methods. Consider an example goal: At the end of the course, students will be able to apply social science data collection and analysis techniques. Several course sessions or units will be needed to teach students the knowledge and skills necessary to meet this goal. One class session might teach students how to design a survey; another could teach them how to conduct a research interview.

A syllabus usually includes a learning goals section that begins with a statement such as, “At the end of this course, students will be able to:” that is followed by 4-6 learning goals clearly defining the skills and knowledge students will be able to demonstrate.

Faculty should start with a general list of course learning goals and then refine the list to make the goals more specific. Edit the goals by taking into consideration the different abilities, interests, and expectations of your students and the amount of time available for class instruction. How many goals can your students accomplish over the length of the course? Consider including non-content goals such as skills that are important in the field.

Content goal: Analyze the key forces that influenced the rise of Japan as an economic superpower.
Non-content goal: Conduct a literature search.

The following list characterizes clearly-defined learning goals. Consider these suggestions when drafting goals.

Specific – Concise, well-defined statements of what students will be able to do.
Measurable – The goals suggest how students will be assessed. Use action verbs that can be observed through a test, homework, or project (e.g., define, apply, propose).

Non-measurable goal: Students will understand Maxwell’s Equations.
Measurable goal: Students will be able to apply the full set of Maxwell’s Equations to different events/situations.

Attainable – Students have the pre-requisite knowledge and skills and the course is long enought that students can achieve the goals.
Relevant – The skills or knowledge described are appropriate for the course or the program in which the course is embedded.
Time-bound – State when students should be able to demonstrate the skill (end of the course, end of semester, etc.).

The most difficult aspect of writing learning goals for most instructors is ensuring the goals are measurable and attainable. In an introductory science course, students may be expected to recall or describe basic facts and concepts. In a senior humanities course, students may be expected to conduct deep critical analysis and synthesis of themes and concepts. There are numerous aids online that suggest action verbs to use when writing learning goals that are measurable and achievable. These aids are typically structured by Bloom’s Taxonomy – a framework for categorizing educational goals by their challenge level. Below is an example of action verbs aligned with Bloom’s Taxonomy.

Chart showing verbs aligned with Bloom's Taxonomy levels.

Avoid vague verbs like “understand” or “know” because it can be difficult to come to consensus about how the goal can be measured. Think more specifically about what students should be able to demonstrate.

Here are examples of learning goals for several different disciplines using a common introductory statement. “By the end of this course, students will be able to do the following…

“Propose a cognitive neuroscience experiment that justifies the choice of question, experimental method and explains the logic of the proposed approach.” (Cognitive Science)
“Articulate specific connections between texts and historical, cultural, artistic, social and political contexts.” (German and Romance Languages and Literature)
“Design and conduct experiments.” (Chemistry)
“Design a system to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability.” (Biomedical Engineering)

Additional Resources
Bloom’s Taxonomy article. http://cer.jhu.edu/files/InnovInstruct-BP_blooms-taxonomy-action-speakslouder.pdf
Blog post on preparing a syllabus. http://ii.library.jhu.edu/2017/02/23/lunch-and-learn-constructing-acomprehensive-syllabus

Authors
Richard Shingles, Lecturer, Biology Department, JHU
Dr. Richard Shingles is a faculty member in the Biology department and also works with the Center for Educational Resources at Johns Hopkins University. He is the Director of the TA Training Institute and The Teaching Institute at JHU. Dr. Shingles also provides pedagogical and technological support to instructional faculty, postdocs and graduate students.
Michael J. Reese Jr., Associate Dean and Director, CER
Mike Reese is Associate Dean of University Libraries and Director of the Center for Educational Resources. He has a PhD from the Department of Sociology at Johns Hopkins University.

Images source: © 2017 Reid Sczerba, Center for Educational Resources

Lunch and Learn: Team-Based Learning

Posted on by
Reply

Logo for Lunch and Learn program showing the words Lunch and Learn in orange with a fork above and a pen below the lettering. Faculty Conversations on Teaching at the bottom.On Friday, December 16, the Center for Educational Resources (CER) hosted the second Lunch and Learn—Faculty Conversations on Teaching, for the 2016-1017 academic year. Eileen Haase, Senior Lecturer in Biomedical Engineering, and Mike Reese, Director, Center for Educational Resources, and Instructor in Sociology, discussed their approaches to team-based learning (TBL).

Eileen Haase teaches a number of core courses in Biomedical Engineering at the Whiting School of Engineering, including Freshmen Modeling and Design, BME Teaching Practicum, Molecules and Cells, and System Bioengineering Lab I and II, as well as being course director for Cell and Tissue Engineering and assisting with System Bioengineering II. She has long been a proponent of team work in the classroom.

In her presentation, Haase focused on the Molecules and Cells course, required for BME majors in the sophomore year, which she co-teaches with Harry Goldberg, Assistant Dean at the School of Medicine, Director of Academic Computing and faculty member, Department of Biomedical Engineering. The slides from Haase’s presentation are available here.

In the first class, Haase has the students do a short exercise that demonstrates the value of teamwork. Then the students take the VARK Questionnaire. VARK stands for Visual Aural Read/Write Kinesthetic and is a guide to learning styles. The questionnaire helps students and instructors by suggesting strategies for teaching and learning that align with these different styles. Haase and Goldberg found that 62% of their students were “multimodal” learners who will benefit from having the same material presented in several modes in order to learn it. In Haase’s class, in addition to group work, students work at the blackboard, use clickers, have access to online materials, participate in think-pair-share exercises, and get some content explained in lecture form.

Team work takes place in sections most FridSlide from Eileen Haase's presentation on Team-based Learning showing a scratch card test.ays. At the start of class, students take an individual, 10 question quiz called the iRAT, Individual Readiness Assurance Test, which consists of multiple-choice questions based on pre-class assigned materials. The students then take the test as a group (gRAT). Haase uses IF-AT scratch cards for these quizzes. Both tests count towards the students’ grades.

To provide evidence for the efficacy of team-based learning, Haase and Goldberg retested students from their course five months after the original final exam (99 of the 137 students enrolled in the course were retested). The data showed that students scored significantly better on the final exam on material that had been taught using team-based learning strategies and on the retest, retained significantly more of the TBL taught material.

Slide from Mike Reese's presentation on Team-based Learning showing four students doing data collection at a Baltimore neighborhood market.Mike Reese, Director of the Center for Educational Resources and instructor in the Department of Sociology, presented on his experiences with team-based learning in courses that included community-based learning in Baltimore City neighborhoods [presentation slides]. His courses are typically small and discussion oriented. Students read papers on urban issues and, in class, discuss these and develop research methodologies for gathering data in the field. Students are divided into teams, and Reese accompanies each team as they go out into neighborhoods to gather data by talking to people on the street and making observations on their surroundings. The students then do group presentations on their field work and write individual papers. Reese says that team work is hard, but students realize that they could not collect and analyze data in such a short time-frame without a group effort.

Reese noted that learning is a social process. We are social beings, and while many students dislike group projects, they will learn and retain more (as Haase and Goldberg demonstrated). This is not automatic. Instructors need to be thoughtful about structuring team work in their courses. The emotional climate created by the teacher is important. Reese shared a list of things to consider when designing a course that will incorporate team-based learning.

  1. Purpose: Why are you doing it? For Reese, teamwork is a skill that students should acquire, but primarily it serves his learning objectives.  If students are going to conduct a mini-research project in a short amount of time, they need multiple people working collectively to help with data collection and analysis.
  2. Group Size: This depends on the context and the course, but experts agree that having three to five students in a group is best to prevent slacking by team members.
  3. Roles: Reese finds that assigning roles works well as students don’t necessarily come into the course with strong project management skills, and projects typically require a division of labor. It was suggested that assigning roles is essential to the concept of true team-based learning as opposed to group work.
  4. Formation: One key to teamwork success is having the instructor assign students to groups rather than allowing them to self-select. [Research supports this. See Fiechtner, S. B., & Davis, E. A. (1985). Why some groups fail: A survey of students’ experiences with learning groups. The Organizational Behavior Teaching Review, 9(4), 75-88.] In Reese’s experience assigning students to groups helps them to build social capital and relationships at the institution beyond their current group of friends.
  5. Diversity: It is important not to isolate at-risk minorities. See: Heller, P. and Hollabaugh, M. (1992). Teaching problem solving through cooperative grouping. American Journal of Physics, 60 (7), 637-644.
  6. Ice Breakers: The use of ice breakers can help establish healthy team relationships. Have students create a team name, for example, to promote an identity within the group.
  7. Contracts: Having a contract for teamwork is a good idea. In the contract, students agree to support each other and commit to doing their share of the work. Students can create contracts themselves, but it is best if the instructor provides structured questions to guide them.
  8. Persistence: Consider the purpose of having groups and how long they will last. Depending on learning goals, teams may work together over an entire semester, or reform after each course module is completed.
  9. Check-ins: It is important to check in with teams on a regular basis, especially if the team is working together over an entire semester, to make sure that the group hasn’t developed problems and become dysfunctional.
  10. Peer Evaluation: Using peer evaluation keeps a check on the students to ensure that everyone is doing a fair share of the work. The instructor can develop a rubric, or have students work together to create one. Evaluation should be on specific tasks. Ratings should be anonymous (to the students, not the instructor) to ensure honest evaluation, and students should also self-evaluate.

In the discussion that followed the presentation, mentoring of teams and peer assessment were key topics. Several faculty with experience working with team-based learning recommended providing support systems in the form of mentors and or coaches who are assigned to the groups. These could be teaching assistants or undergraduate assistants who have previously taken the course. Resources for team-based learning were mentioned. CATME, “which stands for ‘Comprehensive Assessment of Team Member Effectiveness,’ is a free set of tools designed to help instructors manage group work and team assignments more effectively.”

Doodle was suggested as another tool for scheduling collaborative work. Many are familiar with the Doodle poll concept, but there are also free tools such as Connect Calendars and Meet Me that can be used by students.

An Innovative Instructor print article, Making Group Projects Work by Pam Sheff and Leslie Kendrick, Center for Leadership Education,  August 2012, covers many aspects of successful teamwork.

Another resource of interest is a scholarly article by Barbara Oakley and Richard Felder, Turning Student Groups into Effective Teams [Oakley, B., Felder, R.M., Brent, R., Elhajj, I. Journal of student centered learning, 2004]. “This paper is a guide to the effective design and management of team assignments in a college classroom where little class time is available for instruction on teaming skills. Topics discussed include forming teams, helping them become effective, and using peer ratings to adjust team grades for individual performance. A Frequently Asked Questions section offers suggestions for dealing with several problems that commonly arise with student teams, and forms and handouts are provided to assist in team formation and management.”

If you are an instructor on the Homewood campus, staff in the Centerfor Educational Resources will be happy to talk with you about team-based learning and your courses.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Sources: Lunch and Learn logo by Reid Sczerba, presentation slides by Eileen Haase and Mike Reese

To Curve or Not to Curve

Posted on by
4

A version of this post appeared in the print series of The Innovative Instructor.

Yellow traffic signs showing a bell curve and a stylized graph referencing criterion-referenced grading.Instructors choose grading schemes for a variety of reasons. Some may select a method that reflects the way they were assessed as students; others may follow the lead of a mentor or senior faculty member in their department. To curve or not to curve is a big question. Understanding the motivations behind and reasons for curving or not curving grades can help instructors select the most appropriate grading schemes for their courses.

Curving defines grades according to the distribution of student scores. Grades are determined after all student scores for the assignment or test are assigned. Often called norm-referenced grading, curving assigns grades to students based on their performance relative to the class as a whole. Criterion-referenced grading (i.e., not curving) assigns grades without this reference. The instructor determines the threshold for grades before the assignment is submitted or the test is taken. For example, a 92 could be defined as the base threshold for an A, regardless of how many students score above or below the threshold.

Choosing to curve grades or use a criterion referenced grading system can affect the culture of competition and/or the students’ sense of faculty fairness in a class. Curving grades provides a way to standardize grades. If a department rotates faculty responsibility for teaching a course (such as a large introductory science course), norm-referenced grading can ensure that the distribution of grades is comparable from year-to-year. A course with multiple graders, such as a science lab that uses a fleet of graduate students in the grading, may also employ a norm referencing technique to standardize grades across sections. In this case, standardization across multiple graders should begin with training the graders. Curving grades should not be a substitute for instructing multiple graders how to assign grades based on a pre-defined rubric (The Innovative Instructor: “Calibrating Multiple Graders”).

In addition to standardizing grades, norm-referenced grading can enable faculty to design more challenging assignments that differentiate top performers who score significantly above the mean. More challenging assignments can skew the grade distribution; norm-referenced grading can then minimize the impact on the majority of students whose scores will likely be lower.

A critique of curving grades is that some students, no matter how well they perform, will be assigned a lower grade than they feel they deserve. Shouldn’t all students have an equal chance to earn an A? For this reason, some instructors do not pre-determine the distribution of grades. The benefit of using a criterion-referenced grading scheme is that it minimizes the sense of competition among students because they are not competing for a limited number of A’s or B’s. Their absolute score, not relative performance, determines their grade.

There are multiple ways to curve grades.

Image showing a bell curve.I. The Bell Curve

Normalizes scores using a statistical technique to reshape the distribution into a bell curve. An instructor then assigns a grade (e.g., C+) to the middle (median) score and determines grade thresholds based on the distance of scores from this reference point. A spreadsheet application like Excel can be used to normalize scores. CER staff can assist instructors in normalizing scores.

Image showing clumping.II. Clumping

The instructor creates a distribution of the scores and identifies clusters of scores separated by breaks in the distribution, then uses these gaps as a threshold for assigning grades.

 

Image showing quota system.III. Quota Systems

Often used in law schools, the instructor pre-determines the number of students who can earn each grade. The instructor applies these quotas after rank ordering student scores.

 

Image showing criterion-reference grading.IV. Criterion-reference grading

Using a pre-determined scale, assessments are based on clearly defined learning objectives and grading rubrics so students know the instructor’s expectations for an A, B, C, etc.

 

During the 2011 Robert Resnick Lecture at Johns Hopkins, Carl Wieman, Nobel Laureate and Associate Director for Science at the President’s Office of Science and Technology, argued that most instructors are not trained to create valid assessments of student learning. Curving can be used as a tool to adjust grades on a poorly designed test, but consistent use of curving should not be a substitute for designing assessments that accurately assess what the instructor wants students to learn by the end of the course. CER staff are happy to talk to faculty about defining learning objectives and/or strategies for designing challenging and accurate student assessment instruments.

Additional Resources

• Campbell, C. (2012). Learning-centered grading practices. Leadership. 41(5), 30-33

• Jacobson, N. (2001). A method for normalizing students’ scores when employing multiple graders.  ACM SIGCSE Bulletin. 33(4), 35-38.

• Joe Champion’s Grading Transformation Spreadsheet. This spreadsheet automatically curves students’ scores after the instructor copies the scores into the spreadsheet and sets a variable defining the amount of curve.

Michael J. Reese, Associate Director
Center for Educational Resources

Image Sources: © Reid Sczerba, 2013.