Lunch and Learn: Working with Teaching Assistants

Lunch and Learn LogoOn December 16, 2020, the Center for Educational Resources (CER) hosted a Lunch and Learn: Working with Teaching Assistants (TAs). Rebecca Pearlman, Senior Lecturer in Biology, and Reid Mumford, Instructional Resource Advisor in Physics and Astronomy, shared strategies for how to best work with TAs to support student learning. Mike Reese from the CER moderated the discussion.

Pearlman manages a mix of graduate and undergraduate level TAs in the Biology department each semester and has worked with hundreds of TAs during her career at Hopkins. Mumford manages approximately 30 graduate level TAs each semester as well as a group of undergraduate learning assistants who help with classroom tasks but do not take part in grading. The following is a combined list of strategies that the presenters shared during the presentation:

  • Set clear expectations – This could include a contract, job overview, checklist of duties, etc.
  • Provide tips for TAs so they are prepared for the semester – This may include training documents, resources, and tools they will need.
  • Take advantage of technology – Google Forms for identifying availability, Doodle polls for arranging meetings, Blackboard site to store answer keys, Slack/MS Teams to communicate among the instructional team.
  • Build community – Take an interest in the TAs and get to know them. This helps to establish positive relationships between the instructor and TAs and among the TAs themselves.
  • Meet weekly – Review upcoming course content, monitor progress of each section, ensure TAs understand their upcoming tasks. Mumford also uses his weekly meetings to provide feedback and encouragement to the TAs.
  • Designate a head TA, if possible – If you have a large number of TAs, it may be beneficial to designate a head TA to help you manage the rest of the group. With his large number of TAs each semester, Mumford relies heavily on his head TA to make sure weekly instructions and tasks are assigned appropriately.
  • Consider feedback from students – TA evaluations can offer helpful feedback to both TAs and instructors. Pearlman noted that student feedback is especially helpful if she needs to make decisions about rehiring a TA. She can pull quotes directly from the evaluations. Mumford noted that he always meets individually with each of his TAs to review their evaluations.group of people having a discussion in front of bulletin board

There were some questions from other faculty members after the presentation:

Q: To what extent do you use undergraduate TAs?

A: Mumford responded that by policy, all of the TAs in his department are graduate students. He went on to say that he finds that experienced TAs are valuable, but it is harder to change their behavior. If they are first year TAs, he has more of an opportunity to shape them.  He does hire undergraduate learning assistants who assist with day to day tasks in the classroom but do not take part in any grading.

Pearlman stated that it seems to vary by department, but in biology, they have a mix of graduate and undgraduate TAs, and all of them participate in grading. She enjoys working with both populations, noting that the undergraduates tend to be self-starters, while the graduates bring a wealth of knowledge from the lab.

Q: To what extent is it useful for TAs to TA twice?

A: Pearlman responded that the undergraduate TAs often stay with her for years. They help each other and learn a lot by continuing in the role. They enjoy teaching the material more than once.

Q: How do you select your TAs?

A: Pearlman replied that the graduate level TAs are assigned to her department, but the undergraduates apply for the position. In the Fall 2020 semester she had 50 students apply for 18 positions. She uses a Google Form to ask them questions such as “why are you excited to be a TA? – or “what can you bring to the position?” Pearlman says she does review their grades, but they are not necessarily a priority; it often comes down to who is available at the right time.

Mumford responded that graduate TAs are assigned, but for the undergraduate learning assistants, he relies on referrals from the graduate TAs. He specifically asks them to recommend  great students. He then follows up with an interview process. Mumford is able to choose his head TA and strongly considers diversity when choosing this person.

Other faculty shared various characteristics that they use to select TAs: enthusiasm for the subject matter, respect for others, patience with students, familiarity with course material, interest in teaching, and above average grades.

Q: What can I do about TAs who do not prioritize their TA duties, such as grading?

A: Reese suggested making expectations very clear but acknowledged that it is a struggle when the TAs have obligations to research faculty as well as their TA duties.

Mumford responded that he checks in with his TAs every week to try and keep them on task. If they are non-responsive, he escalates the issue to the graduate committee to handle. If it continues, he will also reach out to the student’s research advisor and discuss the situation with that person as well.

Pearlman suggested contacting the people who run the graduate program or the department chair for assistance. She also suggested bringing the issue up at a faculty meeting.

Amy Brusini, Senior Instructional Designer
Center for Educational Resources

Image Source: Lunch and Learn Logo, Pixabay

 

 

 

 

 

 

 

Lunch and Learn: Creating Rubrics and Calibrating Multiple Graders

Logo for Lunch and Learn program showing the words Lunch and Learn in orange with a fork above and a pen below the lettering. Faculty Conversations on Teaching at the bottom.On Friday, December 15, the Center for Educational Resources (CER) hosted the second Lunch and Learn—Faculty Conversations on Teaching—for the 2017-2018 academic year.  Laura Foster, Academic Advisor, Public Health Studies, and Reid Mumford, Instructional Resource Advisor, Physics & Astronomy, presented on Creating Rubrics and Calibrating Multiple Graders.

Laura Foster led by giving us a demonstration of her use of Blackboard for creating rubrics. She noted that she might be “preaching to the choir” but hoped that those present might take back these best practices to their colleagues. Noting that many faculty have negative opinions of Blackboard, she put in a plug for its organizational benefits and facilitation of communication with students.

Foster started using Blackboard tools for a Public Health Studies class where she was grading student reflections. The subject matter—public health studies in the media—was outside of her field of physical chemistry. Blackboard facilitates creating a rubric that students can see when doing an assignment and the instructor then uses to grade that work. She showed the rubric detail that students see in Blackboard, and how the rubric can be used in grading. [See the CER Tutorial on Blackboard Rubrics and Rubrics-Helpful Hints] The rubric gives the students direction and assures that the instructor (or other graders) will apply the same standards across all student work.

It empowers students when they know exactly what criteria will be used in evaluating their work and how many points will be assigned to each component. Foster has found that using rubrics is an effective way to communicate assignment requirements to students, and that it helps her to clarify for herself what at the most important points. She noted that a rubric is very useful when there are multiple graders, such as Teaching Assistants (TAs), as it helps to calibrate the grading.

In response to questions from the audience, Foster stated that rubrics can be developed to cover both qualitative and quantitative elements. Developing good rubrics is an iterative process; it took her some time to sharpen her skills. There is flexibility in differentiating points allotted, but the instructor must be thoughtful, plan for a desired outcome, and communicate clearly. The rubric tool can be used to grade PDF files as well as Word documents. Foster noted that it is important to take opportunities to teach students to learn to write, learn to use technology, learn to read instructions, and learn to look at feedback given on assignments. Being transparent and explaining why you are using a particular technology will go a long way.

Reid Mumford gave his presentation on how he calibrates multiple graders (see slides). Mumford oversees the General Physics lab courses. This is a two semester, required sequence, so not all students are excited to be there. The sequences are on Mechanics and Electricity and Magnetism; both labs are taught every semester with multiple sections for each course. Approximately 600 to 700 students are taking these lab sequences each semester; students are divided into sections of about 24 students. The labs are open-ended and flexible, so students aren’t filling in blanks and checking boxes, which would be easier to grade. Lab sections are taught and graded by graduate student TAs, with about 30 TAs teaching each semester. Teaching and grading styles vary among these TAs as would be expected. Clearly, calibrating their grading is a challenge.

Grades are based on the best 9 of 10 lab activities, which consist of a pre-lab quiz and a lab note. All activities are graded using the same rubric. The grading scale used can be seen in the slides. One of the criteria for grading is “style,” which allows some flexibility and qualitative assessment. Students have access to the rubric, which is also shown in the slides.

About three years ago, Mumford adopted Turnitin (TII), the plagiarism detection tool, for Screen shot of Quick Mark grading tool.its efficient grading tools. It works well for his use because it is integrated with Blackboard. TII does its job in detecting cheating (and Mumford noted that lots of students are cheating), but it is the grading tools that are really important for the TAs. TAs are encouraged to be demanding in their grading and leave a lot of feedback, so grading takes them two to four hours each week. TII’s Feedback Studio (formerly known as GradeMark) allows TAs to accomplish their mission. [See CER tutorial on Feedback Studio and The Innovative Instructor post on GradeMark.] It was the QuickMark feature that sold Mumford on Feedback Studio and TII grading. Using the rubric for each activity, QuickMark can be pre-populated with commonly-used comments, which can then be dragged and dropped onto the student’s submitted work.

Graph showing General Physics Laboratory Section Grading Trends.These tools helped make the grading load more efficient, but calibrating the multiple graders was another challenge. Mumford found that the TAs need lots of feedback on their grading. Each week he downloads all the grades from Blackboard grade centers. He creates a plot that shows the average score for the weekly lab assignment. Outliers to the average scores are identified and these TAs are counseled so that their grading can be brought into line. Mumford also looks at section grading trends and can see which sections are being graded more leniently or harshly than average. He works with those TAs to standardize their grading.

In calculating final grades for the course, Mumford keeps three points in mind: final letter grades must be calculated, there should be no “easy” or “hard” sections of lab, and distribution should not vary (significantly) between sections. He makes use of per-section mapping and uses average and standard deviation to map results to a final letter grade model. Mumford noted that students are made aware, repeatedly, of the model being used. He is very transparent—everything is explained in the syllabus and reiterated weekly in lab sessions.

In conclusion, Mumford offered these take-aways:

  • Calibrating Multiple Graders is not easy
  • Tools are needed to handle multiple sections efficiently
  • Rubrics help but do not solve the calibration problem
  • Regular feedback to graders is essential
  • Limit of the system: student standing is ambiguous

In the future Mumford plans to give students a better understanding of course standing, to calculate a per-section curve each week, and to overcome some technical issues and the greater time investment that will be required with weekly calibrating and rescaling.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Sources: Lunch and Learn Logo, slides from Mumford presentation