Grading in the fast lane with Gradescope

Posted on February 15, 2018 by Macie Hall

[Guest post by Scott Smith, Professor, Computer Science, Johns Hopkins University]

Three speedometers for quality, grades per hour, and efficiency. Grading can be one of the most time consuming and tedious aspects of teaching a course, but it’s important to give prompt and meaningful feedback to your students. In large courses, aligning grading practices across multiple teaching assistants (TAs) necessitates a level of coordination that includes scheduling grading meetings, reviewing materials for correct answers, and calibrating point evaluations, all of which can take up valuable time during the semester.

In courses that teach programming, we typically assign students projects that require them to write programs to solve problems. When instructors grade this type of assignment, they not only have to observe the program’s results but also the student’s approach. If the results are not correct or the program doesn’t run, we have to spend time reviewing hundreds of lines of code to debug the program to give thoughtful feedback.

In the past, my method for grading assignments with my TAs may have been arduous but it worked. However, last year, no TAs were assigned to my Principles of Programming Languages course. Concerned that I wouldn’t have enough time to do all the work, I looked for another solution.

Consistent grading and providing meaningful feedback for student’s every submission, especially with multiple teaching assistants (TAs) can be challenging. Typically, when grading, I would schedule a time to sit down with all of my TAs, review the assignment or exam, give each TA a set of questions to grade, pass the submissions around until all were graded, and finally calculate the grades. When a TA had a question, we could address it as a group and make the related adjustments throughout the submissions as needed. While this system worked, it was tedious and time consuming. Occasionally, inconsistencies in the grades came up, which could prompt regrade requests from students. I kept thinking that there had to be a better way.

About year and a half ago, a colleague introduced me to an application called Gradescope to manage the grading of assignments and exams. I spent a relatively short amount of time getting familiar with the application and used it in a course in the fall of 2016, for both student-submitted homework assignments and in-class paper exams. In the case of the homework, students would upload a digital version of the assignment to Gradescope. The application would then prompt the student to designate the areas in the document where their answers can be found so that the application could sort and organize the submissions for the ease of grading. For the in-class exams, I would have the students work on a paper-based exam that I set up in Gradescope with the question areas established. I then would scan and upload the exams so that Gradescope could associate the established question areas to the student submissions automatically. The process of digitizing the completed tests and correlating them to the class roster was made easy with a scanner and Gradescope’s automatic roster matching feature. Gradescope became a centralized location where my TAs and I could grade student work.

There are a few ways to consider incorporating Gradescope into your course. Here is a non-exhaustive list of scenarios for both assignments and exams that can be accommodated:

Handwritten/drawn homework (students scan them and upload the images/PDFs)

Electronic written homework (students upload PDFs)

In-class exams (instructor scans them and uploads the PDFs)

Coding scripts for programming assignment (students upload their program’s files for auto-grading)

Code assignments graded by hand (students upload PDFs of code)

The real power of Gradescope is that it requires setting up a reusable rubric (a list of competencies or qualities used to assess correct answers) to grade each question. When grading, you select from or add to the rubric to add or deduct points. This keeps the grading consistent across multiple submissions. As the rubric is established as a part of the assignment, you can also update the point values at any time if you later determine that a larger point addition/deduction is advisable, and the grade calculations will update automatically.

Screenshot from Gradescope--Review grade for assignment feature.

Screenshot of Gradescope’s Review Grade for an assignment

After being informed that I wouldn’t have any TAs for my Principles of Programming Languages course the following semester, I was motivated to use one of Gradescope’s [features, the programming assignment auto-grader platform. Being able to automatically provide grades and feedback for students’ submitted code has long been a dream of instructors who teach programming. Gradescope offers a language-agnostic environment in which the instructor sets up the components and libraries needed for the students’ programs to run. The instructor establishes a grading script that is the basis for the analysis, providing grades and feedback for issues found in each student’s submitted program.

Overall, the use of Gradescope has reduced time spent grading and improves the quality of feedback that I am able to provide students. For instance, when I release grades to the students, they are able to review each of the descriptive rubrics that were used when grading their submissions, as well as any additional comments. Auto-grader was really the star feature in this case. Students were able to submit their code, determine if it would run, and make corrections before the deadline to increase their chances of a better grade. There are features to reduce the number of allowed submissions, but I choose not to set a limit so that the students could use an iterative approach to getting the right solution.

Gradescope is only effective if your rubrics and grading criteria are well thought out, and the auto-grading scripts require some time to set up. Creating the grading scripts for the programming assignments may seem time intensive, but by frontloading the work with detailed rubrics and test cases, more time is saved in the grading process. The value of this preparation scales as enrollment increases, and the rubrics and scripts can be reused when you teach the course again. With more time during the semester freed up by streamlining the grading process, my TAs and I were able to increase office hours, which is more beneficial in the long run for the students.

Student’s submission with rubric items used in grading

The process for regrading is much easier for both students and instructors. Before Gradescope, a regrade request meant determining which TA graded that question, discussing the request with them, and then potentially adjusting the grade. With the regrade feature, students submit a regrade request, which gets routed to that question’s grader (me or the TA) with comments for the grader to consider. The grader can then award the regrade points directly to the student’s assignment. As the instructor, I can see all regrade requests, and can override if necessary, which helps to reduce the bureaucracy and logistics involved with manual regrading. Additionally, regrade requests and Gradescope’s assignment statistics feature may allow you to pinpoint issues with a particular question or how well students have understood a topic.

I have found that when preparing assignments with Gradescope, I am more willing to create multiple mini-assignments. With large courses, the tendency would be to create fewer assignments that are larger in scope to lessen the amount of grading. When there are too few submission points for students who are deadline oriented, I find that they wait till the last few days to start the assignment, which can make the learning process less effective. By adding more assignments, I can scaffold the learning to incrementally build on topics taught in class.

After using Gradescope for a year, I realized that it could be used to detect cheating. Gradescope allows you to see submissions to specific questions in sequence, making it easy to spot submissions that are identical, a red-flag for copied answers. While not a feature, it is an undocumented bonus. It should also be noted that Gradescope adheres to FERPA (Family Educational Rights and Privacy Act) standards for educational tools.

Additional Resources:

Gradescope website: https://gradescope.com
NOTE TO JHU READERS ONLY: The institutional version of Gradescope is currently available to JHU faculty users through a pilot program. If you are faculty at Johns Hopkins University’s Homewood campus interested in learning more about how Gradescope might work for your courses, contact Reid Sczerba in the Center for Educational Resources at rsczerb1@jhu.edu.

Scott Smith, Professor
Department of Computer Science, Johns Hopkins University

Scott Smith has been a professor of Computer Science at Hopkins for almost 30 years. His research specialty is programming languages. For the past several years, he has taught two main courses, Software Engineering, a 100 student project-based class, and Principles of Programming Languages, a mathematically-oriented course with both written and small programming assignments.

Images Sources: CC Reid Sczerba, Gradescope screenshots courtesy Scott Smith

Quick Tips: How to Manage Your Teaching During a Personal Crisis

Posted on February 7, 2018 by Macie Hall

There are a number of resources available to help instructors deal with teaching when an institution or our country has experienced a disaster or crisis [see The Silhouette of a man seated with elbows on his knees suggesting despair. Innovative Instructor post Quick Tips: Teaching in Challenging Times and Facilitating Difficult Discussions, November 15, 2016], and this blog has covered how to handle student crises, real and not [The Dead Grandmother Syndrome and How to Treat It, December 7, 2016], but what happens when the death or serious illness in the family is your family?

A recent article in Faculty Focus, Teaching During a Personal Crisis (Elizabeth Barnett, February 2, 2018) deals with these issues. Barnett outlines 7 strategies, recommended by experienced colleagues and mental health professionals for teaching during a time of personal difficulty. These are: acknowledge the crisis, triage, consider telling your colleagues, consider telling your students, adjust your expectations, try new routines, protect yourself.

Telling your students may seem like the most controversial piece of advice, but Barnett says:

Wait, you say, this is totally inappropriate. And it would be if you told them in the same way as you did your colleagues, as peers. But if your performance and availability are limited due to the crisis, you can reduce guilt and minimize questions by letting them know something is going on. …[Emphasize] to your students that you don’t need help from them, you have that support elsewhere, and you only want to inform them of the situation.

In other words, you don’t need to go into the gory details, just let your students know that you are experiencing some personal issues that may affect your teaching, for example, that you may be absent, or that someone else may be filling in as instructor for some classes or handling grading of some assignments.

Barnett also offers some tips on dealing with a colleague in crisis, particularly how to listen respectfully and things that you can offer to do to help.

The article is short, but useful. If you have suggestions from your own experiences with dealing with personal crisis, please share them in the comments.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Fail (in order) to Succeed

Posted on January 31, 2018 by Macie Hall

Last week I attended the CUE2 Symposium held here on the Homewood Campus of Johns Hopkins University. CUE2 stands for the Second Commission on Undergraduate Education. The first CUE was formed in 2002 “…to identify the core values that should characterize the undergraduate experience of our students.” The charge of the second commission “… is to interpret the mission of an undergraduate education in the 21st century and develop a new model that will serve us for the next decade or more.” In fulfilling that charge, CUE2 leaders have invited noted thinkers and innovators in undergraduate education to campus to address these issues. Three speakers addressed us last week, Edward B. Burger, President, Southwestern University, Randy Bass, Vice Provost for Education and Professor of English at Georgetown University, and Steven Mintz, Professor of History at University of Texas, Austin and founding director of The University of Texas System’s Institute for Transformational Learning.

All three had interesting things to say about transforming undergraduate education, but I found Edward Burger’s remarks to be particularly intriguing.

Burger is the co-author of a book, The Five Elements of Effective Thinking, with Michael Starbird [Princeton University Press, 2012]. Burger and Starbird describe five elements that characterize effective thinking: understanding deeply, failing in order to succeed, raising questions, seeing the flow of ideas, and engaging change. A review of the book on Farnam Street Blog summarizes the elements. The book is a quick read and one that I think instructors will find valuable in thinking at a high level about learning and teaching. The chapter on how mistakes can ignite insight, along with Burger’s remarks at the symposium on effective failure, struck a chord.

Last summer The Innovative Instructor featured two posts on successful failure, When Failure is a Good Thing (July 14, 2017) and How Pretesting Can Help Your Students Fail Well (July 18, 2017. The first post discussed a program at Smith College that encourages students to understand the value of failure as a learning tool, the second offered a practical means, pre-testing, for instructors to use failure as an instructional strategy.

Burger maintains that effective failure is key to successful learning. It’s what you do after you make a mistake that informs your understanding. If a student receives an 80% on a test, does s/he think that is an adequate grade and move on or does the student take a careful look at what questions were answered incorrectly and seek to understand the topic? Burger characterizes acceptance of the 80% grade without further refelction as “ineffective failure.”

In Burger’s math courses, 5% of a student’s grade is based on effective failure; in order to get an A, students must first fail. Taking this approach opens students to taking risks and learning from their mistakes. When a student answers a question in class incorrectly, the instructor should ask, “Why is this wrong?” Let the students be the teachers and transform the failure into moving thinking and understanding forward. Burger maintains that “every mistake is a teacher and holds a lesson.” [The Five Elements of Effective Thinking, p. 50].

Burger summarizes the concept of making mistakes: “Fail to succeed. Intentionally get it wrong to inevitably get it even more right. Mistakes are great teachers — they highlight unforeseen opportunities and holes in your understanding. They also show you which way to turn next, and they ignite your imagination.” [p. 6].

These are valuable strategies for students and instructors alike.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Quick Tips: Subscribe to the STEM|PROF Newsletter

Posted on January 19, 2018 by Macie Hall

STEM|PROF Newsletter Logo. In order to keep the readers of The Innovative Instructor informed, I follow a number of blogs and read a lot of books and articles on pedagogy and instruction in higher education. Recently I became aware of a new-to-me resource that I’d like to share, the STEM|PROF weekly newsletter.

The newsletter is part of the outreach for a National Science Foundation-funded and Center for the Integration of Research, Teaching, and Learning-supported longitudinal study looking at the effectiveness of professional development programs for STEM doctoral students on their early-career teaching. The website for the Longitudinal Study of Future STEM Scholars has a wealth of information on the research and related resources, including the STEM|PROF newsletter.

From the Longitudinal Study of Future STEM Scholars STEM|PROF webpage:

The Longitudinal Study of Future STEM Scholars publishes a weekly newsletter, STEM|PROF, that helps advance the preparation of future STEM faculty as effective undergraduate instructors and mentors. STEM|PROF curates news, research, and events from four areas:

Teaching development for current and future STEM faculty;
Undergraduate STEM education;
Doctoral education in STEM fields; and
Academic career formation.

Articles cited range widely in topic and are relevant to those outside of the STEM disciplines. For example, a recent newsletter linked to an article in the Washington Post, Why we still need to study humanities in a STEM world (Valerie Strauss, October 18, 2017). Often newsletters include links to upcoming webinars and professional development opportunities. This week’s topics included a link to an article on strategies for generating discussion and engagement in large classes: Class Size Matters by Deborah J. Cohan, Inside Higher Ed, September 19, 2017. And there are links to useful teaching tools such as Icebreakers that Rock from the blog Cult of Pedagogy by Jennifer Gonzalez.

To subscribe to STEM|PROF, send a blank email to join-stem-prof@lists.wisc.edu. The newsletter will be delivered to your email inbox on a weekly basis.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: STEM|PROF Newsletter Logo

New Mobile Application to Improve Your Teaching

Posted on January 9, 2018 by Macie Hall

Finding time to implement effective teaching strategies can be challenging, especially for professors where teaching is only one of their many responsibilities. PhD student John Hickey is trying to solve this problem with Tcrunch, a new application (available on the Apple and Google App stores for free) he has created.

Tcrunch enables more efficient and frequent teacher-student communication. You can think about it as an electronic version of the teaching strategy called an “exit ticket.” An “exit ticket” is traditionally a 3×5 card given to students at the end of class; the teacher asks a question to gain feedback from the students and the students write a brief response. Here you can do the same thing, but Tcrunch eliminates any paper and performs all collecting and analyzing activities in real-time.

There is both a teacher and student portal into the app. Teachers can create and manage different classes. Within a class, teachers can create a question or prompt and release it to their students, who will also have Tcrunch. Students can then see this question, click on it, and answer it. Student answers come into the teacher’s app in real-time. Teachers can evaluate the results in the app or email themselves the results in the form of an Excel document. Other functionalities include multiple choice, a bank of pre-existing questions to help improve teaching, and an anonymous setting for student users.

John developed Tcrunch because of his own struggles with time and improving learning in the classroom:

“I taught my first university-level class at Johns Hopkins, and I wanted more regular feedback to my teaching style, classroom activities, and student comprehension than just the course evaluation at the end of the year. As an engineer, frequent feedback is critical to iterative improvements. I also knew that I was not going to handout, collect, read, and analyze dozens of papers at the end of each class. So, I created Tcrunch.”

The app development process took nearly a year, with iterative coding and testing with teachers and students. Both student and teacher users have enjoyed using Tcrunch. They have referenced enjoying the ease of use, being able to create and answer questions on the go, and having a platform for all their classes in one place. John has personally found Tcrunch has helped him to restructure classroom time and assignment load, and even to find out why students are missing class.

John cites this development process as the main difference between his app and already existing polling technologies.

“Finding out what the professors and students wanted allowed me to see the needs that were not filled by existing technologies. This resulted in an app specifically designed to help teachers, instead of the other way around, for example, a generalized polling tool that is also applied to teaching. The specificity in design gives it its unique functionality and user experience.”

In the future John wants to extend the reach of Tcrunch to more teachers through advertising and partnering with Edtech organizations.

While the app may not be as flashy as Pokemon Go, Tcrunch has great utility and potential in the classroom.

To find and use the app, search Tcrunch in the Apple or Google App stores and download. John Hickey can be contacted at jhickey8@jhmi.edu

John Hickey
National Science Foundation Fellow
Biomedical Engineering Ph.D. Candidate
Johns Hopkins University

Images source: John Hickey 2018

Lunch and Learn: Creating Rubrics and Calibrating Multiple Graders

Posted on December 28, 2017 by Macie Hall

On Friday, December 15, the Center for Educational Resources (CER) hosted the second Lunch and Learn—Faculty Conversations on Teaching—for the 2017-2018 academic year. Laura Foster, Academic Advisor, Public Health Studies, and Reid Mumford, Instructional Resource Advisor, Physics & Astronomy, presented on Creating Rubrics and Calibrating Multiple Graders.

Laura Foster led by giving us a demonstration of her use of Blackboard for creating rubrics. She noted that she might be “preaching to the choir” but hoped that those present might take back these best practices to their colleagues. Noting that many faculty have negative opinions of Blackboard, she put in a plug for its organizational benefits and facilitation of communication with students.

Foster started using Blackboard tools for a Public Health Studies class where she was grading student reflections. The subject matter—public health studies in the media—was outside of her field of physical chemistry. Blackboard facilitates creating a rubric that students can see when doing an assignment and the instructor then uses to grade that work. She showed the rubric detail that students see in Blackboard, and how the rubric can be used in grading. [See the CER Tutorial on Blackboard Rubrics and Rubrics-Helpful Hints] The rubric gives the students direction and assures that the instructor (or other graders) will apply the same standards across all student work.

It empowers students when they know exactly what criteria will be used in evaluating their work and how many points will be assigned to each component. Foster has found that using rubrics is an effective way to communicate assignment requirements to students, and that it helps her to clarify for herself what at the most important points. She noted that a rubric is very useful when there are multiple graders, such as Teaching Assistants (TAs), as it helps to calibrate the grading.

In response to questions from the audience, Foster stated that rubrics can be developed to cover both qualitative and quantitative elements. Developing good rubrics is an iterative process; it took her some time to sharpen her skills. There is flexibility in differentiating points allotted, but the instructor must be thoughtful, plan for a desired outcome, and communicate clearly. The rubric tool can be used to grade PDF files as well as Word documents. Foster noted that it is important to take opportunities to teach students to learn to write, learn to use technology, learn to read instructions, and learn to look at feedback given on assignments. Being transparent and explaining why you are using a particular technology will go a long way.

Reid Mumford gave his presentation on how he calibrates multiple graders (see slides). Mumford oversees the General Physics lab courses. This is a two semester, required sequence, so not all students are excited to be there. The sequences are on Mechanics and Electricity and Magnetism; both labs are taught every semester with multiple sections for each course. Approximately 600 to 700 students are taking these lab sequences each semester; students are divided into sections of about 24 students. The labs are open-ended and flexible, so students aren’t filling in blanks and checking boxes, which would be easier to grade. Lab sections are taught and graded by graduate student TAs, with about 30 TAs teaching each semester. Teaching and grading styles vary among these TAs as would be expected. Clearly, calibrating their grading is a challenge.

Grades are based on the best 9 of 10 lab activities, which consist of a pre-lab quiz and a lab note. All activities are graded using the same rubric. The grading scale used can be seen in the slides. One of the criteria for grading is “style,” which allows some flexibility and qualitative assessment. Students have access to the rubric, which is also shown in the slides.

About three years ago, Mumford adopted Turnitin (TII), the plagiarism detection tool, for its efficient grading tools. It works well for his use because it is integrated with Blackboard. TII does its job in detecting cheating (and Mumford noted that lots of students are cheating), but it is the grading tools that are really important for the TAs. TAs are encouraged to be demanding in their grading and leave a lot of feedback, so grading takes them two to four hours each week. TII’s Feedback Studio (formerly known as GradeMark) allows TAs to accomplish their mission. [See CER tutorial on Feedback Studio and The Innovative Instructor post on GradeMark.] It was the QuickMark feature that sold Mumford on Feedback Studio and TII grading. Using the rubric for each activity, QuickMark can be pre-populated with commonly-used comments, which can then be dragged and dropped onto the student’s submitted work.

These tools helped make the grading load more efficient, but calibrating the multiple graders was another challenge. Mumford found that the TAs need lots of feedback on their grading. Each week he downloads all the grades from Blackboard grade centers. He creates a plot that shows the average score for the weekly lab assignment. Outliers to the average scores are identified and these TAs are counseled so that their grading can be brought into line. Mumford also looks at section grading trends and can see which sections are being graded more leniently or harshly than average. He works with those TAs to standardize their grading.

In calculating final grades for the course, Mumford keeps three points in mind: final letter grades must be calculated, there should be no “easy” or “hard” sections of lab, and distribution should not vary (significantly) between sections. He makes use of per-section mapping and uses average and standard deviation to map results to a final letter grade model. Mumford noted that students are made aware, repeatedly, of the model being used. He is very transparent—everything is explained in the syllabus and reiterated weekly in lab sessions.

In conclusion, Mumford offered these take-aways:

Calibrating Multiple Graders is not easy
Tools are needed to handle multiple sections efficiently
Rubrics help but do not solve the calibration problem
Regular feedback to graders is essential
Limit of the system: student standing is ambiguous

In the future Mumford plans to give students a better understanding of course standing, to calculate a per-section curve each week, and to overcome some technical issues and the greater time investment that will be required with weekly calibrating and rescaling.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Sources: Lunch and Learn Logo, slides from Mumford presentation

Thinking about Accessibility Part 2

Posted on December 15, 2017 by Macie Hall

Last week ‘s post summarized the first part of an important article by Anne-Marie Womack, Teaching Is Accommodation: Universally Designing Composition Classrooms and Syllabi, College Composition and Communication, February 2017. Womack, assistant director of writing at Tulane University, encourages instructors to rethink accommodation as normative rather than an exception to a rule.

The second part of the essay focuses on creating an accessible syllabus both in theory and in practice by “moving from syllabus as contract to syllabus as accommodation.” This can be accomplished in a number of ways, starting by not limiting the notion of accommodation to a paragraph stating the institutional disability policy. Syllabi are often heavy on text and present inflexible policies written in a punitive and/or defensive tone. Womack suggests instead that instructors strive to create accessible documents that engage students with “cooperative language” and flexibility in assignments. She does acknowledge that it is important to consider the particular student audience in designing a syllabus—one solution doesn’t fit all. Instead, she suggests three strategies.

The first strategy is Creating Accessible Document Design. Reducing the amount of text in your syllabus is the primary consideration. Womack starts with the boilerplate policies that may be required by the institution, school, or department. Such information may be reduced to basics, linked to, or included at the end in an appendix. Other information can be provided during the semester as it is needed, e.g., assignment specifics, writing prompts.

How the text is formatted can make a big difference—Womack suggests trading some text for accessible images. This will necessitate the creation of alternative text (alt text) to describe the image for those who use screen readers. She suggests that alt text be “focused on data, not extraneous visual details.” Since images can introduce color that may not be visible to all readers, there are guidelines for best practices. (I’ve included some resources for creating accessible images and documents at the end of this post.) Womack shows examples of how she converted her syllabus from text heavy to accessible by using images, blocks of text, bulleted lists, and icons, and increasing the amount of white space. Using a larger font size and a san serif font will also improve readability. Your goal is to “make the syllabus user-friendly” because you want your students to use it. Other tips include creating a table of contents for improved navigation, making internal hyperlinks to connect items within the syllabus, using headings to establish hierarchy, bolding text to emphasize key ideas.

The second strategy Womack advocates is Engaging Students with Cooperative Language. You want to convey “approachability and empathy” rather than making students feel uncomfortable requesting accommodations. Don’t focus on negative consequences and punitive rules. Instead look at inclusive practices in your syllabus as a means to engage student cooperation.

Womack suggests beginning with an “inclusive learning statement” and offers an example:

Your success in this class is important to me. We will all need accommodations because we all learn differently. If there are aspects of this course that prevent you from learning or that form barriers to your inclusion, please let me know as soon as possible. Together we’ll develop strategies that can enable you to succeed in the course.

I encourage you to visit the Office of Disability Services to determine how you could improve your learning as well. If you need official accommodations, you have a right to have these met. There is also a range of resources on campus, including the Writing Center, Tutoring Center, and Academic Advising Center. (Figure 5, p.513).

Further, she gives examples of positive versus punitive language, commands versus invitations, and cooperative versus paternalistic language.

The third strategy is labeled Empowering Students through Flexible Course Plans. Womack saw that “traditional accommodations, such as longer time and adaptable assessment, provide a starting point to improve course practices.” Citing a number of research studies, she states that students are more motivated to learn when they feel they have autonomy. Womack suggests ways in which instructors can give students control. She discusses allowing students flexibility in deadlines. Her approach combines student-set and instructor-set deadlines, which allows an extended time-frame but prevents students from falling behind. Low-stakes writing assignments and incremental assignments may also provide flexibility. Minimal grading for some of this work is recommended. Womack also examines building flexibility into grading distributions. She focuses on those which work well for her writing courses, including contract grading where a system of assessment is negotiated. While this aspect of flexibility may be more difficult to incorporate into large lecture courses, the point is that assessments and grading should be carefully considered. Allowing students to drop the lowest grade in a series of quizzes or homeworks is an easy means of allowing students flexibility and control in a larger class.

In concluding Womack reminds us that accommodation does not give disabled students an unfair advantage; “it is more likely all students have been given an unfair disadvantage through inaccessible pedagogy.” We should look at the “best versions of accommodation” as a way of teaching inclusively to a diverse audience and adapting in response to individual needs.

While Womack offers some good examples in her essay, there are some great resources with “how-to” guides available. Womack created a website at Tulane University on making an accessible syllabus that frequently cited, but alas, no longer “accessible.” However, you can view an archived image of the site here.

If you are someone who likes video tutorials, the University of Minnesota Accessible U has a series of video tutorials on creating an accessible syllabus and making accessible documents and accessible PDFs.

You will also want to check out the University of Colorado Boulder Universally Designed Syllabus Materials website “to employ best practices when creating a course syllabus.” Some of the material is UC system-specific, but much of the information is widely applicable.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

Thinking about Accessibility Part 1

Posted on December 6, 2017 by Macie Hall

Often I read an article or blog post and suddenly find that I am falling down a rabbit hole. Hours, or even days, later I emerge, having uncovered a wealth of information and resources that have to be edited down in order to present a reasonably digestible overview of a topic for you readers. Such was the case with David Gooblar’s post Now is the Time to Think About Accessibility on his Chronicle Vitae Pedagogy Unbound blog (August 8, 2017). From the links in his article, I went on to discover other great material on accessibility to share—enough good stuff for two posts. This first post will cover thinking about accessibility in your classroom in general; next will be a follow-up post focused on creating an accessible syllabus and other documents for your class.

Gooblar starts off by noting that for many instructors, accessibility is given a brief mention at the end of the syllabus and then forgotten. Accessibility is seen as “an exception to the norm” and given little thought. He then notes an article by Anne-Marie Womack, assistant director of writing at Tulane University, which takes issue with that way of “conceptualizing accessibility.”

Teaching Is Accommodation: Universally Designing Composition Classrooms and Syllabi (College Composition and Communication, February 2017) by Anne-Marie Womack should be required reading for all higher ed instructors. [Note: if the link does not work, try copying and pasting this URL directly into your browser: http://www.ncte.org/library/NCTEFiles/Resources/Journals/CCC/0683-feb2017/CCCC0683Teaching.pdf.] It is an important document that asks us to rethink disability and academic accommodations. She starts by discussing “contemporary theories of disability to retheorize accommodation as the process of teaching itself.” Womack provides a history of disability law and American institutions of higher education, noting that students today must “pass substantial hurdles to qualify for accommodations” often at the risk of being stigmatized. Faculty, who may receive little institutional support, come to feel that they are the ones burdened by the process. Any resulting pedagogical changes are seen as affecting only the students with disabilities.

Womack argues that resistance to accommodation by university administration and faculty assumes that accommodations are an exception to a rule, to a best practice, or normal way of teaching. Womack states, “Ultimately, though, there is no normal, primary way of learning, only normalized methods made primary through frequent use. Material always changes as it moves from expert to novice. Every act of teaching is an accommodation because it creates certain conditions for students to learn and display learning.” Even though effective student learning means that the material is accessible, instructors have come to feel that “making material accessible to disabled students threatens academic rigor.”

Seen from another vantage point, inclusive teaching means eliminating barriers to learning, not eliminating intellectual challenges. Womack says, “Accommodation is the most basic act and art of teaching. It is not the exception we sometimes make in spite of learning, but rather the adaptations we continually make to promote learning.” She advocates accommodation of disabled students within a universal design framework.

“Universal Design is the design and composition of an environment so that it can be accessed, understood and used to the greatest extent possible by all people regardless of their age, size, ability or disability.” Universal design is good for everyone. In your classroom, Womack suggests, for example, considering guidelines for dyslexic and blind readers, working under the assumption that by creating documents that more students can read, more students will read. She warns, however, that universal design must be used as a process, not to negate the need for accommodations, but to start to negotiate the means to accessibility for all.

In my next post I will look at the second part of Womack’s article, which provides suggestions for creating accessible documents, engaging students by using “cooperative language” and building flexibility into your course to empower students. “If instructors see the syllabus through the lens of disability, then the question becomes not how policies protect a normative standard but how far they extend inclusion.”

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com

The Open Faculty Patchbook: An OER on Pedagogy

Posted on November 17, 2017 by Macie Hall

We are seeing an increased interest in Open Educational Resources (OER) in our library and among faculty. [See Consider the OER an Innovative Instructor blog post by Marian Feldman, November 7, 2016.] What is an OER? Wikipedia defines open educational resources as being “freely accessible, openly licensed text, media, and other digital assets that are useful for teaching, learning, and assessing as well as for research purposes.”

With the skyrocketing costs of textbooks used in higher education, colleges have responded to student demand by instituting programs to support the production of OER texts, especially for discipline-standard introductory courses. There is a listing of OER initiatives, resources, and projects on the SPARC (the Scholarly Publishing and Academic Resources Coalition) website. The Open Education Consortium also lists OER initiatives, including a list of open textbooks.

By happenstance, I came across an OER written by faculty for faculty on pedagogy. The Open Faculty Patchbook was created by faculty at Fleming College in Peterborough, Ontario. The Learning Design and Support Team at Fleming was tasked in 2016 with revising the faculty development model. Inspired by a presentation given by Robin DeRosa at the 2016 Open Education Conference in Richmond, Virginia, the team decided to create a how-to teach manual. They came up with the metaphor of a quilt, with each contributor creating a “patch” to add to the “community quilt of pedagogy.” Currently, there are 21 pieces describing how instructors do their work. Topics include cohort-based-learning, teaching within the Universal Design for Learning (UDL) framework, activating students’ background knowledge, formative assessments, facilitating deep learning, laboratory assessments, co-teaching, group work, and more.

The Patchbook was initially designed to cover the University of Michigan’s School of Education High Leverage Practices, which are described as “…the basic fundamentals of teaching. These practices are used constantly and are critical to helping students learn important content. The high-leverage practices are also central to supporting students’ social and emotional development. These high-leverage practices are used across subject areas, grade levels, and contexts. They are “high-leverage” not only because they matter to student learning but because they are basic for advancing skill in teaching.”

The patches in the Open Faculty Patchbook are relatively short, making this an easy to digest guidebook. It is also open to contributors for additional patches. At the end of the current book is the statement: “Future versions planned include ones focused on professional learning, digital pedagogy (online learning) and course design. If you would like to contribute, email ldsteam@flemingcollege.ca with the subject heading ‘I’m awesome. I want to add a Patch’. Your patch, when complete, would immediately appear on our WordPress site facultypatchbook.wordpress.com and be added to a Pressbook publication.”

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Open Faculty Patchbook

Teaching Transparently

Posted on November 7, 2017 by Macie Hall

Back in September 2015, The Innovative Instructor posted Do Your Students Understand the Assignment?, an article that examined the concept of transparent teaching. Transparent teaching helps students understand the why and how of their learning. Research from the University of Nevada, Las Vegas (UNLV) Transparency in Learning and Teaching Project (TILT) has shown that when students understand the task, its purpose, and the criteria for evaluating their work, they are more motivated and feel the work is more relevant. The TILT website has some excellent suggestions and resources for instructors, including examples of assignments from various disciplines presented in two versions, less transparent and more transparent, for comparison.

A recent post on Teaching Tidbits, a blog sponsored by the Mathematical Association of America to keep higher ed math faculty up on advances in educational research and pedagogical practices by providing “…quality, evidence-based ideas with high impact and low time commitment that can be used by a wide audience,” examined teaching with transparency. [October 24, 2017, How Transparency Improves Learning by Darryl Yong] Although the focus is on teaching college mathematics, the key points are applicable to a range of subjects.

Yong starts by citing the work done at UNLV, noting in particular the finding that underrepresented students experienced the greatest improvement in learning outcomes when transparent teaching methods were used. Yong speculates that transparent teaching helps to level the playing field for these students.

A key to teaching more transparently is to see things from your students’ vantage points. What would they find “bewildering, frustrating or alienating?” Being transparent does not mean that you don’t expect the work to be challenging rather that you will “engage your students in a productive struggle.”

Providing instructions in more than one format is helpful. For example, you should include information on assignments in writing on your syllabus, verbally in class, and again in written form in handouts to be sure that students aren’t missing important details.

Yong says, “The amount of transparency that you provide to students depends on their maturity and the level of the course. There are times when you don’t want to be explicit about everything. For example, you don’t want to constrain their creativity by priming them with examples, you want them to struggle with figuring out what the first step should be, or you want them to be more independent in their learning.”

He concludes the post with some suggestions on transparency for mathematics courses, but even these can be translated to more general use. The concepts are:

Be sure that students understand discipline-specific terminology.
Be clear about the tools, applications, and resources are students allowed to use for assignments and exams.
Explain why you have chosen a particular assignment, project, or type of exam. Connect these choices to their learning outcomes. Share strategies that successful students have used in the past for assignments and evaluations. Share rubrics when used for grading. Share examples of successful projects.
Tell students why you have chosen the pedagogical strategies you use to teach.
Start each class by highlighting a relevant current area of research and the people doing it. If your field has not been inclusive in the past, acknowledge that and “showcase women and people of color in these highlights to engage in counter-stereotyping.”

Teaching transparently will involve more planning and preparation for your course. It also means teaching intentionally. Improved learning outcomes and greater student satisfaction will make it worth your effort.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Source: Pixabay.com