Writing Course Learning Goals

Posted on August 10, 2017 by Macie Hall

Today’s post is timely—many instructors are putting together syllabi for fall courses. This year, Johns Hopkins’ faculty who teach undergraduates are being urged to include course learning goals in their syllabi. Mike Reese, Associate Dean and Director of the Center for Educational Resources (CER), and Richard Shingles, a lecturer in Biology and Pedagogy Specialist in the CER, and created an Innovative Instructor print series article as an aid, shared below. If you are looking for other information on creating effective syllabi, type syllabus in the search box for this blog to see previous articles on the topic. Another resource for writing course learning goals is Arizona State University’s free Online Objectives Builder. It runs instructors through a logical process for creating course goals and objectives. Take the short tutorial and you are on your way.

What are course learning goals and why do they matter?

Effective teaching starts with thoughtful course planning. The first step in preparing a course is to clearly define your course learning goals. These goals describe the broad, overarching expectations of what students should be able to do by the end of the course, specifically what knowledge students should possess and/or what skills they should be able to demonstrate. Instructors use goals to design course assignments and assessments, and to determine what teaching methods will work best to achieve the desired outcomes.

Course learning goals are important for several reasons. They communicate the instructor’s expectations to students on the syllabus. They guide the instructor’s selection of appropriate teaching approaches, resources, and assignments. Learning goals inform colleagues who are teaching related or dependent courses. Similarly, departments can use them to map the curriculum. Departmental reviews of the learning goals ensure prerequisite courses teach the skills necessary for subsequent courses, and that multiple courses are not unnecessarily teaching redundant skills.

Once defined, the overarching course learning goals should inform the class-specific topics and teaching methods. Consider an example goal: At the end of the course, students will be able to apply social science data collection and analysis techniques. Several course sessions or units will be needed to teach students the knowledge and skills necessary to meet this goal. One class session might teach students how to design a survey; another could teach them how to conduct a research interview.

A syllabus usually includes a learning goals section that begins with a statement such as, “At the end of this course, students will be able to:” that is followed by 4-6 learning goals clearly defining the skills and knowledge students will be able to demonstrate.

Faculty should start with a general list of course learning goals and then refine the list to make the goals more specific. Edit the goals by taking into consideration the different abilities, interests, and expectations of your students and the amount of time available for class instruction. How many goals can your students accomplish over the length of the course? Consider including non-content goals such as skills that are important in the field.

Content goal: Analyze the key forces that influenced the rise of Japan as an economic superpower.
Non-content goal: Conduct a literature search.

The following list characterizes clearly-defined learning goals. Consider these suggestions when drafting goals.

Specific – Concise, well-defined statements of what students will be able to do.
Measurable – The goals suggest how students will be assessed. Use action verbs that can be observed through a test, homework, or project (e.g., define, apply, propose).

Non-measurable goal: Students will understand Maxwell’s Equations.
Measurable goal: Students will be able to apply the full set of Maxwell’s Equations to different events/situations.

Attainable – Students have the pre-requisite knowledge and skills and the course is long enought that students can achieve the goals.
Relevant – The skills or knowledge described are appropriate for the course or the program in which the course is embedded.
Time-bound – State when students should be able to demonstrate the skill (end of the course, end of semester, etc.).

The most difficult aspect of writing learning goals for most instructors is ensuring the goals are measurable and attainable. In an introductory science course, students may be expected to recall or describe basic facts and concepts. In a senior humanities course, students may be expected to conduct deep critical analysis and synthesis of themes and concepts. There are numerous aids online that suggest action verbs to use when writing learning goals that are measurable and achievable. These aids are typically structured by Bloom’s Taxonomy – a framework for categorizing educational goals by their challenge level. Below is an example of action verbs aligned with Bloom’s Taxonomy.

Avoid vague verbs like “understand” or “know” because it can be difficult to come to consensus about how the goal can be measured. Think more specifically about what students should be able to demonstrate.

Here are examples of learning goals for several different disciplines using a common introductory statement. “By the end of this course, students will be able to do the following…

“Propose a cognitive neuroscience experiment that justifies the choice of question, experimental method and explains the logic of the proposed approach.” (Cognitive Science)
“Articulate specific connections between texts and historical, cultural, artistic, social and political contexts.” (German and Romance Languages and Literature)
“Design and conduct experiments.” (Chemistry)
“Design a system to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability.” (Biomedical Engineering)

Additional Resources
Bloom’s Taxonomy article. http://cer.jhu.edu/files/InnovInstruct-BP_blooms-taxonomy-action-speakslouder.pdf
Blog post on preparing a syllabus. http://ii.library.jhu.edu/2017/02/23/lunch-and-learn-constructing-acomprehensive-syllabus

Authors
Richard Shingles, Lecturer, Biology Department, JHU
Dr. Richard Shingles is a faculty member in the Biology department and also works with the Center for Educational Resources at Johns Hopkins University. He is the Director of the TA Training Institute and The Teaching Institute at JHU. Dr. Shingles also provides pedagogical and technological support to instructional faculty, postdocs and graduate students.
Michael J. Reese Jr., Associate Dean and Director, CER
Mike Reese is Associate Dean of University Libraries and Director of the Center for Educational Resources. He has a PhD from the Department of Sociology at Johns Hopkins University.

Lunch and Learn: Team-Based Learning

Posted on December 29, 2016 by Macie Hall

On Friday, December 16, the Center for Educational Resources (CER) hosted the second Lunch and Learn—Faculty Conversations on Teaching, for the 2016-1017 academic year. Eileen Haase, Senior Lecturer in Biomedical Engineering, and Mike Reese, Director, Center for Educational Resources, and Instructor in Sociology, discussed their approaches to team-based learning (TBL).

Eileen Haase teaches a number of core courses in Biomedical Engineering at the Whiting School of Engineering, including Freshmen Modeling and Design, BME Teaching Practicum, Molecules and Cells, and System Bioengineering Lab I and II, as well as being course director for Cell and Tissue Engineering and assisting with System Bioengineering II. She has long been a proponent of team work in the classroom.

In her presentation, Haase focused on the Molecules and Cells course, required for BME majors in the sophomore year, which she co-teaches with Harry Goldberg, Assistant Dean at the School of Medicine, Director of Academic Computing and faculty member, Department of Biomedical Engineering. The slides from Haase’s presentation are available here.

In the first class, Haase has the students do a short exercise that demonstrates the value of teamwork. Then the students take the VARK Questionnaire. VARK stands for Visual Aural Read/Write Kinesthetic and is a guide to learning styles. The questionnaire helps students and instructors by suggesting strategies for teaching and learning that align with these different styles. Haase and Goldberg found that 62% of their students were “multimodal” learners who will benefit from having the same material presented in several modes in order to learn it. In Haase’s class, in addition to group work, students work at the blackboard, use clickers, have access to online materials, participate in think-pair-share exercises, and get some content explained in lecture form.

Team work takes place in sections most Fridays. At the start of class, students take an individual, 10 question quiz called the iRAT, Individual Readiness Assurance Test, which consists of multiple-choice questions based on pre-class assigned materials. The students then take the test as a group (gRAT). Haase uses IF-AT scratch cards for these quizzes. Both tests count towards the students’ grades.

To provide evidence for the efficacy of team-based learning, Haase and Goldberg retested students from their course five months after the original final exam (99 of the 137 students enrolled in the course were retested). The data showed that students scored significantly better on the final exam on material that had been taught using team-based learning strategies and on the retest, retained significantly more of the TBL taught material.

Mike Reese, Director of the Center for Educational Resources and instructor in the Department of Sociology, presented on his experiences with team-based learning in courses that included community-based learning in Baltimore City neighborhoods [presentation slides]. His courses are typically small and discussion oriented. Students read papers on urban issues and, in class, discuss these and develop research methodologies for gathering data in the field. Students are divided into teams, and Reese accompanies each team as they go out into neighborhoods to gather data by talking to people on the street and making observations on their surroundings. The students then do group presentations on their field work and write individual papers. Reese says that team work is hard, but students realize that they could not collect and analyze data in such a short time-frame without a group effort.

Reese noted that learning is a social process. We are social beings, and while many students dislike group projects, they will learn and retain more (as Haase and Goldberg demonstrated). This is not automatic. Instructors need to be thoughtful about structuring team work in their courses. The emotional climate created by the teacher is important. Reese shared a list of things to consider when designing a course that will incorporate team-based learning.

Purpose: Why are you doing it? For Reese, teamwork is a skill that students should acquire, but primarily it serves his learning objectives. If students are going to conduct a mini-research project in a short amount of time, they need multiple people working collectively to help with data collection and analysis.
Group Size: This depends on the context and the course, but experts agree that having three to five students in a group is best to prevent slacking by team members.
Roles: Reese finds that assigning roles works well as students don’t necessarily come into the course with strong project management skills, and projects typically require a division of labor. It was suggested that assigning roles is essential to the concept of true team-based learning as opposed to group work.
Formation: One key to teamwork success is having the instructor assign students to groups rather than allowing them to self-select. [Research supports this. See Fiechtner, S. B., & Davis, E. A. (1985). Why some groups fail: A survey of students’ experiences with learning groups. The Organizational Behavior Teaching Review, 9(4), 75-88.] In Reese’s experience assigning students to groups helps them to build social capital and relationships at the institution beyond their current group of friends.
Diversity: It is important not to isolate at-risk minorities. See: Heller, P. and Hollabaugh, M. (1992). Teaching problem solving through cooperative grouping. American Journal of Physics, 60 (7), 637-644.
Ice Breakers: The use of ice breakers can help establish healthy team relationships. Have students create a team name, for example, to promote an identity within the group.
Contracts: Having a contract for teamwork is a good idea. In the contract, students agree to support each other and commit to doing their share of the work. Students can create contracts themselves, but it is best if the instructor provides structured questions to guide them.
Persistence: Consider the purpose of having groups and how long they will last. Depending on learning goals, teams may work together over an entire semester, or reform after each course module is completed.
Check-ins: It is important to check in with teams on a regular basis, especially if the team is working together over an entire semester, to make sure that the group hasn’t developed problems and become dysfunctional.
Peer Evaluation: Using peer evaluation keeps a check on the students to ensure that everyone is doing a fair share of the work. The instructor can develop a rubric, or have students work together to create one. Evaluation should be on specific tasks. Ratings should be anonymous (to the students, not the instructor) to ensure honest evaluation, and students should also self-evaluate.

In the discussion that followed the presentation, mentoring of teams and peer assessment were key topics. Several faculty with experience working with team-based learning recommended providing support systems in the form of mentors and or coaches who are assigned to the groups. These could be teaching assistants or undergraduate assistants who have previously taken the course. Resources for team-based learning were mentioned. CATME, “which stands for ‘Comprehensive Assessment of Team Member Effectiveness,’ is a free set of tools designed to help instructors manage group work and team assignments more effectively.”

Doodle was suggested as another tool for scheduling collaborative work. Many are familiar with the Doodle poll concept, but there are also free tools such as Connect Calendars and Meet Me that can be used by students.

An Innovative Instructor print article, Making Group Projects Work by Pam Sheff and Leslie Kendrick, Center for Leadership Education, August 2012, covers many aspects of successful teamwork.

Another resource of interest is a scholarly article by Barbara Oakley and Richard Felder, Turning Student Groups into Effective Teams [Oakley, B., Felder, R.M., Brent, R., Elhajj, I. Journal of student centered learning, 2004]. “This paper is a guide to the effective design and management of team assignments in a college classroom where little class time is available for instruction on teaming skills. Topics discussed include forming teams, helping them become effective, and using peer ratings to adjust team grades for individual performance. A Frequently Asked Questions section offers suggestions for dealing with several problems that commonly arise with student teams, and forms and handouts are provided to assist in team formation and management.”

If you are an instructor on the Homewood campus, staff in the Centerfor Educational Resources will be happy to talk with you about team-based learning and your courses.

Macie Hall, Senior Instructional Designer
Center for Educational Resources

Image Sources: Lunch and Learn logo by Reid Sczerba, presentation slides by Eileen Haase and Mike Reese

To Curve or Not to Curve

Posted on May 13, 2013 by Macie Hall

A version of this post appeared in the print series of The Innovative Instructor.

Instructors choose grading schemes for a variety of reasons. Some may select a method that reflects the way they were assessed as students; others may follow the lead of a mentor or senior faculty member in their department. To curve or not to curve is a big question. Understanding the motivations behind and reasons for curving or not curving grades can help instructors select the most appropriate grading schemes for their courses.

Curving defines grades according to the distribution of student scores. Grades are determined after all student scores for the assignment or test are assigned. Often called norm-referenced grading, curving assigns grades to students based on their performance relative to the class as a whole. Criterion-referenced grading (i.e., not curving) assigns grades without this reference. The instructor determines the threshold for grades before the assignment is submitted or the test is taken. For example, a 92 could be defined as the base threshold for an A, regardless of how many students score above or below the threshold.

Choosing to curve grades or use a criterion referenced grading system can affect the culture of competition and/or the students’ sense of faculty fairness in a class. Curving grades provides a way to standardize grades. If a department rotates faculty responsibility for teaching a course (such as a large introductory science course), norm-referenced grading can ensure that the distribution of grades is comparable from year-to-year. A course with multiple graders, such as a science lab that uses a fleet of graduate students in the grading, may also employ a norm referencing technique to standardize grades across sections. In this case, standardization across multiple graders should begin with training the graders. Curving grades should not be a substitute for instructing multiple graders how to assign grades based on a pre-defined rubric (The Innovative Instructor: “Calibrating Multiple Graders”).

In addition to standardizing grades, norm-referenced grading can enable faculty to design more challenging assignments that differentiate top performers who score significantly above the mean. More challenging assignments can skew the grade distribution; norm-referenced grading can then minimize the impact on the majority of students whose scores will likely be lower.

A critique of curving grades is that some students, no matter how well they perform, will be assigned a lower grade than they feel they deserve. Shouldn’t all students have an equal chance to earn an A? For this reason, some instructors do not pre-determine the distribution of grades. The benefit of using a criterion-referenced grading scheme is that it minimizes the sense of competition among students because they are not competing for a limited number of A’s or B’s. Their absolute score, not relative performance, determines their grade.

There are multiple ways to curve grades.

I. The Bell Curve

Normalizes scores using a statistical technique to reshape the distribution into a bell curve. An instructor then assigns a grade (e.g., C+) to the middle (median) score and determines grade thresholds based on the distance of scores from this reference point. A spreadsheet application like Excel can be used to normalize scores. CER staff can assist instructors in normalizing scores.

II. Clumping

The instructor creates a distribution of the scores and identifies clusters of scores separated by breaks in the distribution, then uses these gaps as a threshold for assigning grades.

III. Quota Systems

Often used in law schools, the instructor pre-determines the number of students who can earn each grade. The instructor applies these quotas after rank ordering student scores.

IV. Criterion-reference grading

Using a pre-determined scale, assessments are based on clearly defined learning objectives and grading rubrics so students know the instructor’s expectations for an A, B, C, etc.

During the 2011 Robert Resnick Lecture at Johns Hopkins, Carl Wieman, Nobel Laureate and Associate Director for Science at the President’s Office of Science and Technology, argued that most instructors are not trained to create valid assessments of student learning. Curving can be used as a tool to adjust grades on a poorly designed test, but consistent use of curving should not be a substitute for designing assessments that accurately assess what the instructor wants students to learn by the end of the course. CER staff are happy to talk to faculty about defining learning objectives and/or strategies for designing challenging and accurate student assessment instruments.

Additional Resources

• Campbell, C. (2012). Learning-centered grading practices. Leadership. 41(5), 30-33

• Jacobson, N. (2001). A method for normalizing students’ scores when employing multiple graders. ACM SIGCSE Bulletin. 33(4), 35-38.

• Joe Champion’s Grading Transformation Spreadsheet. This spreadsheet automatically curves students’ scores after the instructor copies the scores into the spreadsheet and sets a variable defining the amount of curve.

Michael J. Reese, Associate Director
Center for Educational Resources

The Innovative Instructor

Pedagogy – Best Practices – Technology

Tag Archives: Mike Reese

Writing Course Learning Goals

Lunch and Learn: Team-Based Learning

To Curve or Not to Curve