[Guest post by Mike Reese, Associate Dean of the Center for Teaching Excellence and Innovation & Associate Teaching Professor of Sociology, Johns Hopkins University]

My colleagues and I are regularly invited  to speak with faculty about the impact of generative artificial intelligence (AI) on teaching and learning including leading workshops about the topic. A faculty friend suggested over lunch, “Let’s stop talking about it. Help us start using it!” That was the genesis for the workshop, Prompt Engineering for Instructors – a workshop hosted by the Center for Teaching Excellence and Innovation (CTEI) in February in which we modeled prompting strategies for instructors developing course materials.  The workshop focused on text-based, large language models (LLMs) rather than other tools like DALL·E,  a generative AI tool which generates images.

Before starting, we reviewed important considerations when using generative AI applications for teaching:

• FERPA – You should not enter any personally-identifiable student data into non-university approved tools (e.g., general subscription to OpenAI’s ChatGPT). Doing so would be a violation of FERPA.
• Access – The most powerful models are not surprisingly fee-based. If you ask your students to work with generative AI tools, remember some students may have access to better models than others if they can afford to pay a subscription fee.
• Resources –  While these tools are powerful, they are also resource intensive (e.g., power and water usage). Consider the need for using these tools with the environmental impact.
• Knowledge Cutoff – The LLMs you use were trained on sources up to a certain date, known as the knowledge cutoff. Asking questions or generating assignments about events after that knowledge cutoff are likely to lead to hallucinations unless the LLM is provided additional information when generating its response.

I shared some general strategies that I use when prompting LLMs. The following strategies are inspired by Jules White’s Prompt Engineering workshops on Coursera and Jordan Wilson’s Everyday AI’s Prime-Prompt-Polish workshop.

• Be specific and detailed – Providing more specificity and detail in your prompt will focus the response. The LLM will follow your directions literally so make sure you communicate exactly what you want.
• Give it a role / Motivate it / Provide context – Consider telling the model who you want it to respond as (e.g., your role!) and include helpful context (e.g., information about your course!) to inform its response. “You are a college faculty member teaching Introduction to Sociology…”
• Provide examples – This is often called one-shot or few-shot prompting. For example, when asking a LLM to draft homework problems, I include examples of problems from previous years.
• Optional: Describe desired output/format – Describe the format for the response (e.g., create a rubric in a table format with the criteria in the rows and rating categories in the columns).
• Optional: Intentionally choose which LLM to prompt – Each LLM has its own personality. It takes time to learn these, but as you do, you may find some models are better for specific tasks.
• Optional: Suggest (iterative) improvements – If you don’t like the response, repeat with follow up prompts explaining what you don’t like and improvements to make.

Prompt Examples

Below are the prompts demonstrated during the workshop with notes on each to explain the principles above. Generally, we used Anthropic’s Claude 3.5 Sonnet.

Writing Learning Objectives

We started by asking Claude to write learning objectives for an introductory sociology course that I teach. The purpose of these three prompts was to show how providing more specificity and detail generated different responses.  The last response incorporates assigning Claude a role: me, a faculty member teaching a college-level sociology course.

• Prompt 1: Develop learning objectives for a sociology course.
• Prompt 2: Develop learning objectives on culture for a sociology course.
• Prompt 3: You are a faculty member teaching an introductory-level sociology course at a college. One unit is on culture. Write three objectives for a unit on culture at each of the lowest levels of Bloom’s taxonomy: Remembering, understanding, applying. The format should start with, “By  the end of this unit, students will be able to,” then list the objectives in bullet format. The action verb for the learning objective should not include unmeasurable words including understand, know, learn, etc.

Creating Homework Questions

In the next example, I showed how I use LLMs to create homework and test questions. This has been one of the most productive uses of generative AI in my teaching. I write new homework problems each year for my social statistics course. Using a model to draft initial questions has cut that time from 6 hours to 2 hours for each homework.

The examples below show how to use one-shot prompting (i.e., giving the LLM a past homework question). Once it gives me several options, I choose the one I like best. I may make some edits before I solve the problem to see if I think it is evaluating students at the appropriate level and on the objectives I intended.

Prompt: You are a professor teaching Introduction to Social Statistics at a college. You want to create a homework on confidence intervals. Please create 3 questions based on the following question in brackets that assess students on the same statistical concept but a different sociological context.

[The JHU police force has been debated at JHU. You conduct a random survey of Charles Village residents about whether they think JHU should have its own police force: 57% are for it and 43% are against. Construct a 99% confidence interval for the proportion of people who are for the police force if the sample size is a) 500 residents and b) 50 residents. Show your work. For each case indicate if you would be willing to suggest if the residents of Charles Village are for or against the JHU police force.]

Writing Assignment

I also demonstrated how to develop a writing assignment for my introductory sociology course. The interesting part about this example was that the response did not follow the overall word limit I requested. It created a homework prompt with word limits associated with different sections that summed to more than 300 words. This is an example of hallucination.

Prompt: You are a professor teaching Introduction to Sociology at a college. You want to create a homework prompt in which students need to summarize the main points of an opinion essay from a newspaper and then apply sociological concepts to it. Students should use those concepts to provide a critique of the main argument including both strong arguments and weak arguments. Using the following essay in brackets, create a prompt for the homework that should be no more than 300 words. 

[Essay Example not provided for length and copyright reasons]

Rubric

In this example, we created a rubric for the previous writing assignment in my introductory sociology course. The two prompts show different responses when you provide more detail and communicate the format for the final rubric.

• Prompt 1: Develop a rubric for this assignment.
• Prompt 2: Rewrite the rubric using the same criteria, but include the following ratings with the associated points: Excellent (8-10 points), Good (5-7 points), Needs Improvement (1-4 points). Format it as a table.

Adapt the examples above for your courses. Share tips, strategies, and prompt examples from your course in the comments below!

Mike Reese
Associate Dean of the Center for Teaching Excellence and Innovation and Associate Teaching Professor in Sociology, Johns Hopkins University

Image Source:  flyalone – stock.adobe.com, gguy – stock.adobe.com, terovesalainen – stock.adobe.com