You are here Resources > Generative AI Hub > Examples from the Disciplines

Using ChatGPT in a Biopharmaceutics workshop to enhance critical thinking skills

• Dr Deirdre D'Arcy Dr Deirdre D’Arcy is a lecturer in Pharmaceutics and Pharmaceutical Technology in the School of Pharmacy and Pharmaceutical Sciences in Trinity.

Context

A Generative Pretrained Transformer (GPT) is a generative artificial intelligence (GenAI) tool referring to a large language model (LLM) which uses machine learning to generate text outputs which resembling human text communication (Briganti, G.,2023). Since the launch of ChatGPT in November 2022, there has been understandable concern from the academic community about the use and abuse of such GenAI tools by students, in particular in assessment integrity. However, there is also great interest in exploiting the potential of GenAI tools in healthcare and education (Sallam, M., 2023) with suggestions for its use in higher education including stimulating critical thinking (Riley, S., and Cramblet Alvarez, L., 2023).

This exercise related to a biopharmaceutics workshop in year 4 of the 5-year Pharmacy (Integrated) Programme. In this context, biopharmaceutics relates to the study of formulation design of pharmaceutical products with a focus on assessing how the medicinal product releases the drug in the patient. Assessment approaches need to account for relatively large class sizes with multiple professional assessments, generally comprising 65-75 students. The workshop and assessment were conducted in February-March 2023.

What was your goal in utilising GenAI as part of the teaching process?

In using GenAI within this biopharmaceutics workshop my goal was to support students to reflect critically on, and analyse, the outputs of ChatGPT and to consider the need for effective prompt engineering in order to optimise use of this GenAI tool.

How did you use GenAI to enhance teaching, learning and/or assessment?

A new Biopharmaceutics workshop was introduced to students which considered specific technical methods used to investigate drug release, especially supporting efficient development of generic medicinal products. Part 2 of the workshop focused on use of ChatGPT to provide information on regulatory requirements for specific technical criteria for gaining a “biowaiver”, or waiver of clinical studies for a medicinal product, depending on demonstration of certain biopharmaceutical properties.

At the time of the workshop there was some media focus on academic integrity and use of GenAI tools such as ChatGPT, with university policies on their use still in their infancy. To ensure that students would not be concerned about being registered users with OpenAI, they were not required to engage directly with the platform. Instead, students were provided with three ChatGPT input queries (prompts) generated by the academic and the ChatGPT (v 3.5) outputs to each prompt. The regulatory guidance documents with the correct information were also provided to the students. It was emphasised to students in the workshop guidance that pharmacists in all professional roles are responsible for interpreting, generating and providing information, and therefore it is important to be familiar with these GenAI tools in order to ensure their safe use.

The relevant assessment was in two sections:

1. In section 1, students were provided with two critical reflection statements, requiring selection of the statement which best reflected each ChatGPT output. While each of the ChatGPT outputs provided was well-structured with some correct information, each contained either some erroneous information or information that was too vague to address the requirements. Due to these clear shortcomings, for this particular example, the attribution of the “correct” critical reflection statement was unambiguous once the source regulatory material was consulted.
2. Section 2 of the assessment involved students suggesting a new prompt to retrieve a more precise or accurate answer, along with a reasoned explanation for their choice of terms/phrases in the prompt. These prompt suggestions were assessed directly based on their reasoning, and not through inputting to the GenAI platform and assessing outputs.

What were the outcomes of using GenAI in this way?

An optional, anonymous feedback survey on workshop quality was conducted after the workshop . Of 27 (ex 66) responses received, 24 agreed/strongly agreed with the statement “The ChatGPT exercise was helpful in prompting me to critically reflect on using such an artificial intelligence/natural language processing tool in practice”. In terms of workshop design going forward, 24 respondents also considered that it would be somewhat/very beneficial to include an opportunity in the workshop to input their prompt to ChatGPT and reflect on the generated output. Comments concerning the ChatGPT element of the workshop included positive feedback on providing an opportunity to learn about these tools rather than being advised against their use.

What did you learn as part of this process and is there anything you would do differently?

Overall, the workshop was a very positive experience, enabling an applied introduction to the use of ChatGPT. This assignment was given to a relatively large class size, was of a low-stakes nature and at a very busy time of year for these year 4 students (and academics). It was therefore essential that the assessment method employed would be suitably concise, while being of an adequate (QQI level 8) standard. The provision of critical reflection statements could be the subject of debate, however given the novel nature of the exercise it was considered a valuable teaching and learning aid to scaffold such critical thinking. The use of prompt design as an assessment approach was a particularly efficient method of assessing knowledge application and reasoning. Going forward, given the rapidly evolving field of use of GenAI in education, theoretical concepts in the nascent field of prompt engineering will be included (e.g. Lo, L.S., 2023).