Introduction to Generative AI

What is generative artificial intelligence (GenAI)?

Put simply, GenAI is any system which uses AI to ‘generate’ content on a user’s behalf. Where data analytics and AI had previously been focused on analysing text, images, and speech, GenAI generates new content based on the user’s question, query, instruction or prompt. Thus, it can generate content tailored to the user’s needs, requirements or instructions. The content it generates can be in the form of explanations, plans, process descriptions, questions, images, or conversational dialogue. However, GenAI tools do not always generate fully-correct answers and those using it are advised to read about the risks and restrictions on its usage.

While there are many definitions of GenAI, the UNESCO definition provides a useful starting point:

... an artificial intelligence (AI) technology that automatically generates content in response to prompts written in natural-language conversational interfaces

UNESCO, 2023, page 8

As noted, GenAI is a subset of Artificial Intelligence (AI) and thus it is important to explore how we define AI. While there is some dispute about what is meant by 'intelligence', there have been significant developments in AI technologies over recent years and the European Union (EU) Artificial Intelligence Act (2024) defines an AI system as:

a machine-based system designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments

EU Artificial Intelligence Act 2024, page 46

It is worth noting that while the UNESCO definition of GenAI refers to ‘prompts written’, this is a feature that has already being superseded by new GenAI technologies that can respond to audio and visual prompts. This highlights the rapid evolution of these technologies.

Figure 1 illustrates how GenAI draws from Machine Learning (defined as a set of algorithms that attempt to model the underlying patterns or distributions in data) and Deep Learning.  A subset of machine learning, Deep Learning is based on using so-called artificial neural networks to find patterns in training data. They are called ‘deep’ because multiple layers of nodes and activation functions are used to explore the data.

Fig. 1: GenAI as a subarea of AI.

How does GenAI work?

One of the reasons that GenAI has come to the fore recently is the emergence of Large Language Models (LLMs) which were enabled by the development of the transformer model by researchers at Google in 2017.

LLMs are used to determine statistical relationships from text-based documents and thus develop an understanding of language and how to generate it. Techniques such as fine-tuning, and prompt engineering can be used to enable the LLM to accomplish specific tasks.

When determining the statistical representation of language, early LLMs focused on nearby words. For example, if an LLM was processing the word ‘work’, it would scan all the available text and note all the words that are near ‘work’. It would then use this to predict what word, in a given situation, follows the word ‘work’.

In the transformer model, attention is also given to the sequence of words in a sentence, thus every sentence with the word ‘work’ is analysed and the importance of different words in a sequence is noted, allowing the software to capture patterns of language use more accurately. However, the fact that an LLM generates its outputs based on predictions, it can also be considered as having fabricated the output and this has led to so called ‘hallucinations’ when the LLM invents untrue or fictional information.

What can GenAI do?

 GenAI can produce various types of content such as text, imagery, code, audio and video.

To illustrate, the explanation below was generated by Microsoft Copilot (a GenAI tool) in response to the prompt ’what types of content can GenAI create’:

Generative AI, also known as GenAI, allows users to input a variety of prompts to generate new content, such as text, images, videos, sounds, code, 3D designs, and other media. It “learns” and is trained on documents and artifacts that already exist online. The technology has evolved significantly, and today, generative AI can create a wide range of novel content, including images, video, music, speech, text, software code, and product designs (Microsoft Copilot, May 25 version) [Large Language Model]).

Examples of content that can be generated by GenAI, along with potential uses in higher education are shown in Figure 2:

Fig. 2: Types of content and examples of GenAI use in higher education