Application of ChatGPT in software testing

What is ChatGPT?

ChatGPT (Generative Pre-trained Transformer) is a large-scale language model developed by OpenAI, based on the transformer architecture. It is designed to generate human-like responses to natural language input, making it useful for various applications such as chatbots, conversational agents, and question-answering systems.

Users can fine-tune the model on their specific task or application by providing additional training data and defining the specific output format required. The fine-tuning process adapts the model to the specific task and improves its performance, making it more accurate and relevant to the specific domain.

Why is ChatGPT new buzz word in tech industry:

One of the main reasons for the curiosity around ChatGPT is that it represents a major breakthrough in the field of NLP. It has shown that it is possible to train a single model to perform multiple NLP tasks at a high level of accuracy, which is a significant improvement over previous approaches that required specialised models for each task.

Another reason for the hype around ChatGPT is that it has the potential to revolutionise the way we interact with computers and digital devices. With its ability to generate natural language responses, ChatGPT could be used to create more realistic and engaging chatbots, virtual assistants, and other types of conversational interfaces. This could lead to a wide range of applications in areas such as customer service, education, and healthcare.

Possible use cases of ChatGPT for software testing:

Test Data Generation: ChatGPT can be trained to generate realistic test data that can be used in automated testing. For instance, if you need to test an application that requires user input, you can use ChatGPT to generate a large number of realistic user inputs, which can be fed into the test automation framework.

Test Case Generation: ChatGPT can be used to generate test cases based on the requirements of the application under test. This can be especially helpful in cases where there are numerous test cases, and manual test case creation can be tedious and time-consuming. We can not only generate generic test cases but also very specific test scenario based on the query requirement and details being fed as input query. ChatGPT also let us create test case from requirements in any customised format or framework like BDD or karate api test.

Test documentation and artefact creation: ChatGPT can be very useful in creating various test documentation and artefacts like test strategy & test plans. We can write very specific test strategies and test plan if we provide adequate details for creating customised testing artefacts.

Automation code review and correction: Using ChatGPT We can easily write new piece of code or review existing piece of code to improve it. Instead of searching stack overflow or other forum to find answer to any technical query ChatGPT can be easily trusted to become one stop shop for all technical queries and help. Lets see below example where we get a clean piece of code to implement visual testing using wdio. We not only getting entire working code but full explanation of code as well, Also the code generated is not generic in the sense you can write custom query and ask chatGPT to find error or suggest fix and it does it effortlessly .

Getting Point of view and tool comparisons : ChatGPT can be used to create point of view or perform quick comparisons between various tools and technologies. In below example we try to successfully get different open source libraries for accessibility testing without any need to spend team doing POC’s or comparisons.

Writing Xpath for UI automation : QE team spends a lot of team writing Xpath for web automation testing, ChatGPT can help to automate this task of automation engineers. ChatGPT can create with ease very specific and customised Xpath for any web page like in below example we created 4 different valid Xpath for a particular web element .

Above are just few of endless use cases that are possible using ChatGPT. Below is the Nodejs implementation of openAI API which can be integrated with any framework or tool.

const { Configuration, OpenAIApi } = require("openai");

        const configuration = new Configuration({
            apiKey: 'YOUR_API_KEY',
          });

          const openai = new OpenAIApi(configuration);
          const response = await openai.createCompletion({
            model: "text-davinci-003",
            prompt: ask,
            temperature: 1.0,
            max_tokens: 3917,
            top_p: 1,
            frequency_penalty: 0,
            presence_penalty: 0,
          });

          var responsebody = response['data'].choices[0]
         return responsebody

Overall, ChatGPT is a powerful tool that can be used to generate natural language responses, answer questions, and carry out various tasks in a conversational manner, making it a valuable tool for many automation tasks and process. With easy integration API coming to market we will soon see tools and services built on top of ChatGPT leading the new wave of change in testing industry.