Usage of artificial intelligence (AI) has spiked dramatically across various industries, revolutionizing the way business operates and interacts with customers. The OpenAI API is one such powerful tool that enables developers to integrate the state-of-the-art language models into their models easily. It provides access to advanced language models GPT-3 and GPT-4 which can perform a wide range of natural language processing tasks like text generation, translation, and summarization.
In this blog, we will see the network communications that happens in the background when we call the OpenAI API and how we can call the API using Keysight’s BreakingPoint Superflow.
Network Traffic Analysis
The ATI team in Keysight has analyzed the network traffic of OpenAI API call and found some interesting insights, which can be helpful for other researchers.
When we call the OpenAI API using `curl` or python’s `requests` library, it generally sends a POST request to the OpenAI API endpoint. This request includes different HTTP headers and a JSON payload. It utilizes TLS 1.3 for encryption and HTTP/1.1 for communication.
Let’s break down the traffic in detail:
Request Components:
Figure 1: Sample OpenAI API HTTP Request.
- Request Line
When we call the OpenAI API, in the decrypted traffic the HTTP packet contains the “/v1/chat/completions” in the Request Line, which indicates the client is making a POST request to the “/v1/chat/completions” endpoint of the OpenAI API using HTTP version 1.1. This endpoint is specifically designed for requesting chat completions from the OpenAI Large Language Models (LLMs).
- Headers
The request contains some specific headers like-
- Host: api.openai.com (indicating the request is directed towards the OpenAI API server)
- Connection: Keep-Alive
- User-Agent: python-requests/2.25.1 (when the API is called using python’s “requests” library)
- Accept: */*
- Accept-encoding: gzip, deflate
- Content-Type: application/json
- Authorization: Bearer <OpenAI_API_Key> (It contains the bearer token i.e., the OpenAI API Key to authenticate the client. Here the “Bearer” indicates that possession of the token grant access.)
- Content-Length: <Length of the JSON payload>
- Payload
Figure 2: OpenAI API Request Payload.
The request payload contains the actual JSON-formatted data that the client wants to send to the OpenAI Server. In contains the following information:
- Model: The OpenAI Large Language Model (LLM) to be used for the completion task.
- System Prompt: Used to set the behavior, rules or context for the assistant. It provides instructions that define how the assistant should respond throughout the conversation.
- User Prompt: The input or question from the end-user interacting with the assistant. It is the primary content to which the assistant responds.
Response Components:
After the successful authentication and processing of the API request, the OpenAI server responds with a 200 Ok HTTP response which looks like below –
Figure 3: Sample OpenAI API HTTP Response
The response header includes:
- openai-organization: user-atpoysm5au5t2tDj7MP4lmvP
- openai-processing-ms: 848
- openai-version: 2020-10-01
- Server: cloudflare
These headers provide some important information such as Organization ID associated with the request, the processing time in milliseconds, the verion of OpenAI API used and the server handling the request. This information is valuable for fingerprinting, as it helps to identify the server and client characteristics in network traffic analysis.
This response also contains the answer of the user prompt from the LLM in JSON format like below –
Figure 4: OpenAI API Response Payload
OpenAI API Traffic Simulation in Keysight ATI
At Keysight Technologies, our Application and Threat Intelligence (ATI) team, researchers have examined the traffic pattern of OpenAI API call and added its support in ATI-2024-12 Strike Pack release on June 20, 2024.
We have added 2 new OpenAI API superflows: 1-arm (Client-side simulation) and 2-arm (both Client and Server-side simulation) –
- OpenAI API Call
This simulates the scenario of making an API request to the OpenAI API server using the GPT-3.5 Turbo language model to get the completion of a user prompt. Here, the client sends a POST request to the OpenAI API server with the specified system and user prompts, and the server responds with the completion of the user prompt using the specified language model. Here the HTTP transaction is encrypted using TLS1.2.
Figure 5: OpenAI API Call 2-arm Superflow in BPS
- ClientSim OpenAI API Call
This simulates the scenario of making an API request to the actual OpenAI API server using the GPT-3.5 Turbo language model to get the completion of a user prompt. Here, the client sends a POST request to the OpenAI API server with the specified system and user prompts over TLS1.2.
Figure 6: OpenAI API Call 1-arm Superflow in BPS
Note: To configure the “ClientSim OpenAI API Call” (1-arm) superflow in BreakingPoint Systems, the gateway IP address of “api.openai.com” must be mentioned as the “Base IP Address” of the “IPV4 EXTERNAL HOSTS” which is present inside “Network Neighborhood” configuration as shown below –
Figure 7: Example of Network Neighborhood Configuration for OpenAI API 1-arm superflow in BPS
While OpenAI/ChatGPT can be useful they are still a prohibited tool by many companies and most government entities. Policy and technical systems must be in place to prevent usage and it is vital to confirm this via test using BreakingPoint or another test tools.
Leverage Subscription Service to Stay Ahead of Attacks
Keysight’s Application and Threat Intelligence subscription provides daily malware and bi-weekly updates of the latest application protocols and vulnerabilities for use with Keysight test platforms. The ATI Research Centre continuously monitors threats as they appear in the wild. Customers of BreakingPoint now have access to attack campaigns for different advanced persistent threats, allowing BreakingPoint Customers to test their currently deployed security control’s ability to detect or block such attacks.