Complete Unreal Engine plugin for the OpenAI API

This plugin is a comprehensive Unreal Engine wrapper for the OpenAI API. It supports all OpenAI endpoints, including:

All requests are available in both C++ and Blueprints:

void AAPIOverview::CreateImage()
{
    auto* Provider = NewObject<UOpenAIProvider>();
    Provider->SetLogEnabled(false);
    Provider->OnRequestError().AddUObject(this, &ThisClass::OnRequestError);
    Provider->OnCreateImageCompleted().AddLambda(
        [](const FImageResponse& Response)
        {
            FString OutputString{};
            Algo::ForEach(Response.Data, [&](const FString& Data) { OutputString.Append(Data).Append(LINE_TERMINATOR); });
            UE_LOG(LogAPIOverview, Display, TEXT("%s"), *OutputString);
        });

    FOpenAIImage Image;
    Image.Prompt = "Tiger is eating pizza";
    Image.Size = UOpenAIFuncLib::OpenAIImageSizeToString(EImageSize::Size_512x512);
    Image.Response_Format = UOpenAIFuncLib::OpenAIImageFormatToString(EOpenAIImageFormat::URL);
    Image.N = 2;

    Provider->CreateImage(Image, Auth);
}

Supported Unreal Engine Versions

Unreal Engine 5.2

Installation

Marketplace link

Complete OpenAI API plugin

C++

Create a new C++ project in Unreal Engine.
Create a Plugins folder at the root of your project.
Add the OpenAI plugin to your Unreal Engine Plugins folder.
The easiest way to add plugins is by adding the current plugin as a submodule. Use the following command in the root folder of your project:

git submodule add https://github.com/life-exe/UnrealOpenAIPlugin Plugins/OpenAI

Alternatively, you can download the source code from the current repository and copy the files to your Plugins folder.
When complete, your Unreal Engine project structure should resemble the following:

Do the authentication steps
Generate Visual Studio project files.
Build your project and run Editor.
In the Editor, navigate to Edit->Plugins. Find and activate the OpenAI plugin.
Restart the editor.
Start using the plugin.

Blueprints

Create a blueprint project.
Create a Plugins folder in the root of your project.
Download precompiled plugin from the Releases.
Unzip plugin to the Plugins folder, specifically to Plugins/OpenAI.
Do the authentication steps
Run the Editor <YourProjectName>.uproject
In the Editor, navigate to Edit->Plugins. Find and activate the OpenAI plugin.
Restart the editor.
Start using the plugin.

Authentication

Create an OpenAI account
Generate and store your API Key: sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Record your Organization ID: org-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
At the root of your Unreal Engine project, create a file named OpenAIAuth.ini with the following content:

APIKey=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
OrganizationID=org-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Once completed, your Unreal Engine project structure might look like this:

Actually you can left OrganizationID empty. It doesn't affect on auth.

Finally, compile your project and launch the Unreal Editor.

Examples of Usage

Widget example

This provides a basic example of request usage with Blueprints.

Make sure that plugin content activated:

Open the plugin content folder.
Open the level:

Run the game.
Test the API with the widget examples:

Feel free to modify requests in the widget code to test the API with different parameters:

Unreal Editor Chat-GPT

This is the Chat-GPT implementation with token streaming support.

Open the plugin content folder.
Navigate to the EditorWidgets folder.
Right-click on the ChatGPT editor utility widget:

Start chatting:

You can save the chat history to a file using the Dump button. The specific location where the history is saved can be checked in the chat or logs:

Blueprint Nodes Overview

This blueprint demonstrates all available nodes.

Open the plugin content folder:

Open BP_APIOverview blueprint.
Check the available nodes: functions and structs:

There are also several nodes that could be useful in other projects. Feel free to copy and paste the plugin code if you need them:

C++ example

Open Plugins\OpenAI\Source\OpenAI\Private\Sample\APIOverview.cpp actor
Uncomment the function that executes the request you want to test. Navigate to the function and adjust all the request parameters as needed.
Compile your project and open Unreal Editor.
Drop the AAPIOverview actor into any level.
Run the game.
Check the Output Log.

Plugin structure:

OpenAIModule - core classes.
OpenAIEditorModule - Chat-GPT editor utility widget.
OpenAITestRunnerModule - unit tests.

Documentation

I highly recommend reading the OpenAI documentation for a better understanding of how all requests work:

In addition plugin code has its own documentation generated with help of Doxygen:

Documentation

You can generate the plugin documentation locally by following these steps:

Update all submodules. You can use the batch script located at the root of the plugin folder: update_submodules.bat
Ensure that Doxygen is installed on your system.
Ensure that Python is installed on your system.
Generate the documentation using the batch script at the root of the plugin folder: generate_docs.bat
After the generation process, the documentation will be available in the Documentation folder at the root of the plugin: Documentation\html\index.html

Tests

Unit tests are available in the OpenAITestRunnerModule. You can initiate them using the Session Frontend:

Packaging

Verify that your .uproject file contains the following:

{
    "Name": "OpenAI",
    "Enabled": true
}

Package the project as you always do (kudos if you use a CI solution like Jenkins).
You can handle authentication in your project as you see fit. However, if you choose to use a file-based method, such as in a plugin, please remember to include the OpenAIAuth.ini file in your packaged folder Build/Windows/<YourProjectName>/OpenAIAuth.ini If you are having problems loading the file, please check the logs to see where it might be located:

LogOpenAIFuncLib: Error: Failed loading file: C:/_Projects/UE5/OpenAICpp/Build/Windows/OpenAICpp/OpenAIAuth.ini

Limitations

Chat GPT-4 models are not available for everyone via the API. You need to request access for it:

OpenAI hosts a variety of different models. Please check the models that are compatible with the particular request.

Miscellaneous

I would appreciate bug reports and pull-request fixes.

Enjoy! 🚀️

wanghuabing / UnrealOpenAIPlugin