Flutter OpenGL

In this tutorial we will see how to run OpenGL in Flutter on Android.

We will run a simple gaussian blur filter and control the blur radius using a slider in the app.

Architecture

There are 2 projects in this demo

filter_plugin - This is where we will write our OpenGL filter, we write on as a separate Flutter Plugin.
filter_app - This is our main app that will use the filter plugin.

Here is a simple diagram of the entire solution:

Filter App

Nothing special in here, just a simple program that loads an image and renders it using the FilterPreview widget from the filter plugin.

Filter Plugin

The plugin is also split into 2:

The "Flutter" side, this is where we will develop the FilterPreview Widget that our app will use
The "Android" side, this is where we will develop the OpenGL filter, this is native Android code with a regular OpenGL program.

In the Android side of our plugin we are going to ask Flutter to get us a Surface Texture, and we will use this surface texture as the output of our OpenGL program.

Then in the Flutter side of our plugin we are going to develop a FilterPreview widget just renders that SurfaceTexture using Flutter's Texture widget (which renders a native surface texture using its id).

The Texture widget

Ok, so in order to display OpenGL content in Flutter we need to use the Texture Widget, per the documentation:

A rectangle upon which a backend texture is mapped... Backend textures are images that can be applied (mapped) to an area of the Flutter view. They are created, managed, and updated using a platform-specific texture registry. This is typically done by a plugin that integrates with host platform video player, camera, or OpenGL APIs, or similar image sources

The way we use this Texture widget is as follow:

return AspectRatio(
	aspectRatio: ...,
	child: Texture(
		textureId: the id of and output surface texture,
	)
);

So how do we get a SurfaceTexture and how do we connect it with OpenGL?

The SurfaceTexture

Our filter plugin class inherits from FlutterPlugin (like all plugins). When our plugin is getting loaded its onAttachedToEngine method will get called, and one of the parameters is a FlutterPluginBinding. Using this FlutterPluginBinding we can creaet a surface texture:

flutterSurfaceTexture = pluginBinding!!.textureRegistry.createSurfaceTexture()

Connecting to OpenGL

Once we have our output surface texture we can use it when we initialize our OpenGL window surface:

mEGLSurface = EGL14.eglCreateWindowSurface(mEGLDisplay, configs[0], outSurfaceTexture, surfaceAttribs, 0)

And then after we called the drawArrays method to run our OpenGL program we can render the output onto our surface texture by calling to:

EGL14.eglSwapBuffers(mEGLDisplay, mEGLSurface)

End

That's pretty much a high-level overview of the solution, more details and code walkthrough are in this YouTube video.

Demo app Matterhorn image ingmar on Unsplash

ValYouW / flutter-opengl