FNA-XNA / FNA

What is the proper way to setup a content pipeline project in an FNA environment when the pipeline needs to reference classes from my main source code?

Normally I just reference the FNA libraries for all my projects, except content projects which build the content via original XNA references, as described in the wiki. However, if for example CustomModelType in CustomModelProcessor needs to reference CustomRenderer, which is found in my Engine project, which is also referenced by Game, both of which reference FNA. CustomModelProcessor needs a reference to FNA as well as XNA, which creates a collision and the build cannot be completed because FNA can't build that content.

I've tried two solutions for this.

Using FNA as an alias in CustomModelProcessor, I only reference by Engine classes where needed, and create helpers to convert between FNA.Vector3 and XNA.Vector3, Matrix, BoundingBox and others which are used. The content still cannot be built though, since the content project has a reference to FNA through Engine and I can't seem to make it only use the XNA reference to build the content. Not sure if it would work in the end anyway.
Having a seperate copied tree of XNA and FNA versions of each project where the hierarchy is Game->Engine->FNA, and GameContent->CustomModelProcessor->EngineXNA->XNA. (I figure the EngineXNA can be a lightweight copy having only the required classes the CustomModelType uses, but they names/namespaces must match exactly!) However, upon loading the content built on the pure XNA reference hierarchy, I get the following error when loading my asset. I can't really understand what this fixup() method is doing?

ContentReader.cs (line 217): Microsoft.Xna.Framework.Content.ContentLoadException: 'Error loading shared resource. Expected type VertexBuffer, received type List`1'

Any help? Or is there an even easier solution altogether?

Could be a good idea to have a mention of how this should be done on the Wiki, if there is a decent way.

At the moment there isn't really a solution to do this, only to build the content with a pure XNA project then build the FNA version separately. We have this in the docs:

One thing you will NOT need to recreate is the content projects. When deploying to FNA, your content should already be complete and built. You can keep this in your solution if you like, but you will need to reference the XNA content pipeline here. You should not reference FNA in content projects!

One way to deal with this is have both an XNA csproj and an FNA csproj within the same solution, and have all the output in one folder. Depending on how much your content cares about the exe name (easily the worst feature of the pipeline) you may have to separate the folders and just have your FNA Debug config reference "../DebugXNA/Content" as your ContentManager.RootDirectory, for example.

Ok yet some more detective work. :) As I like neat solutions, I was trying to make a more automated solution than having a completely separate XNA solution to build the content. After I gave up on my two previous tries I figured I would setup a project as you suggested. Turns out I got the same runtime error as I did before, so the problem must lie elsewhere!

First the good news! It's not perfect, but instead of having a separate game project to build the content, it is completely fine to build content from a FNA game project even if it has custom content processors that requires access to base engine code, as long as the content project doesn't reference FNA itself, thus the best setup I've found is the following (in case someone else finds this and wants a solution, or se example attachment).

Create an XNA copy of your base engine project, lets call it EngineXNA.csproj. It can be identical to your EngineFNA.csproj, or should atleast contain all classes your CustomContentProcessor needs access to. Change the build dir to something like "bin\x86\DebugXNA". You won't need the built files.
Setup your project as such:

Game
- Content
- Engine
Content
- Processor
- EngineXNA
Processor
- EngineXNA

So why didn't this work the first time around for me? Turns out there is a very obscure (at least for me) difference between FNA and XNA when the content is loaded.

If my ContentProcessor attaches any kind of class, that has any type of List property, that property must contain a set accessor, else FNA will crash giving the error in my original post. However, when running on XNA, the set accessor is not required. This is true for content built both through the FNA connected content and XNA project connected content. So yet again this is a situation where a game may contain content, in the form of custom processed content, that works fine with XNA, but not with FNA. Not sure if this is something you'd want to fix or not? Or possibly give a more verbose exception on where a set accessor is missing.

Here is an example project that shows this problem, it also shows how to do the setup properly. Right now the project works fine, comment out line 20 in EngineClass.cs to reproduce the crash.
Note: The GameXNA is not needed for a FNA game, it's just there to test the differences.
FNAContentProcessorTest2.zip

If anyone will be able to fix this it probably won't be me... I think I've seen something like this maybe one time ever out of 50-something games and if I remember correctly there was some kind of [Attribute] that could hide certain properties to work correctly. You'll want to debug this yourself since you know your data better than me. Here's where you will most likely want to start:

https://github.com/FNA-XNA/FNA/blob/master/src/Content/ContentReader.cs#L275

Now I remember, it was [ContentSerializerIgnore]:

https://github.com/FNA-XNA/FNA/blob/master/src/Content/ContentSerializerIgnoreAttribute.cs

Shows how often I deal with the content pipeline...

Oh good to know! Except it doesn't help if that attribute is set to something in the processor, as it will be ignored by the serializer. Anyway, sadly this isn't my strongest area or expertise either or I might try and fix it, but as you say it's probably very seldom anyone would run into this.. If they do hopefully they find this thread, or maybe it would be something just to add some info to that exception message, I don't know. Cheers anyhow, I've got mine working for now! :)

@flibitijibibo sorry to jump in, but I've a somewhat related question, I haven't had much luck with MonoGame, even after 3.6 which I thought would be quite similar to FNA under the hood I'm having all kinds of problems, especially on integrated cards, so I'm sticking with FNA since I don't plan to target mobiles/consoles anytime soon. I know your efforts are focused on preserving the existing games, but after reading this:

At the moment there isn't really a solution to do this

I'm wondering if you've grander plans for FNA :). I especially like how you have full access to SDL2-CS in FNA, and performance is really great. After XNA was discontinued there's not much option for a simpler 2D game in C# that doesn't involve a lot of overhead, there's UV but it apparently isn't getting much love, and I see you're planning to implement modern APIs in XNA, what strikes me as odd since I don't think existing games will really benefit from that, unless you're doing that to push to MG or something. While SFML.Net is an alternative, it's certainly not getting Vulkan or Metal support anytime soon, and I really prefer SDL as a backend, but there's no modern reliable alternative to FNA that uses it and includes audio, input and possibly a binary format for contents.

Other than that I just want to congratulate on your fantastic work on both SDL2-CS and FNA! They're terrific high-quality products that so far have proven to be extremely dependable and it's a bit sad to see that they're apparently geared towards the past :P

@Alan-FGR Accurate reimplementation is grand enough for me, so that's probably where it will stay.

The nice thing about XNA is that all the lower-level stuff that the high-level stuff uses is almost entirely exposed - if you want to Build a Better SpriteBatch, that's entirely possible to do (you can even rip off the existing SpriteBatch to do it). Vulkan/Metal/D3D12 still have benefits even for the existing XNA library if only to remove a lot of validation that we currently go through with OpenGL, and having explicit API compatibility makes it a whole lot more compatible with systems where explicit APIs are your only real option (i.e. consoles).

In a similar vein, I don't have a whole lot planned for the Content subsystem because XNA's content pipeline is kind of terrible in a whole lot more ways than even I was expecting. XNA has a lot of bad ideas in it that I've been willing to follow through on, but the content pipeline might be my limit in that regard. Writing your own content pipeline that's optimized for the data that you have along with better ways to compress the files you have in your game would probably pay off way more than trying to replicate a system that's pretty much designed to be as Not Streamlined as humanly possible. Grab some DXT compressors, audio encoders/decoders, and PhysicsFS, and you could probably write a superior content editing workflow for the programmer and the artist in a weekend of goofing around.

@flibitijibibo thanks for your reply. Yeah, I definitely agree. Effects need to be compiled though, and although the latest compilers in the DX SDK still work perfectly (when specifying the version), effects are obsolete at this point, so even though there's a way to build a game from scratch with FNA, some things are a bit weird when doing so. What I'm saying is that although I use and love FNA, I always have this feeling that it's not really intended for what I'm doing 😛.

This might be worth looking at, though a lot of it isn't helpful until we get Shader Model 5 and a source compiler working:

https://blogs.msdn.microsoft.com/chuckw/2012/10/23/effects-for-direct3d-11-update/

https://github.com/Microsoft/FX11

Here's MojoShader's existing compiler:

https://hg.icculus.org/icculus/mojoshader/file/tip/mojoshader_compiler.c

We could bring back Effect.FromSource as an extension if we really wanted to... but that's like 3 new projects all at once.

Most importantly, FNA can be used for whatever; the only difference is that we can't change the API at all. MonoGame could do that but at the moment it's probably not profitable to do so (and probably won't be until XNA games stop showing up...?).

I want to share my bit of wisdom on this. I have been using MonoGame's Pipeline Tool since forever in combination with FNA.

The Content project (Content.mgcb) is set to Windows as a platform, and is included inside a Content-folder of the main project. The Content project is set to MonoGameContentReference as Build action, which comes from the <Import Project="$(MSBuildExtensionsPath)\MonoGame\v3.0\MonoGame.Content.Builder.targets" /> inside the .csproj itself. This causes the assets to be compiled on a build. Since you need MonoGame installed for the Pipeline Tool, why not take advantage of its build targets too, right?

I've only made two small adjustments to the Pipeline Tool by adding two custom Content Processors (using MonoGame.Framework.Content.Pipeline.dll as a reference):

OggSongProcessor which forces all songs to be compiled using the DesktopGL target, regardless of the one set by default. This makes sure all songs are compatible with Ogg Vorbis.
FxcEffectProcessor which compiles effects using a locally provided fxc.exe and outputs a .fxc at the location of the original .fx (which I then check in on my source control), before handing it off to the target .xnb. The .fxc is compatible with MojoShader and works on all platforms. By having the precompiled .fxc available, you can also run this pipeline on other platforms where running fxc.exe is not possible. For me this was the case when I was debugging my project on Mac.

The combination of this gives you pretty much the same experience as XNA, while allowing Content to be build for all platforms at once.

Digging this up - most of what's been discussed here should probably go into what fixes #154 but I would like to return to this:

If my ContentProcessor attaches any kind of class, that has any type of List property, that property must contain a setaccessor, else FNA will crash giving the error in my original post. However, when running on XNA, the setaccessor is not required. This is true for content built both through the FNA connected content and XNA project connected content. So yet again this is a situation where a game may contain content, in the form of custom processed content, that works fine with XNA, but not with FNA. Not sure if this is something you'd want to fix or not? Or possibly give a more verbose exception on where a set accessor is missing.

@jsmars, might this block have anything to do with this?

https://github.com/FNA-XNA/FNA/blob/master/src/Content/ContentReaders/ReflectiveReader.cs#L248

Should we also skip ReadSharedResource for the block on line 268?

Here's a patch that might fix this @jsmars:

diff --git a/src/Content/ContentReaders/ReflectiveReader.cs b/src/Content/ContentReaders/ReflectiveReader.cs
index 90950bc..98197a8 100644
--- a/src/Content/ContentReaders/ReflectiveReader.cs
+++ b/src/Content/ContentReaders/ReflectiveReader.cs
@@ -256,7 +256,10 @@ namespace Microsoft.Xna.Framework.Content
 				}
 				else
 				{
-					setter = (o, v) => { };
+					/* If there's no setter, don't read!
+					 * The value is probably somewhere else.
+					 */
+					setter = null;
 				}
 			}
 			else
@@ -266,7 +269,8 @@ namespace Microsoft.Xna.Framework.Content
 			}
 
 			if (	contentSerializerAttribute != null &&
-				contentSerializerAttribute.SharedResource	)
+				contentSerializerAttribute.SharedResource &&
+				setter != null	)
 			{
 				return (input, parent) =>
 				{

Let me know if this fixes it and I'll commit to master.

Thanks for looking into this again! I tried it out with my test-case project, sadly for my example it seems the exception persists. Here is an updated archive of the test-case with the patch.
As before, commenting in/out line 20 on EngineClass.cs reproduces the error, and swapping between GameFNA/GameXNA to confirm it's working. Sometimes I get missing assembly when swapping, just clean solution to have it rebuild properly.
FNAContentProcessorTest2-patchtest.zip

It is using the same FNA version as it was before with only the patch applied though, I'm not sure if that could make any difference?

Content hasn’t changed in a while so that shouldn’t be too bad - we do need a full trace though. Think running msbuild in a VS command prompt will give us the whole thing?

Any progress here? I'm inclined to close this due to old age, but I'm sure I'll get another question about this so I dunno.

@LennardF1989, I may need your Processor files as part of dealing with #154. In documenting MG compatibility it may be needed when talking about MGCB.

@flibitijibibo Here is everything I have right now: https://gist.github.com/LennardF1989/a5d7d54c89cb6cd0e6bc9551b6fa6a48

And older revision off the OggSongProcessor used the SongProcessor as a base, which allows you to do whatever MonoGame does. It's much safer for changes in the future, but also restraints the process. Eg. Even if you know you won't have to regenerate, you still will have to wait for the original SongProcessor to do it's thing. But the version as it is right now, doesn't retain Duration-information.

To be honest I probably won't be the one to find the solution on this issue as it's a bit outside my area of expertise and I can't put too much more time into this at the moment - though I may look into it more in the future if it's needed.

For my own solution it's enough to know how to avoid the errors. I would suggest adding a verbose exception message here to any custom content being loaded with a list type property that doesn't include a set accessor, to inform that this is in fact needed, to avoid having anyone get stuck in the same way again though.

@LennardF1989 Thanks for this! I'll probably link directly to that if that's okay.

@jsmars I finally figured out what's happening. First of all, I was actually right about the ReflectiveReader stuff being too aggressive:

466594b

With this change it almost fixes the problem. What's happening is that the very last object is the Tag, and while it does read in your EngineClass, it does NOT read the List inside, which means that when we start reading shared resources, it just reads the next object which is actually that inner List, meaning the shared list was off by one and all the fixups broke. A fun exercise: Change this line in your old FNA version to arbitrarily read sharedResourceCount + 1 objects (and skip the fixup functions):

https://github.com/FNA-XNA/FNA/blob/master/src/Content/ContentReader.cs#L306

But that wasn't enough. It was also failing here:

https://github.com/FNA-XNA/FNA/blob/master/src/Content/ContentReaders/ReflectiveReader.cs#L222

And for your convenience:

FNA/src/Content/ContentTypeReaderManager.cs

Line 64 in 5aa2ed0

public ContentTypeReader GetTypeReader(Type targetType)

As you will see, the type we're looking for is actually in the Dictionary... it's looking for Vector3 (Microsoft.Xna.Framework.dll) and the Dictionary holds Vector3 (FNA.dll). Pretty dumb, right? So in addition to doing fixups as we parse the ContentTypeReader list, we also have to double check nested types like the Tag objects you might find in Models:

d8da2e6

With this, the model FINALLY loads properly without a set accessor. Unless there are any further issues, I think this is finished?

Ah yes, there it is! That should wrap this up I think. As always, thanks a lot @flibitijibibo for all your hard work! :)

Excellent! I’ll go ahead and close this then. Anything regarding Lennard’s stuff can be continued in #154.

@flibitijibibo No problem, much obliged! My way of contributing back to FNA :)

How to properly setup Content Pipeline projects