State of Linux (Cont)
Exzap opened this issue · comments
This is a continuation of #1
The current top priorities for our Linux build are:
- Providing AppImage releases
- Wayland support (at least for Vulkan)
- Finding a way to correctly emulate Wii U's case-insensitive filesystem on case-sensitive host filesystems
There are also some features that haven't been ported over yet:
- HID support (Wii U nsyshid.rpl driver). Required for USB portals (Skylanders Portal of Power, Lego Dimension etc.)
- nlibcurl. Has been ported but not tested a lot
- Some debugger features like memory breakpoints
Nice to have:
- Providing flatpak releases
(List last updated on 16-03-2024)
When building from source we use vcpkg for managing dependencies on all platforms and our officially supported and recommended compiler is Clang 14 or higher (and MSVC on Windows). With the -DENABLE_VCPKG=OFF option you can use system libraries instead of vcpkg but we generally discourage it as we prefer having builds with the same exact library configuration across all platforms for simplicity of debugging. We also won't officially support distributing Cemu as a distro package. Of course package maintainers are free to package it, but our focus is to streamline the troubleshooting and bug reporting process and it's way easier if there aren't dozens of different builds.
Finding a way to correctly emulate Wii U's case-insensitive filesystem on case-sensitive host filesystems
ext4 has case-folding support. Would it be possible to make it a requirement for Cemu? Users would have to store stuff accessed by the Wii U in a separate volume with this feature enabled though, which could be hard to do.
ext4 has case-folding support. Would it be possible to make it a requirement for Cemu? Users would have to store stuff accessed by the Wii U in a separate volume with this feature enabled though, which could be hard to do.
That's a very bad idea. This requires, as you said, a seprate volume. Users might want to use a case-sensitive filesystem, so they need a own partition/drive just for Cemu.
It is also not very end user friendly to show a Message to new Linux Users, who want to play their Games: "Welcome to Cemu on Linux! Linux is a user friendly OS and not only a nerd OS as everyone says. Now please reformat your drive and create a partition with a case-insensitive filesystem to use Cemu."
Especially Steam Deck Users will have fun.
Yeah I was just pointing out that the thing exists. It would be cool if Cemu supported it though, as I doubt that any workaround for case-folding would be more performant than an in-kernel implementation.
Steam Deck wouldn't have this issue afaik since it's using a case-insensitive filesystem. But yeah, really only Wine does this to my knowledge and they do heavy caching using a virtual filesystem to prevent it from hurting performance.
To have a bit more context: what data used by Cemu needs casefolding support? Only the mlc01 folder? Only the games one? Both? Something else?
Case-insensitivity is needed for all files that map to the virtual Wii U file system in some way. Which is mlc and game folder files. So basically a majority of our file access code done by many different HLE modules
This might be too hacky and im not too much of a programmer but cant all the files and directories which will be mapped to the wii u filesystem just be all uppercased or all lowercased
That wouldn't work because the case must be kept intact. Anything else is not accurate emulation and some games might rely on case-sensitive checks internally
Edit - I was wrong
Another possibility though - https://www.collabora.com/news-and-blog/blog/2020/08/27/using-the-linux-kernel-case-insensitive-feature-in-ext4/
How do other emulators cope with this?
It seems that y'all are talking about the Wii U's case-insensitive FS, but I'll throw this issue I'm having here since I think it's related to the main discussion:
Porting the remaining features (e.g. online play and some UI features)
About the online play feature(s), has anyone managed to get online actually working on Linux? I'm using the pre-compiled binary for Ubuntu on Arch, and after symlinking the missing dependencies the games I play work fine but when I try to go online by copying mlc01/ (recursively, of course), otp.bin and seeprom.bin it just throws this error in the account settings:
The currently selected account is not valid or dumped online account
AccountId missing (The account is not connected to a NNID)
My host machine is running Arch. Online works just fine on a Windows 10 VM (it says that the account is valid on Windows), so it's definitely not me following the instructions incorrectly. Does anyone know what could be causing this issue?
Users might want to use a case-sensitive filesystem, so they need a own partition/drive just for Cemu.
You can create an image file and format it as ext4
Users might want to use a case-sensitive filesystem, so they need a own partition/drive just for Cemu.
You can create an image file and format it as ext4
To do it properly someone would need to write a Wii/U filesystem driver just like this one for FATX...
https://github.com/mborgerson/fatx
... which uses a userspace FUSE based library and can seamlessly handle mounting Xbox & Xbox 360 images directly.
I'm on latest ArchLinux and compiled Cemu with gcc and g++ and make. Few issues:
- Changing Controller Settings(rumble and sticks) doesn't work it's just blank displaying what was in foreground before opening it
- I can't get any game I own working. Smash bros launches but after all the dlc messages popped up it crashes, wind waker crashes after opening it same with tp same with star fox zero and breath of the wild. Bayonetta 1&2 also crash :<
Tried it on my NTFS drive and ext4 both have the same issue
You can create an image file and format it as ext4
This may work for the mlc01 folder, but not for the games one. The latter has to be easily accessible to the user. Still, better than requiring a different partition :)
To do it properly someone would need to write a Wii/U filesystem driver just like this one for FATX which uses a userspace FUSE based library and can seamlessly handle mounting Xbox & Xbox 360 images directly.
Mh, I believe that the issue here is the opposite; you don't need to mount a Wii U filesystem on the host, but you have to read the host's filesystem on the (emulated) Wii U
Wine handles the case mismatch problem by recursively iterating directories and comparing file/folder names manually. With lots of caching it can be made fast.
Are you using Vulkan or OpenGL as the graphics API?
Mh, I believe that the issue here is the opposite; you don't need to mount a Wii U filesystem on the host, but you have to read the host's filesystem on the (emulated) Wii U
There's no functional difference between the two, once a filesystem is mounted on the host it is accessible by both the host and the emulator. So you create a 32GB image, format it for the WiiU, mount it with FUSE and its good. Emulator can read it as a native volume and the user can drag/drop files as needed.
Edit Still not exactly ideal though since you could only ever hold 32GBs worth of games at one time and would need to swap them out as needed. Also any kind of image solution would break the GUI game list.
About the online play feature(s), has anyone managed to get online actually working on Linux?
Nope, they're currently stubbed on Linux, like a few other things. One of the things listed by Exzap. Task is still up for grabs for anyone who wants :p.
@Crementif I managed to get Cemu to say that my account is valid by compiling from source. It was a problem with the binary in the releases. I'll update if I encounter any issues, but the problem I mentioned is fixed for now.
Wine handles the case mismatch problem by recursively iterating directories and comparing file/folder names manually. With lots of caching it can be made fast.
I don't know what does Wine implement below the hood, but, maybe, a small SQLite db can be used to cache the entire tree?
Wine handles the case mismatch problem by recursively iterating directories and comparing file/folder names manually. With lots of caching it can be made fast.
I don't know what does Wine implement below the hood, but, maybe, a small SQLite db can be used to cache the entire tree?
For Unix, Redis would be a better choice than SQLite.
Are you using Vulkan or OpenGL as the graphics API?
Using Vulkan, OpenGL does the same thing
XDG Base Directory Specification
The files that Cemu writes and reads from should not be next to the binary. It should be in the XDG directories.
graphicsPacks should go in XDG_DATA_HOME: e.g. ~/.local/share/cemu/graphicsPacks, as should memorySearcher and mlc01. Controller profiles and custom game profiles should also go here. log.txt should be written here as well or don't do that and write to syslog.
settings.xml should go in XDG_CONFIG_HOME. keys.txt shoudl also be placed here.
shaderCache should go in XDG_CACHE_HOME.
Wine handles the case mismatch problem by recursively iterating directories and comparing file/folder names manually. With lots of caching it can be made fast.
I don't know what does Wine implement below the hood, but, maybe, a small SQLite db can be used to cache the entire tree?
For Unix, Redis would be a better choice than SQLite.
Why would there be any need to persistently store such a cache? I can't think of any good reason for that.
Nor would there be any need for a database engine to manage it.
Controller profiles and custom game profiles should also go here.
To me they look more like configuration files than general data, so why not ~/.config/cemu/?
log.txtshould be written here as well or don't do that and write to syslog.
Mh, shouldn't logs go to ~/.cache/cemu/?
$XDG_CACHE_HOMEdefines the base directory relative to which user-specific non-essential data files should be stored.
Controller profiles and custom game profiles should also go here.
To me they look more like configuration files than general data, so why not
~/.config/cemu/?
log.txtshould be written here as well or don't do that and write to syslog.Mh, shouldn't logs go to
~/.cache/cemu/?
$XDG_CACHE_HOMEdefines the base directory relative to which user-specific non-essential data files should be stored.
Yeah, cache is more for what programs do internally; if the files it generates are usable/useful for the end user it shouldn't be stored in cache generally.
Good news! I had a quick chat with Krisman, the guy who implemented casefolding (case-insensitive) support for ext4, and he said the following:
Unfortunately, there is no way to mount a directory from a filesystem over a different filesystem. But, the Ext4/F2FS implementations support case-insensitiveness per-directory, so you could make only those specific directories case-insensitive.
Since you cannot guarantee the user is on ext4, one way forward is to store the emulation directories inside a qcow image, that you will later mount on a specific path. Since you control the image creation/distribution, you can guarantee it is ext4 and correctly configured to expose the right case-insensitive directories.
As you said, there is also the userspace solution, which you emulate case-insensitiveness in userspace, if it detects the underlying filesystem doesn't support it. Valve, who is a big user of case-insensitiveness, actually has their own library for supporting older systems. If they find the system to be ext4 on a fairly recent kernel, they will use the kernel implementation. otherwise, they fallback to the library. A userspace implementation is always racy, but it usually works well enough as a fallback. I wrote a library (unmaintained) for that here:
https://gitlab.collabora.com/krisman/libcasefold
Please, let me know if you decide to use it, I can revive the project and start maintaining it again. It deserves some love.
Finally, a piece of good news is that my team is working on briging case-insensitive support for more filesystems. Right now, we support ext4 and f2fs, but the plan is to enable it on btrfs and tmpfs in the near future.
But this would still require the users to re-format their main ext4 partition, right? At least on my Debian system, it seems that the casefold feature is not enabled.
You don't need to reformat. But before enabling it on a directory, you need to configure the superblock, which is basically telling it which encoding will be used for case-insensitive directories:
tune2fs -O encoding /dev/sdx
It won't make the filesystem case-insensitive, just configure the superblock. Unless you use overlayfs, there isn't any downside on doing that. We are still working on mixing case-insensitive and overlayfs. If you have any problems, the new versions of tune2fs allow disabling the flag too.
All things considered, I think that using an ext4 image managed by Cemu (as also mentioned by @JPabloLassala) might be the best solution so far. Opportunistic use of the casefold feature on the host's filesystem might not be helpful in practice, because as far as I know there are no distros that ship with this feature enabled by default, and enabling it requires unmounting the filesystem (that could be hard/impossible to do for users that store everything in a single partition, like me).
Using an image would likely mean that the mlc01 and games folder won't be normally accessible by the user, but I guess Cemu could transparently mount and unmuont the image when launched and closed.
IMO using fs built-in casefolding when available and using libcasefold as a fallback sounds good. It will be much more user-friendly than images, because of images not being always mounted or being too small when moving new games in. Also it sounds like libcasefold would benefit from having a big project like Cemu using it.
using fs built-in casefolding when available and using libcasefold as a fallback sounds good
In theory yes, but I fear that in practice it would lead to poor performance, as kernel casefolding is not that widely available by default. I may be wrong though, as the only way to know is to implement the feature :)
It will be much more user-friendly than images, because of images not being always mounted or being too small when moving new games in.
True. Both ext4 and qcow2 should support live growing though, so moving new games in shouldn't be an issue.
Also it sounds like libcasefold would benefit from having a big project like Cemu using it.
Also true, and I'd really like to see libcasefold improve too.
In theory yes, but I fear that in practice it would lead to poor performance
I don't think any solution would really suffer from performance issues, even when translating each path manually (and not caching). The only thing that might suffer is loading times and that's not really critical (correct me if I'm wrong).
By the way, not saying that this isn't an important issue to solve, but is there even a game which regresses from not having case-insensitive support?
Controller profiles and custom game profiles should also go here.
To me they look more like configuration files than general data, so why not
~/.config/cemu/?
Yes I think those should go there instead.
log.txtshould be written here as well or don't do that and write to syslog.Mh, shouldn't logs go to
~/.cache/cemu/?
$XDG_CACHE_HOMEdefines the base directory relative to which user-specific non-essential data files should be stored.
Not sure. There is not a good spot for logs in the XDG specification. It makes me think they want everything writing to syslog (preferably journal from systemd).
Also I forgot to mention, the CMake files have no install() targets. It should be standard include(GNUInstallDirs) and putting the main bin and stuff in bin/ (in the repo; should be in /usr/share/cemu) and dist/linux into the correct locations.
Never used this before, but XDG says logs go in XDG_STATE_HOME, defaulting to ~/.local/state.
By the way, not saying that this isn't an important issue to solve, but is there even a game which regresses from not having case-insensitive support?
I have been also been playing on a case-sensitive filesystem and I've been waiting to see the "case sensitivity induced error" so I can know what one would look like, so I would second this. I'd like to know a reproducible scenario in which playing from mlc01 stored in a case-sensitive filesystem results in a symptomatic bug, so that I can test whether any given potential solution is correctly applied and actually working. I hope it doesn't require a specific game to observe, otherwise people who don't own that game won't be able to test the solutions.
All things considered, I think that using an ext4 image managed by Cemu (as also mentioned by @JPabloLassala) might be the best solution so far. Opportunistic use of the casefold feature on the host's filesystem might not be helpful in practice, because as far as I know there are no distros that ship with this feature enabled by default, and enabling it requires unmounting the filesystem (that could be hard/impossible to do for users that store everything in a single partition, like me).
.. or we could just do it like WINE does and have a solution that does not have any of those restrictions. Remember, we always want to be as user-friendly as possible.
Case-insensitivity can be emulated transparently by doing the path lookup manually. I'll try to explain the algorithm:
Assuming we want to look up ./content/data/foo.bin but the host filesystem stores it under ./CONTENT/DATA/FOO.BIN. The algorithm starts at the root of the path and recursively checks each sub-directory for the actual case of the next lower child. The steps would go something like this:
- Iterate
./in the host file system and case-insensitive compare all file and folder names againstcontent. This gives youCONTENT. - Using the corrected case from the previous step, we now scan
./CONTENT/fordata. This gives usDATA. - Scan
./CONTENT/DATA/forfoo.bin. This gives usFOO.BIN. - The final path we end up is the case-corrected
./CONTENT/DATA/FOO.BINwhich we then use to access the file.
The actual algorithm is quite simple (it's really only one step but repeated recursively). It would likely be fast enough for our needs even without any kind of caching of previously looked up paths. And if not, the priority is to get it to work and then we can always make it faster afterwards. If nobody else steps up to the task I will implement this in the future, but I'm currently swamped with other tasks
I have been also been playing on a case-sensitive filesystem and I've been waiting to see the "case sensitivity induced error" so I can know what one would look like, so I would second this. I'd like to know a reproducible scenario in which playing from mlc01 stored in a case-sensitive filesystem results in a symptomatic bug, so that I can test whether any given potential solution is correctly applied and actually working. I hope it doesn't require a specific game to observe, otherwise people who don't own that game won't be able to test the solutions.
Pick any game without an update, go to it's content folder (it must be in extracted format). Rename a bunch of folders and files to have random upper and lowercase letters. This will break the game on case-sensitive file systems. But I doubt you even have to go that far, most games don't have coherent case to begin with and probably crash on boot.
Controller profiles and custom game profiles should also go here.
To me they look more like configuration files than general data, so why not
~/.config/cemu/?Yes I think those should go there instead.
log.txtshould be written here as well or don't do that and write to syslog.Mh, shouldn't logs go to
~/.cache/cemu/?
$XDG_CACHE_HOMEdefines the base directory relative to which user-specific non-essential data files should be stored.Not sure. There is not a good spot for logs in the XDG specification. It makes me think they want everything writing to syslog (preferably journal from systemd).
System logs should go in /var/logs, user logs should go in /home/$USER/.local/share/appname.
Since XDG is about interoperability between different DEs it doesn't have a spec for logs, it would be redundant. Unix already has well established log locations.
Never used this before, but XDG says logs go in
XDG_STATE_HOME, defaulting to~/.local/state.
Thats news to me and honestly, I've never encountered an app in the wild that uses this part of the spec (on my system right now that folder doesn't even exist). Heck even X dumps its logs into /home/$USER/.local/share/Xorg.
Still I guess it does make sense, logs are indeed state files.
Edit - Using the ~ alias is frowned upon when coding paths into an application, if the user somehow runs as root you will end up writing to the /root folder and this can cause issues for the entire environment. Always use the full path with the $USER variable, this way if the user somehow runs as root and avoids the fact the root environment doesn't have any XDG vars set the operation will still fail since /home/root doesn't exist.
Yeah I know this is kinda nitpicky but remember, users are almost always idiots and if it can happen its likely someone will do it.
.. or we could just do it like WINE does and have a solution that does not have any of those restrictions. Remember, we always want to be as user-friendly as possible.
Case-insensitivity can be emulated transparently by doing the path lookup manually. I'll try to explain the algorithm:
Assuming we want to look up
./content/data/foo.binbut the host filesystem stores it under./CONTENT/DATA/FOO.BIN. The algorithm starts at the root of the path and recursively checks each sub-directory for the actual case of the next lower child. The steps would go something like this:
Iterate
./in the host file system and case-insensitive compare all file and folder names againstcontent. This gives youCONTENT.Using the corrected case from the previous step, we now scan
./CONTENT/fordata. This gives usDATA.Scan
./CONTENT/DATA/forfoo.bin. This gives usFOO.BIN.The final path we end up is the case-corrected
./CONTENT/DATA/FOO.BINwhich we then use to access the file.The actual algorithm is quite simple (it's really only one step but repeated recursively). It would likely be fast enough for our needs even without any kind of caching of previously looked up paths. And if not, the priority is to get it to work and then we can always make it faster afterwards. If nobody else steps up to the task I will implement this in the future, but I'm currently swamped with other tasks
Yeah, this doesn't look complex at all, but how it this going to handle non-ascii paths? This article contains some helpful information about how the casefold-for-ext4 team decided to handle cases like the aforementioned one. It doesn't look trivial.
Since we don't want to use images (that I understand how they could be confusing to users), what's the benefit in implementing case-insensitivity manually instead of using libcasefold? Krisman has also said that he would love to start working on it again if someone would actually use it.
Using libcasefold instead of manually emulating case-insensitive path lookups would have the following benefits:
- We don't have to implement any algorithm, that regardless of its complexity would still require time and energy (libcasefold can be used without even having to recompile the binary)
- On casefold-enabled filesystems libcasefold would use the kernel implementation, leading to a lower / no performance impact (currently has to be configured manually, but this is likely because at the time case-insensitive filesystems on Linux weren't a thing)
- Corner cases would be handled
Yeah, this doesn't look complex at all, but how it this going to handle non-ascii paths?
All Wii U paths are ASCII.
Since we don't want to use images (that I understand how they could be confusing to users), what's the benefit in implementing case-insensitivity manually instead of using libcasefold?
Custom solutions are always fine-tuned for the specific problem they are trying to solve which has all sorts of benefits, usually it's better performance and much greater flexibility. I know that outside of high performance software, it's common to reuse as much as possible, but we don't really do this in Cemu. There is a little bit of reinventing the wheel here and there and usually the end result is better for it. That said, I'm not against dependencies in general but the problem needs to be sizeable enough to justify introducing another dependency. And this isn't. My personal opinion of course.
On casefold-enabled filesystems libcasefold would use the kernel implementation, leading to a lower / no performance impact
It's obvious that we would not do translation when it's not necessary.
All Wii U paths are ASCII.
Good, I wasn't sure about it.
Custom solutions are always fine-tuned for the specific problem they are trying to solve which has all sorts of benefits, usually it's better performance and much greater flexibility. [...] I'm not against dependencies in general but the problem needs to be sizeable enough to justify introducing another dependency. And this isn't.
I'm not against custom stuff either, if it makes sense to do this for Cemu than it is indeed appropriate. I was just trying to make sure that this was the case :)
It's obvious that we would not do translation when it's not necessary.
Perfect.
I'm hitting these assertions from GX2_Texture when going in game in BOTW cemu_assert_debug(texUnit < Latte::GPU_LIMITS::NUM_TEXTURES_PER_STAGE);
with texUnit == 18
Are these assertions valid? Continuing past them in the debugger gets in game without visible issues.
I'm hitting these assertions from GX2_Texture when going in game in BOTW
cemu_assert_debug(texUnit < Latte::GPU_LIMITS::NUM_TEXTURES_PER_STAGE);with texUnit == 18
Are these assertions valid? Continuing past them in the debugger gets in game without visible issues.
When building did you set -DPUBLIC_RELEASE=ON?
I'm hitting these assertions from GX2_Texture when going in game in BOTW
cemu_assert_debug(texUnit < Latte::GPU_LIMITS::NUM_TEXTURES_PER_STAGE);
with texUnit == 18
Are these assertions valid? Continuing past them in the debugger gets in game without visible issues.When building did you set
-DPUBLIC_RELEASE=ON?
That would disable these assertions, but IIRC I hit some other crash with that on that I didn't look into.
Nevermind, that does work. Might have just ran into something intermittent before.
Edit - Using the ~ alias is frowned upon when coding paths into an application, if the user somehow runs as root you will end up writing to the /root folder and this can cause issues for the entire environment. Always use the full path with the $USER variable, this way if the user somehow runs as root and avoids the fact the root environment doesn't have any XDG vars set the operation will still fail since /home/root doesn't exist.
Why would things in /root be a problem? That's the home directory for the root user, so it's exactly the right place to put things.
Don't ever hardcode /home/$USER, that's going to break for anyone who has a different home directory set in /etc/passwd.
I'm on latest ArchLinux and compiled Cemu with gcc and g++ and make. Few issues:
- Changing Controller Settings(rumble and sticks) doesn't work it's just blank displaying what was in foreground before opening it
- I can't get any game I own working. Smash bros launches but after all the dlc messages popped up it crashes, wind waker crashes after opening it same with tp same with star fox zero and breath of the wild. Bayonetta 1&2 also crash :<
Tried it on my NTFS drive and ext4 both have the same issue
I also cannot get Star Fox Zero to start. Just gets stuck at the startup/logo screen.
Cemu crashes on loading games in Ubuntu too, though at least it compiles under certain circumstances. It will take some time before games actually run given that the linux side of the codebase is still quite new and there are plenty of issues remaining to resolve (hence this report),
With cf598e3 we now emulate case-insensitive file/directory lookup on case-sensitive file systems. The fix is baked into our core file access helper (FileStream) and thus covers almost all operations. The only thing remaining is to refactor legacy code that directly accesses files (via fstream or fopen) to use FileStream instead.
amazing progress, honestly. Only thing left is the flatpak/appimage (atleast only thing left that causes the majority of the linux issues)
Wayland support, without going through XWayland (in which no games boot), isn't implemented yet either.
Edit: nevermind looks like it works through XWayland for others, proper Wayland support not implemented though
Wayland support, without going through XWayland (in which no games boot), isn't implemented yet either.
What do you mean? It's working for me through xwayland. (on sway)
Wayland support, without going through XWayland (in which no games boot), isn't implemented yet either.
What do you mean? It's working for me through xwayland. (on sway)
Wayland support != XWayland
Wayland support, without going through XWayland (in which no games boot), isn't implemented yet either.
What do you mean? It's working for me through xwayland. (on sway)
Wayland support != XWayland
That was not my point. I read Latrolage's post as saying games don't work in xwayland, which doesn't seem to be true.
That was not my point. I read Latrolage's post as saying games don't work in xwayland, which doesn't seem to be true.
Indeed if games boot on XWayland
Titles are starting to work in testing, at least on Ubuntu-based systems, and Wayland or not, I'm impressed that they're already working so well. It's not just that rendering seems as smooth and GPU-efficient as on Windows, but also secondary features such as rumble and motion (on a DS5 pad) that I never expected to be functional so soon given the intra-platform library differences.
I don't want to derail the intent of this thread, but I think it speaks to the quality of the codebase from Exzap and the original team that so few changes had to be made to it, relatively speaking, to approach performance and feature parity so soon after source release. This isn't a common thing among projects that take this path.
Okay, back on point: in short, the "State of Linux" is already quite good, and continued testing will help nail it all down.
Will there be a arch Linux PKGBuild or releases soon? The only one available is Cemu on the AUR but that uses wine. This would also help Linux user who haven’t compiled before and are used to the AUR.
We'll officially release easy ways of accessing Cemu on Linux, using appimage and/or flatpaks, when Cemu 2.0 is in a good state of releasing.
I haven't tested Cemu on Linux in a bit, but afaik it still suffers from:
- crashes
- not working on Wayland (directly?)
- not having motion controls through SDL2
- doesn't have online (not required, but nice to have)
- performance issue and regressions, same as windows
On a few of the issues you mentioned:
-
Crashing: There's an occasional need to delete the shader cache to avoid crashing. I haven't been able to trace the cause. Also, some games (e.g. DK TF) crash consistently on loading even with a fresh shader cache, as reported.
-
Motion controls: as a workaround to build Cemu on Ubuntu platforms, sdl2 can be removed from vcpkg.json, i.e. use system sdl2 instead (2.0.20 on Ubuntu 22.04, for example). However, motion works in this case.
Will there be a arch Linux PKGBuild or releases soon? The only one available is Cemu on the AUR but that uses wine. This would also help Linux user who haven’t compiled before and are used to the AUR.
I've already compiled since the first days, with errors, also now it compiles fine. Just keep eyes on dependencies, then you can do it.
Just want to give a clarification about something I (unintentionally) might've spread, but Exzap informed me about it and said there's not really a reason for there to be a performance regression in this 2.0 version. After benchmarking things myself between Cemu 1.26.2f and Cemu 2.0 on Windows I was fairly surprised to see that I was getting the same FPS (in BotW at least).
Though, it's important to note that from Cemu 1.27.1 onwards accurate barriers is enabled by default (which you can find in Cemu's toolbar under Debug) which might also be one of the reasons why people thought their games were running worse. You can toggle the option in-game to see the FPS difference, but it fixes quite a few effects which is why it's now by default enabled.
This does mean that Linux builds might have some performance issues if the wine version is indeed running better (which from what I've heard is the case, but I could be wrong). If anyone could test the performance difference on their system if they build it from source using the latest build methods in the build.md file, that'd be awesome. Make sure that the settings all match too obviously.
I can confirm that using system sdl2 fixes motion controls (DualSense PS5 Controller) on Ubuntu-based systems.
#286
./Cemu: error while loading shared libraries: libsepol.so.1: cannot open shared object file: No such file or directory
None of the libsepol packages I could find on Ubuntu did anything to fix the issue.
GCC worked compiling under PopOS, clang did not, tried using clang 12 as well.
You can work around the libsepol.so.1 by sym linking (#1). ABI must somehow work out to be the same as I was able to get it working on ubuntu 22.04.
sudo ln -s /lib/x86_64-linux-gnu/libsepol.so.2 /lib/x86_64-linux-gnu/libsepol.so.1
sudo ln -s /usr/lib/x86_64-linux-gnu/libffi.so.8 /usr/lib/x86_64-linux-gnu/libffi.so.7
I've ran into cemu crashing for different reasons when manually compiling with GCC and Clang.
I still have issues with configuring controllers and no audio with the released tarball.
You can work around the libsepol.so.1 by sym linking (#1). ABI must somehow work out to be the same as I was able to get it working on ubuntu 22.04.
Please don't, it will create all kinds of bugs. The soname was bumped because the two ABIs are incompatible. Just install the older version, or download the library and put it in a directory pointed by the LD_LIBRARY_PATH environment variable.
Finding a way to correctly emulate Wii U's case-insensitive filesystem on case-sensitive host filesystems
If I may:
First, a note: I'm not at all familiar with Cemu's code structure so I don't know how feasible this would be.
Also, this may be too slow for emulation purposes, but still, here's my possible contribution.
I'm working on an open source project (Julius) which had the same issue. The project is a reimplementation of the original Caesar III game for Windows, and it also had the same case sensitivity problem when porting to other platforms.
In our case, the solution was to:
- Check to see if the file was found "as-is" (with no case correction);
- If it was found, great. If not, and the platform has a case-sensitive filesystem:
- Decompose the entire path to its directories;
- Create an empty string that will store the real file location after matching its case;
- Starting at the base directory, traverse the directory contents, finding a match for the current directory name using C's case-insensitive comparison functions;
- When a match was found, append the actual cased directory part to the string; If not found, the file doesn't exist;
- Move on to list the files of the actual cased part of the path we found so far;
- Do this recursively until you get to the file and also compare the file in the last directory.
As mentioned, this can be slow as there's a lot of string comparisons going on. However, the results can be cached, and since Cemu needs to track file accesses from the games during gameplay anyway, the cache can always be kept updated.
Hopefully this is helpful!
I built this on Linux using the documentation, but when I go into Options -> General settings it crashes with this output:
(Cemu_release:71638): Gtk-WARNING **: 22:28:16.138: Could not load a pixbuf from /com/ubuntu/themes/Yaru-dark/3.0/assets/bullet-symbolic.symbolic.png.
This may indicate that pixbuf loaders or the mime database could not be found.
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0
Segmentation fault!
Error: signal 11:
./bin/Cemu_release[0x82fc32]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f5da2196520]
Before that it also kept crashing when I was trying to choose file directories on first Cemu startup.
From that error it seems like there is a issue with your system theme
The theme thing sounds like a benign warning to me.
I'd rather investigate the Intel driver. Does it happen when you override the mesa driver to llvmpipe/lavapipe via MESA_LOADER_DRIVER_OVERRIDE? (Obviously won't be playable but if those don't crash we know it's the drivers' fault.)
The theme thing sounds like a benign warning to me.
I'd rather investigate the Intel driver. Does it happen when you override the mesa driver to llvmpipe/lavapipe via
MESA_LOADER_DRIVER_OVERRIDE? (Obviously won't be playable but if those don't crash we know it's the drivers' fault.)
I don't get those messages about Intel anymore for some reason. I also ran MESA_LOADER_DRIVER_OVERRIDE=llvmpipe as well as MESA_LOADER_DRIVER_OVERRIDE=lavapipe but I still have the same issue.
However, I do see more output that I didn't post before. I'm not sure if I just didn't see it or what, but here it is:
(Cemu_release:4806): Gtk-CRITICAL **: 13:41:30.547: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar
The above line appears above the other output I posted.
The latest performance improvements (#370) brought performance in BotW up to parity with the Windows version, at least on my Steam Deck. Great to see!
Got v2.0.9 opening on Ubuntu 22.04.1 by copying libsepol.so.1 and libffi.so.7 to /usr/lib/x86_64-linux-gnu (used sudo find / -name to just copy them from snap).
Then general settings would crash it, along with trying to open botw. So went into settings.xml and changed, under Graphic tag, to the following:
<api>2</api>
<device>00000000030000000000000000000000</device>
So, looks like only OpenGL wants to run for me. I get some vulkan segmentation error when on api 1.
Notice the 3 in device - not all 0's.
Trying to mess with the settings in the UI reverts them to all 0's and causes crashes again.
Sound doesn't seem to want to work.
PS5 controller works great.
Save files and shader cache copied from Windows works for botw.
Using an AMD Ryzen 7 PRO 4750U.
Looking really good so far! Just a few snags.
AMD FSR upscale support?
@Docmine17 That has nothing to do with Linux support.
Using Wayland and Nvidia I got cemu and botw to work partially. They start up and run great, however:
- there is no sound
- when trying to open general settings it just crashes, logging a segmentation fault:
log
Segmentation fault! Error: signal 11: cemu(handlerDumpingSignal(int)+68) [0x5572d902e8a8] /usr/lib/libc.so.6(+0x38a00) [0x7f0725251a00] /usr/lib/libX11.so.6(XGetWindowAttributes+0x1d) [0x7f07272a5e6d]+0x1d) [0x7f07272a5e6d] /usr/lib/libGLX_nvidia.so.0(+0xa5a83) [0x7f06f84a5a83] /usr/lib/libnvidia-glcore.so.520.56.06(+0x17c8c0a) [0x7f05e1dc8c0a] /usr/lib/libnvidia-glcore.so.520.56.06(+0x17bba1b) [0x7f05e1dbba1b] cemu(SwapchainInfoVk::FindQueueFamilies(VkSurfaceKHR_T*, VkPhysicalDevice_T*)+156) [0x5572d90130b6] cemu(VulkanRenderer::IsDeviceSuitable(VkSurfaceKHR_T*, VkPhysicalDevice_T* const&)+30) [0x5572d8ea5d20] cemu(GeneralSettings2::HandleGraphicsApiSelection()+be0) [0x5572d91614f0] cemu(MainWindow::OpenSettings()+1ebc) [0x5572d90a113c] /usr/lib/libwx_baseu-3.2.so.0(_ZN12wxEvtHandler23ProcessEventIfMatchesIdERK21wxEventTableEntryBasePS_R7wxEvent+0x72) [0x7f0725f83452]+0x72) [0x7f0725f83452] /usr/lib/libwx_baseu-3.2.so.0(_ZN16wxEventHashTable11HandleEventER7wxEventP12wxEvtHandler+0xec) [0x7f0725f837ec]+0xec) [0x7f0725f837ec] /usr/lib/libwx_baseu-3.2.so.0(_ZN12wxEvtHandler11TryHereOnlyER7wxEvent+0x4f) [0x7f0725f8618f]+0x4f) [0x7f0725f8618f] /usr/lib/libwx_baseu-3.2.so.0(_ZN12wxEvtHandler19ProcessEventLocallyER7wxEvent+0x2f) [0x7f0725f8620f]+0x2f) [0x7f0725f8620f] /usr/lib/libwx_baseu-3.2.so.0(_ZN12wxEvtHandler12ProcessEventER7wxEvent+0xda) [0x7f0725f8632a]+0xda) [0x7f0725f8632a] /usr/lib/libwx_gtk3u_core-3.2.so.0(_ZN12wxWindowBase8TryAfterER7wxEvent+0x60) [0x7f0726672ce0]+0x60) [0x7f0726672ce0] /usr/lib/libwx_baseu-3.2.so.0(_ZN12wxEvtHandler18SafelyProcessEventER7wxEvent+0xb) [0x7f0725f86cdb]+0xb) [0x7f0725f86cdb] /usr/lib/libwx_gtk3u_core-3.2.so.0(_ZN10wxMenuBase14DoProcessEventEPS_R7wxEventP8wxWindow+0x6c) [0x7f072662f1cc]+0x6c) [0x7f072662f1cc] /usr/lib/libwx_gtk3u_core-3.2.so.0(_ZN10wxMenuBase9SendEventEii+0xc3) [0x7f072662f2d3]+0xc3) [0x7f072662f2d3] /usr/lib/libwx_gtk3u_core-3.2.so.0(ImGui::Begin(char const*, bool*, int) [clone .constprop.0]+28f) [0x7f072682e9cf] /usr/lib/libgobject-2.0.so.0(g_closure_invoke+0x170) [0x7f0725cb9210]+0x170) [0x7f0725cb9210] /usr/lib/libgobject-2.0.so.0(+0x41ea8) [0x7f0725ce6ea8] /usr/lib/libgobject-2.0.so.0(g_signal_emit_valist+0x1045) [0x7f0725cd6f75]+0x1045) [0x7f0725cd6f75] /usr/lib/libgobject-2.0.so.0(g_signal_emit+0x94) [0x7f0725cd7204]+0x94) [0x7f0725cd7204] /usr/lib/libgtk-3.so.0(gtk_widget_activate+0x5d) [0x7f0724b2e69d]+0x5d) [0x7f0724b2e69d] /usr/lib/libgtk-3.so.0(gtk_menu_shell_activate_item+0x14d) [0x7f07249f85cd]+0x14d) [0x7f07249f85cd] /usr/lib/libgtk-3.so.0(PPCRecompiler_invalidateRange(unsigned int, unsigned int)+23a) [0x7f07249f889a] /usr/lib/libgtk-3.so.0(+0x87c89) [0x7f0724887c89] /usr/lib/libgobject-2.0.so.0(g_signal_emit_valist+0x11ec) [0x7f0725cd711c]+0x11ec) [0x7f0725cd711c] /usr/lib/libgobject-2.0.so.0(g_signal_emit+0x94) [0x7f0725cd7204]+0x94) [0x7f0725cd7204] /usr/lib/libgtk-3.so.0(+0x3447f5) [0x7f0724b447f5] /usr/lib/libgtk-3.so.0(std::__detail::_Compiler >::_M_quantifier()+63b) [0x7f07249e20db] /usr/lib/libgtk-3.so.0(gtk_main_do_event+0xd6b) [0x7f07249e2fbb]+0xd6b) [0x7f07249e2fbb] /usr/lib/libgdk-3.so.0(+0x3ccd3) [0x7f0725d42cd3] /usr/lib/libgdk-3.so.0(+0x70c78) [0x7f0725d76c78] /usr/lib/libglib-2.0.so.0(g_main_context_dispatch+0x19b) [0x7f072454887b]+0x19b) [0x7f072454887b] /usr/lib/libglib-2.0.so.0(+0xac279) [0x7f072459f279] /usr/lib/libglib-2.0.so.0(g_main_loop_run+0x6f) [0x7f0724547ddf]+0x6f) [0x7f0724547ddf] /usr/lib/libgtk-3.so.0(gtk_main+0x9f) [0x7f07249d8ebf]+0x9f) [0x7f07249d8ebf] /usr/lib/libwx_gtk3u_core-3.2.so.0(_ZN14wxGUIEventLoop5DoRunEv+0x26) [0x7f07267b2ff6]+0x26) [0x7f07267b2ff6] /usr/lib/libwx_baseu-3.2.so.0(_ZN15wxEventLoopBase3RunEv+0x32) [0x7f0725ed07a2]+0x32) [0x7f0725ed07a2] /usr/lib/libwx_baseu-3.2.so.0(_ZN16wxAppConsoleBase8MainLoopEv+0x78) [0x7f0725eace58]+0x78) [0x7f0725eace58] /usr/lib/libwx_baseu-3.2.so.0(_Z7wxEntryRiPPw+0x50) [0x7f0725f0ef30]+0x50) [0x7f0725f0ef30] cemu(main+98) [0x5572d8d7d558] /usr/lib/libc.so.6(+0x23290) [0x7f072523c290] /usr/lib/libc.so.6(__libc_start_main+0x8a) [0x7f072523c34a]+0x8a) [0x7f072523c34a] cemu(_start+25) [0x5572d8dd46b5]
Please open up a separate issue and provide more details like operation system, how you compiled it/whether you used the appimage etc.
there is no sound
I also had no sound at first and was confused. I needed to select an audio device in General Settings -> Audio -> TV. It doesn't seem to default to the first/default one on setup.
there is no sound
I also had no sound at first and was confused. I needed to select an audio device in General Settings -> Audio -> TV. It doesn't seem to default to the first/default one on setup.
Yep, that's also what I tried, but I get a crash whenever I try to open general settings. More info in issue #527 (there is no reaction there yet tho)
Is it possible to edit the settings without the UI? The sound is honestly the only thing preventing me to start playing
I've been using the AppImages of Cemu (2.0.17, 2.0.18 and 2.0.19) for a few weeks now, no crashes and everything works great and fast (at least for BoTW), except that the app doesn't seem to respect the system dark theme on KDE.
./Cemu-2.0-19-x86_64.AppImage
Gtk-Message: 01:18:26.863: Failed to load module "colorreload-gtk-module"
Gtk-Message: 01:18:26.863: Failed to load module "window-decorations-gtk-module"(AppRun.wrapped:12339): Gtk-CRITICAL **: 01:18:26.904: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar
Is it possible to edit the settings without the UI? The sound is honestly the only thing preventing me to start playing
This is what I've got in mine /home/bunkbail/.config/Cemu/settings.xml
<Audio>
<api>3</api>
<delay>2</delay>
<TVChannels>1</TVChannels>
<PadChannels>1</PadChannels>
<InputChannels>0</InputChannels>
<TVVolume>100</TVVolume>
<PadVolume>0</PadVolume>
<InputVolume>50</InputVolume>
<TVDevice>alsa_output.pci-0000_09_00.4.analog-stereo</TVDevice>
<PadDevice></PadDevice>
<InputDevice></InputDevice>
</Audio>
I updated the original post to match the current state of development.
Compiling from source in Ubuntu 22.04 , following the instructions on the site, gives me this error
cmake -S . -B build -DCMAKE_BUILD_TYPE=release -DCMAKE_C_COMPILER=/usr/bin/clang-12 -DCMAKE_CXX_COMPILER=/usr/bin/clang++-12 -G Ninja -DCMAKE_MAKE_PROGRAM=/usr/bin/ninja
CMake Error at /usr/share/cmake-3.22/Modules/CMakeDetermineSystem.cmake:130 (message):
Could not find toolchain file:
/home/user/Cemu_2.0-25/Cemu-main/dependencies/vcpkg/scripts/buildsystems/vcpkg.cmake
Call Stack (most recent call first):
CMakeLists.txt:31 (project)
Trying to use the ebuild in tatsh-overlay on gentoo; Am getting the following:
-- Could NOT find wxWidgets (missing: wxWidgets_DIR)
^[[31mCMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find wxWidgets (missing: wxWidgets_LIBRARIES) (Required is at
least version "3.2")
Call Stack (most recent call first):
/usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
/usr/share/cmake/Modules/FindwxWidgets.cmake:977 (find_package_handle_standard_args)
cmake/FindwxWidgets.cmake:17 (find_package)
CMakeLists.txt:143 (find_package)
meanwhile:
$ eselect wxwidgets show
Current wxWidgets profile:
gtk3-unicode-3.2-gtk3with wxGTK 3.2.1 being installed as a prereq for the ebuild.
Any ideas?
You should file an issue on my repo if you are having an issue with the ebuilds.
Cemu is covered in the readme: https://github.com/Tatsh/tatsh-overlay#cemu
I REJOICE FOR FREEDOM!
Any way to merge or transition from appimage to flatpak? Like configs and such?
You can copy the Cemu directory from ~/.local/share to ~/.var/info.cemu.Cemu/data and from ~/.config to ~/.var/info.cemu.Cemu/config
Checking in tried to play Lego dimensions and couldn't get the portal to work looks like that's because hid support isn't in the Linux version yet
I think the flatpak is stuck in stable and they don't have a beta branch on flatpak
In case it helps anyone else who couldn't get the audio devices to populate:
- Run
pactl list short sourcesand choose a sink id starting withalsa_output - Open the settings.xml file and in the
<Audio>tag, add the sink id to<TVDevice></TVDevice> - Open Cemu and the audio device should be populated, and you'll be able to select other audio devices too
Is there any tracking issue or PR on the missing HID support?
Just tested Lego dimensions on arch using the AUR package and can confirm that the the postal works and detects figures
Hey all, one suggestion for the Cemu linux builds is to change the directory for shadercache. Currently, they are stored in ~/.cache/Cemu/shaderCache which seems sensible considering the name, but people who clear their cache frequently will also end up deleting this folder, causing them to have to recompile all shaders.
people who clear their cache frequently will also end up deleting this folder, causing them to have to recompile all shaders.
That's the point of the common cache folder, though? Removing that entire folder will affect many other applications: it's a convenient way to clear up some space, splitting that out into separate custom places per app is a step backwards.
Exactly, why would a user delete the cache folder and not expect everything to have to be re-cached again?
Maybe the solution would be to not blindly delete the entire cache folder to try to save space 🙃
I completely understand where you're coming from with the precompiled shader cache, but I think for the transferable cache these are not usually supposed to be removed. They would take a while to regenerate if you were to play again.
So why are these builds experimental? Linux support looks to be completely fine??? I'm on Steam Deck btw
Experimental just means that it didn't go through the rigorous testing and bug fixing period that comes with calling a release stable. But I agree that the state of the Linux releases is pretty good and this issue has runs it's course. Let's close it