thoughts on a potential libsamu

[Stone Tickle]

Hello! Thanks for your work on samurai, I use it every day happily. I was inspired by samurai to work on a re-implementation of meson in C. One of my ideas was to (optionally) statically link samurai inside. This would mean you only need one binary to build meson-based projects, as well as allowing a lot of potential optimizations. So far, I have accomplished this by patching samu.c and renaming main to samu_main. When I was implementing compile_commands.json generation, however, I realized that I could re-use even more of samurai's functionality. Unfortunately samurai was not designed with this use-case in mind.

So my question is: are you interested in/welcoming patches for refactoring samurai slightly to expose a library api?

For reference, here is the re-implementation I have been working on: https://git.sr.ht/~lattis/muon. The patched samurai isn't currently public.

[Stone Tickle]

So, I have done a tiny bit more work. First of all, my branch is public here. I noticed and fixed a few tiny issues when invoking samu (by calling its main) from the same process more than once. I was thinking, that just calling the main function would probably be good enough for my use case. I can get all the functionality by constructing argv, which isn't ideal but also isn't that bad. The only patches samu would need then would be ones that help it clean up enough to be run repeatedly in a parent process.

[Michael Forney]

I was inspired by samurai to work on a re-implementation of meson in C.

Very cool! I was aware of boson, and it seems you have made quite a bit more progress. I have to ask though, what is the expected way to bootstrap it? It seems to defeat the purpose a little bit if I have to install meson in order to build your meson replacement.

One of my ideas was to (optionally) statically link samurai inside. This would mean you only need one binary to build meson-based projects, as well as allowing a lot of potential optimizations.

Could you elaborate on the benefits of a libsamu you hope to gain as opposed to just invoking samu? I'm not convinced that a single binary is better than two, and I'm not quite sure what you have in mind for potential optimizations.

In general, I think writing a good library is much harder than writing a good standalone program. You have to worry about thread safety, namespace issues, avoid all global variables, modify the API so that it can report all failures (including malloc) to the caller instead of just erroring out, free all allocated memory at the end, commit to a stable(ish) API, etc.

[Stone Tickle]

what is the expected way to bootstrap it?

My plan for this was a small script that compiles muon directly with no configuration, and here it is! (Thanks for the reminder 🙂)

what you have in mind for potential optimizations

There are a few micro optimizations like reducing overhead of fork/execing a binary by calling a function, but probably no one cares about those. The biggest use case I can think of would be e.g. development on platforms where getting ninja is difficult.

One of the big motivations for muon is the somewhat painful install process for meson (python3, and then meson, make sure PATH is correct, etc.). If I want to build my project on someone else's machine, it would be cool to just download a single static binary that had everything I needed.

However, in the vast majority of cases, I agree that having two binaries is probably a good thing.

I think writing a good library is much harder than writing a good standalone program

Very true.

I think what I will do is maintain the small patches to get this working in my own branch, and disable embedding samu by default in muon. I will still support it for now though, until I get a chance to try out the above use case, and then I'll re-evaluate!

[Stone Tickle]

So to follow up on this. muon has taken on a port of samurai in its source tree: https://git.sr.ht/~lattis/muon/tree/master/item/src/external/samurai. I have kept as much as I can unmodified so it will be easier to apply patches from upstream. Some changes I made:

• Switch all memory allocation to an arena allocator, and move all globals into a context struct. This makes samurai fully reentrant and removes almost all need to worry about cleanup / leaking memory.
• Replace all platform specific code (poll / posix_spawn / rename / stat) with muon's own platform/ library that provides implementations for both posix systems as well as windows. This means that muon's samurai works on windows (mostly).
• Prefix all symbols with samu_. This lets muon build samu as part of its amalgam build during bootstrap without worry of symbol collisions.