fatal error: hangout_media.pb-c.h: No such file or directory
Massimo-B opened this issue · comments
Emerging purple-hangouts-0_p20181202 on Gentoo fails with
>>> Compiling source in /var/tmp/portage/x11-plugins/purple-hangouts-0_p20181202/work/EionRobb-purple-hangouts-cccf2f62d439 ...
make -j10 -l8 CC=x86_64-pc-linux-gnu-gcc PKG_CONFIG=x86_64-pc-linux-gnu-pkg-config
protoc-c --c_out=. hangouts.proto
x86_64-pc-linux-gnu-gcc -march=native -mtune=native -O2 -pipe -fomit-frame-pointer -fPIC -I/usr/include/protobuf-c -Ipurple2compat -lprotobuf-c `x86_64-pc-linux-gnu-pkg-config purple glib-2.0 json-glib-1.0 zlib --libs --cflags` -c -o hangouts_media.o hangouts_media.c
In file included from /usr/include/libpurple/media/codec.h:35,
from /usr/include/libpurple/media.h:31,
from /usr/include/libpurple/blist.h:111,
from /usr/include/libpurple/buddyicon.h:37,
from /usr/include/libpurple/conversation.h:151,
from /usr/include/libpurple/log.h:53,
from /usr/include/libpurple/account.h:50,
from /usr/include/libpurple/connection.h:149,
from purple2compat/http.h:33,
from hangouts_connection.h:25,
from hangouts_media.h:23,
from hangouts_media.c:21:
/usr/include/libpurple/media/../util.h:1397:13: warning: Deprecated pre-processor symbol, replace with
1397 | G_CONST_RETURN gchar *purple_gai_strerror(gint errnum);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/libpurple/blist.h:111,
from /usr/include/libpurple/buddyicon.h:37,
from /usr/include/libpurple/conversation.h:151,
from /usr/include/libpurple/log.h:53,
from /usr/include/libpurple/account.h:50,
from /usr/include/libpurple/connection.h:149,
from purple2compat/http.h:33,
from hangouts_connection.h:25,
from hangouts_media.h:23,
from hangouts_media.c:21:
/usr/include/libpurple/media.h:165:3: warning: ‘GParameter’ is deprecated [-Wdeprecated-declarations]
165 | guint num_params, GParameter *params);
| ^~~~~
In file included from /usr/include/glib-2.0/gobject/gobject.h:26,
from /usr/include/glib-2.0/gobject/gbinding.h:29,
from /usr/include/glib-2.0/glib-object.h:22,
from /usr/include/libpurple/account.h:31,
from /usr/include/libpurple/connection.h:149,
from purple2compat/http.h:33,
from hangouts_connection.h:25,
from hangouts_media.h:23,
from hangouts_media.c:21:
/usr/include/glib-2.0/gobject/gparam.h:271:8: note: declared here
271 | struct _GParameter /* auxiliary structure for _setv() variants */
| ^~~~~~~~~~~
In file included from /usr/include/libpurple/blist.h:111,
from /usr/include/libpurple/buddyicon.h:37,
from /usr/include/libpurple/conversation.h:151,
from /usr/include/libpurple/log.h:53,
from /usr/include/libpurple/account.h:50,
from /usr/include/libpurple/connection.h:149,
from purple2compat/http.h:33,
from hangouts_connection.h:25,
from hangouts_media.h:23,
from hangouts_media.c:21:
/usr/include/libpurple/media.h:214:3: warning: ‘GParameter’ is deprecated [-Wdeprecated-declarations]
214 | guint num_params, GParameter *params);
| ^~~~~
In file included from /usr/include/glib-2.0/gobject/gobject.h:26,
from /usr/include/glib-2.0/gobject/gbinding.h:29,
from /usr/include/glib-2.0/glib-object.h:22,
from /usr/include/libpurple/account.h:31,
from /usr/include/libpurple/connection.h:149,
from purple2compat/http.h:33,
from hangouts_connection.h:25,
from hangouts_media.h:23,
from hangouts_media.c:21:
/usr/include/glib-2.0/gobject/gparam.h:271:8: note: declared here
271 | struct _GParameter /* auxiliary structure for _setv() variants */
| ^~~~~~~~~~~
In file included from hangouts_media.c:21:
hangouts_media.h:24:10: fatal error: hangout_media.pb-c.h: No such file or directory
24 | #include "hangout_media.pb-c.h"
| ^~~~~~~~~~~~~~~~~~~~~~
It worked fine now with MAKEOPTS -j1. Please check dependencies and parallelization.
Is there any way from inside a makefile to override the -j flag?
It's important that the protoc-c command runs before everything else, and there's only a single gcc command line run, so parallelisation is unnecessary
Seems to be a Gentoo issue. Apparently they patched the Makefile to compile each source file separately. This breaks the dependency (which is not obvious, to be fair).
There's a somewhat-newer (2019-06-06) ebuild available in my overlay that compiles and installs just fine with MAKEOPTS -j4 (for me at least, just re-merged it to check). However it does use the same makefile patch, so ymmv.
I don't recall exactly why I created an updated ebuild; I don't think I had any specific problems with the 20181202 ebuild in the gentoo tree, I just wanted something more up-to-date.
sure, you have a race condition there that gets worse the more cores you add...
Guess I've just been lucky, then. I've updated the package in my overlay to force -j1 on build, thanks for explaining the problem.
Gentoo policy is that packages which don't parallelise cleanly should be patched rather than have the ebuild force -j1. Gentoo's user-set MAKEOPTS="-jN" is applied system-wide to everything that gets built, and from upstream's perspective, the Makefile's only recourse is to include "NOTPARALLEL". Since they insisted on patching the Makefile at all, a quick scan of some documentation tells me that they could have done a much better job of it and probably avoided the whole issue.
I'll open a bug on their bugzilla and reference this issue, hopefully it get's addressed.
Kinda odd that they'd split it out like that. I can't remember if it was on this prpl or a different one, but it turned out to be overall slower to build separate .o's in parallel than it was to do it in a single compile command
Would one of you fancy gentoo folk be kind enough to test the speeds of the patched Makefile vs the original one?
I ran multiple tests with both of them, trying to trigger the race condition. No discernable difference on my system. I've got an arthritic notebook kicking around somewhere, I'll see what the difference is on it.
Well, @Massimo-B triggered it, no?
hangouts_media.c depends on hangout_media.pb-c.h being generated by protoc-c. This depencency is not expressed in the patched Makefile, which ultimately causes the bug. As hangouts_media.c is the 10th file in the list you probably need -j 10 on a machine that has at least 10 cores to trigger it.
The original Makefile basically says "generate all source files before compiling all of them in one step", hence the problem does not occur.
@kgraefe I'm not in doubt that he did trigger it; I only stated that I attempted to do so and failed. I am not arguing with you over whether the problem exists or not.
@EionRobb I used time and ran tests with the patched and unpatched Makefile, on an eight-core and a dual-core machine, both Gentoo boxes with essentially the same configuration. The results are in a gist. The eight-core with -j8 was less than half the compile time vs -j1 on the patched makefile, almost no change (~100ms) on the unpatched. The dual-core saw similar results, but the parallel speed-up wasn't nearly as high.
Max time even on the slower machine was a whopping 12 seconds. This really isn't worth worrying about.
Thanks for that @dallenwilson I guess where I'm going with that is to ascertain whether patching the Makefile is the right thing to do, but if there are penalties for patching then it would make sense to request to get the patch removed.
Otherwise if it were beneficial, I'd roll those changes into this plugin - I'm not really seeing a strong benefit here in those timings though that would warrant parallization.
I agree, there's not enough of a time reduction to be worth worrying about. I don't think you should have to make any changes. Gentoo introduced the problem, so it's only fair they deal with it. There's a bug open to make them aware of it, and I've already removed the makefile changes from the unofficial package I maintain.