The use of getline is forbidden with strict C99 on some UNIX systems
blastwave opened this issue · comments
I was trying to avoid ninja and fell upon your samurai. I was happy to
see that the code is written as portable as possible. Until we try a
compile on a Solaris 10 server which is strictly conforming to the
IEEE Std 1003.1 and IEEE Std 1003.2 which are POSIX.1 and POSIX.2,
respectively. From standards(5) we see :
Solaris 10 also supports the X/Open Common Applications
Environment (CAE) Portability Guide Issue 3 (XPG3) and Issue
4 (XPG4); Single UNIX Specification (SUS, also known as
XPG4v2); Single UNIX Specification, Version 2 (SUSv2); and
Single UNIX Specification, Version 3 (SUSv3). Both XPG4 and
SUS include Networking Services Issue 4 (XNS4). SUSv2
includes Networking Services Issue 5 (XNS5).
Therefore with SUSv3 and a few tweaks here and there I can compile all
the code until we hit "getline" because it is only supported in an
update IEEE Std 1003.1-2008. What I have done thus far is the trivial
remove of your _POSIX_C_SOURCE 200809L define and went backwards to an
older #define _XOPEN_SOURCE 600 thus :
alpha $ diff -u build.c.orig build.c
--- build.c.orig Tue Mar 30 09:43:28 2021
+++ build.c Tue Mar 30 09:54:58 2021
@@ -1,4 +1,17 @@
-#define _POSIX_C_SOURCE 200809L
+
+/*********************************************************************
+ * The Open Group Base Specifications Issue 6
+ * IEEE Std 1003.1, 2004 Edition
+ *
+ * An XSI-conforming application should ensure that the feature
+ * test macro _XOPEN_SOURCE is defined with the value 600 before
+ * inclusion of any header. This is needed to enable the
+ * functionality described in The _POSIX_C_SOURCE Feature Test
+ * Macro and in addition to enable the XSI extension.
+ *
+ *********************************************************************/
+#define _XOPEN_SOURCE 600
+
#include <errno.h>
#include <fcntl.h>
#include <inttypes.h>
alpha $
In this way I may use the very strict C99 compiler on Solaris 10 for all
source files. The exception here would be the use of getline in log.c
and this is a show stopper.
I will see if I can replace getline with some portable code after I take
a look into the FreeBSD 13.0 sources as well as other places. On
the other hand I may rewrite log.c entirely. I am not sure yet.
--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken
GreyBeard and suspenders optional
No great surprise that we have the same problem on FreeBSD :
error: implicit declaration of function 'getline' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
while ((linelen = getline(&line, &linecap, fp)) > 0) {
This being a current FreeBSD UNIX build :
europa$ uname -apKU
FreeBSD europa 14.0-CURRENT FreeBSD 14.0-CURRENT #2: Tue Mar 23 10:32:37
GMT 2021 root@europa:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
amd64 amd64 1400006 1400006
I will see what I can come up with.
Another potential problem with targeting older POSIX versions is the use of sub-second precision timestamps in struct stat. Dropping this requirement would be a regression for systems that do support it, so I think the POSIX.1-2008 requirement will have to stay.
That said, I am open to dropping the use of getline in log.c to prefer standard C APIs as much as possible. In fact, a while ago some one was interesting in porting to windows, and I have a commit that does exactly this in the windows branch: 80f3467
It is a bit ugly since fgets provides no good way to detect whether the line requires a larger buffer than it was given. The best thing I could come up with is to set the second to last byte to 0. If it gets set and is not a newline, then we know we need a larger buffer to store the whole line (or it is the last line with no newline). Do you know of a better approach?
No great surprise that we have the same problem on FreeBSD
Do you mean after your diff to lower the POSIX requirement? I have a continuous build set up for FreeBSD which does not seem to have such problems:
https://builds.sr.ht/~mcf/job/370546
I am thinking on this. One thing that I do know is that Solaris 10 and every FreeBSD server that I have do report sub-second times in the struct timespec but I have never looked to closely at what I get from stat. What I see here is fro ma call to stat(some_filename, &status_buffer) :
(gdb) print status_buffer
$1 = {st_dev = 635284480195933068, st_ino = 5337, st_nlink = 1, st_mode = 33188, st_padding0 = 0, st_uid = 16411, st_gid = 20002,
st_padding1 = 0, st_rdev = 18446744073709551615, st_atim = {tv_sec = 1617157250, tv_nsec = 104886000}, st_mtim = {
tv_sec = 1617148380, tv_nsec = 619529000}, st_ctim = {tv_sec = 1617154372, tv_nsec = 687003000}, st_birthtim = {
tv_sec = 1617148298, tv_nsec = 659562000}, st_size = 746, st_blocks = 2, st_blksize = 4096, st_flags = 2048, st_gen = 0,
st_spare = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}
(gdb)
Looks like sub-second times everywhere in there.
I will dig a bit further here.
I would love to see all of this run on older systems as well as with
-D_XOPEN_SOURCE=600 and that takes care of FreeBSD and Solaris
UNIX and very likely every Linux out there. Sorry I have no clue about
windows stuff.
Further data. On a Solaris 10 server I see the time data returned for stat(some_filename, &status_buffer) :
status_buffer = {
st_dev = 1099511693329U
st_ino = 7503534U
st_mode = 33188U
st_nlink = 1U
st_uid = 16411
st_gid = 20002
st_rdev = 18446744073709551615U
st_size = 565
st_atim = {
__tv_sec = 1617157866
__tv_nsec = 528125800
}
st_mtim = {
__tv_sec = 1617157859
__tv_nsec = 493191900
}
st_ctim = {
__tv_sec = 1617157859
__tv_nsec = 493191900
}
st_blksize = 1024
st_blocks = 4
st_fstype = "zfs"
}
So the sub-second precision timestamps data is there.
Not sure what that helps however.
OKay you are saying "fgets provides no good way to detect whether the line requires a larger buffer than it was given."
Let me ponder that a while as I am sure a world of people have worked around that before. I hope.
We've hit the Solaris st_mtim issue before in #59. My understanding is that it does support the st_mtim member of struct stat, but ironically it hides the POSIX.1-2008 timespec members when you request POSIX.1-2008. To work around this we had to add an ugly preprocessor conditional on Solaris to use the libc-internal __tv_sec/__tv_nsec members instead: c44d05d
This issue was reported to illumos here, but hasn't gotten any responses: https://www.illumos.org/issues/13327
Looking a bit closer, it seems that getline is a similar issue to st_mtim. Solaris (at least illumos) supports getline perfectly well, but hides the declaration if you request POSIX.1-2008:
https://github.com/illumos/illumos-gate/blob/master/usr/src/head/stdio.h#L282-L289
Perhaps you can work with your operating system vendor to stop hiding these interfaces when the latest POSIX is requested? An alternative might be to just remove the _POSIX_C_SOURCE define completely or add -D__EXTENSIONS__ to CFLAGS so that the libc will expose these interfaces?
Closing since this seems to be a Solaris POSIX.1-2008 conformance issue, not a case of absent getline interface. We target POSIX.1-2008 (as documented in the README) not C99, and we need at least issue 7 for sub-second timestamps.
OKay, I will consider the software not portable to an older UNIX. No problem.
If you just build without the flags that select POSIX.1-2008 (or with -D__EXTENSIONS__), I believe Solaris will stop hiding these interfaces and it should build no problem.
FYI, I ended up pushing the fgets change.