Skip to content

Conversation

@stefanv
Copy link
Contributor

@stefanv stefanv commented Nov 19, 2022

The Python headers use this as, e.g.:

PyAPI_FUNC(int) _PyTime_FromTimespec(_PyTime_t *tp, struct timespec *ts);

When GCC sees this, it complains that the definition of timespec will not be visible outside of the function declaration. However, timespec is defined in time.h.

Forward definition is the fix pipewire applied
(https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/230).

Perhaps there is a better way?

The Python headers use this as, e.g.:

PyAPI_FUNC(int) _PyTime_FromTimespec(_PyTime_t *tp, struct timespec *ts);

When GCC sees this, it complains that the definition of `timespec`
will not be visible outside of the function declaration. However,
`timespec` is defined in `time.h`.

Forward definition is the fix pipewire applied
(https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/230).
@seberg
Copy link
Member

seberg commented Nov 19, 2022

Seems a bit weird that this is necessary, found this (closed) Python issue python/cpython#70040. But if that is needed for the warning 🤷 (maybe a comment that it is just to silence a warning on some setups?)...

@mattip
Copy link
Member

mattip commented Nov 20, 2022

On which platforms/compilations are you seeing the warning?

@stefanv
Copy link
Contributor Author

stefanv commented Nov 20, 2022

This is on Linux (Fedora) compiling the meson branch.

@mattip
Copy link
Member

mattip commented Nov 21, 2022

That function is defined in cpython/pytime.h which is included by Python.h. time.h is included by pyport.h which is included before pytime.h in Python.h.

Perhaps in Fedora the upstream code is modified?

@stefanv
Copy link
Contributor Author

stefanv commented Nov 21, 2022

Looking at the Fedora sources for Python 3.10, I see the same. Here is the section of pyport.h that includes time.h:

#ifdef TIME_WITH_SYS_TIME
#include <sys/time.h>
#include <time.h>
#else /* !TIME_WITH_SYS_TIME */
#ifdef HAVE_SYS_TIME_H
#include <sys/time.h>
#else /* !HAVE_SYS_TIME_H */
#include <time.h>
#endif /* !HAVE_SYS_TIME_H */
#endif /* !TIME_WITH_SYS_TIME */

However, sys/time.h does not include bits/types/struct_timespec.h, where timespec is defined, in comparison to time.h, which does.

I can't find a good reference on when sys/time.h shoudl be preferred; only some comments that it may be "more portable". But since it doesn't define the struct needed, I think Python should always use time.h, rather?

@stefanv
Copy link
Contributor Author

stefanv commented Nov 21, 2022

Maybe the question then is why we don't have TIME_WITH_SYS_TIME defined; that could be the proper fix.

pyconfig-64.h already sets TIME_WITH_SYS_TIME to 1, so there should be no problem here.

@mattip
Copy link
Member

mattip commented Nov 23, 2022

Here is the current HEAD CPython version of the code you pointed to:

/********************************************
 * WRAPPER FOR <time.h> and/or <sys/time.h> *
 ********************************************/

#ifdef HAVE_SYS_TIME_H
#include <sys/time.h>
#endif
#include <time.h>

This was simplified from the code you pointed to in CPython PR python/cpython#29441 without a comment, seemingly to adapt to a change in autoconf.

Edit: the change was applied to CPython3.11+ but was not backported.

@stefanv
Copy link
Contributor Author

stefanv commented Nov 23, 2022

This also appears on CI with Python 3.11.

@stefanv
Copy link
Contributor Author

stefanv commented Nov 23, 2022

OK, I think I figured it out. time.h contains the following:

#if defined __USE_POSIX199309 || defined __USE_ISOC11
# include <bits/types/struct_timespec.h>
#endif

The following patches would make the warning go away, but I'm not sure if we're willing to jump to C11:

diff --git a/meson.build b/meson.build
index 775d59688..3784fe7b0 100644
--- a/meson.build
+++ b/meson.build
@@ -9,7 +9,7 @@ project(
   meson_version: '>= 0.64.0',
   default_options: [
     'buildtype=debugoptimized',
-    'c_std=c99',
+    'c_std=c11',
     'cpp_std=c++11',
     'blas=openblas',
     'lapack=openblas',

Alternatively, setting -D_XOPEN_SOURCE to greater than 600 should accomplish the same (and this is indeed done in pyconfig.h, as well as the system's features.h), but somehow that's not working the way it's supposed to. This is from my local features.h:

# define _XOPEN_SOURCE	700
# define _POSIX_C_SOURCE	200809L
#if defined _POSIX_C_SOURCE && (_POSIX_C_SOURCE - 0) >= 199309L
# define __USE_POSIX199309	1
#endif

While investigating, I came upon this S/O answer that explains it all nicely.

@rgommers
Copy link
Member

Thanks @stefanv! Defining _XOPEN_SOURCE is preferred I think.

Jumping to C11 should be relatively safe by now, but it's hard to be sure on niche platforms. This would be less of a problem if we left c_std undefined (but then we have to deal with lower versions otherwise, because C99 is the actual minimum), or if there was a way to express ">=C99". But that last thing isn't possible yet.

@mattip
Copy link
Member

mattip commented Nov 24, 2022

Nice digging. I wonder why setup.py doesn't run into this. Is it because setup.py and meson use different xxx from sysconfig.get_config_var("xxx") to set the CFLAGS?

@rgommers
Copy link
Member

I wonder why setup.py doesn't run into this. Is it because setup.py and meson use different xxx from sysconfig.get_config_var("xxx") to set the CFLAGS?

It's because there -std=c99 isn't passed for all targets with setup.py, only for a few targets where we've had failures before. So it will show up on older platforms/compilers, but not on the ones we use in CI or locally most likely.

@stefanv
Copy link
Contributor Author

stefanv commented Nov 24, 2022

Do you understand why -std=c99 triggers the error? Does that also implicitly unset any of the defines necessary?

@rgommers
Copy link
Member

Yes, it asks for strict C99 mode, not C99 + POSIX or "whatever the compiler can do".

stefanv added a commit to rgommers/numpy that referenced this pull request Nov 24, 2022
@stefanv
Copy link
Contributor Author

stefanv commented Nov 24, 2022

For future reference, looks like all this magic happens inside of features.h.

Here is the diff between preprocessing directives with and without -std=c11 (see echo | gcc -std=c11 -dM -E -):

-#define linux 1
-#define unix 1
-#define __STDC_VERSION__ 201710L
+#define __STDC_VERSION__ 201112L
+#define __STRICT_ANSI__ 1

It's the __STDC_VERSION__ that affects whether or not struct_timespec.h gets imported:

#if defined __USE_POSIX199309 || defined __USE_ISOC11
# include <bits/types/struct_timespec.h>
#endif

I can confirm that setting _ISOC11_SOURCE makes the warning go away!

@rgommers I've pushed a commit to your branch; not sure if a global flag is the right solution or not, so feel free to drop/replace the patch.

@rgommers
Copy link
Member

@rgommers I've pushed a commit to your branch; not sure if a global flag is the right solution or not, so feel free to drop/replace the patch.

I had a look, this silences the warning but isn't quite the right thing to do. The problem was that some files included standard library headers before they included Python.h. That is always wrong, as explained in the Note box under https://docs.python.org/3/extending/extending.html#a-simple-example. I fixed it by reordering headers in the Meson PR.

I'll also note that you always want to use add_project_arguments instead of add_global_arguments - the difference is scoping, the global version affects Meson subprojects as well (we don't have any yet, but that may change - I was just playing with one this morning).

I suggest we close this PR.

@stefanv stefanv closed this Nov 25, 2022
@stefanv
Copy link
Contributor Author

stefanv commented Nov 25, 2022

Closed by 82fd2b8

For completeness, note that compilation now defaults to c99 and c++14.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants