Improve DTrace probe generation with non-default compiler by f4z4on · Pull Request #11643 · php/php-src

f4z4on · 2023-07-08T21:17:04Z

With DTrace support enabled during ./configure, our custom Autoconf macro PHP_INIT_DTRACE creates make rules to generate header and object files using dtrace utility. SystemTap† implementation of dtrace relies on other utilities to provide header preprocessing and final object file compilation. These utilities are configured by common environment variables with common defaults:

preprocessor from CPP defaults to "cpp"
compiler from CC defaults to "gcc"
compiler arguments can be expanded with CFLAGS

This has been in SystemTap since version 1.5 released on 2011-05-23. We have been setting CFLAGS for dtrace since 717b367 released in versions 5.4.20 and 5.5.4 on 2013-09-18. This change fixed build against SystemTap. It fixes majority of cases since practically all free Linux distributions use SystemTap for DTrace-like dynamic tracing and practically all of them use GCC or compatible compiler suite. However, this becomes an issue when cross-compiling using GCC because utility names contain target triplets. Autoconf already handles cross-compilation well—setting correct CC and CPP make macros (variables).

Therefore, we simply set CC and CPP environment variables using respective macros when executing dtrace. Although SystemTap dtrace does not always use CC nor CPP, we set it every time. SystemTap documentation does not talk about this at all, so it is safer to always set it. We also follow how we set CFLAGS every time in the past.

Original (or ported) DTrace mainly used on Oracle Linux, Solaris, macOS, FreeBSD or NetBSD ignores these and does not support cross compilation.

† Well-known dynamic tracing infrastructure for Linux compatible with statically-defined tracing from DTrace.

With DTrace support enabled during ./configure, our custom Autoconf macro PHP_INIT_DTRACE creates make rules to generate header and object files using dtrace utility. SystemTap† implementation of dtrace relies on other utilities to provide header preprocessing and final object file compilation. These utilities are configured by common environment variables with common defaults:‡ * preprocessor from CPP defaults to “cpp” * compiler from CC defaults to “gcc” * compiler arguments can be expanded with CFLAGS This has been in SystemTap since version 1.5 released on 2011-05-23. We have been setting CFLAGS for dtrace since 717b367 released in versions 5.4.20 and 5.5.4 on 2013-09-18. This change fixed build against SystemTap. It fixes majority of cases since practically all free Linux distributions use SystemTap for DTrace-like dynamic tracing and practically all of them use GCC or compatible compiler suite. However, this becomes an issue when cross-compiling using GCC because utility names contain target triplets. Autoconf already handles cross-compilation well —setting correct CC and CPP make macros (variables). Therefore, we simply set CC and CPP environment variables using respective macros when executing dtrace. Although SystemTap dtrace does not always use CC nor CPP, we set it every time. SystemTap documentation does not talk about this at all¶, so it is safer to always set it. We also follow how we set CFLAGS every time in the past. Original (or ported) DTrace mainly used on Oracle Linux, Solaris and macOS ignores these and does not support cross compilation.§ † Well-known dynamic tracing infrastructure for Linux compatible with statically-defined tracing from DTrace. ‡ https://sourceware.org/git/?p=systemtap.git;a=blob;f=dtrace.in;h=73a6f22e2de072773c692e3fea05c4b8cf814e43;hb=ebb424eee5599fcc131901c0d82d0bfc0d2f57ab ¶ https://sourceware.org/systemtap/man/dtrace.1.html § https://docs.oracle.com/cd/E88353_01/html/E72487/dtrace-8.html

petk · 2023-07-14T06:56:43Z

Thank you @f4z4on for this patch. This is ok to merge, yes.

I'm only wondering if we should at the same time enable this also for the OCI8 extension (OCI8 extension is otherwise a bit special and @cjbj usually needs to add this because the extension sources are on two places):

diff --git a/ext/oci8/config.m4 b/ext/oci8/config.m4
index 714c655be4..883e272fb1 100644
--- a/ext/oci8/config.m4
+++ b/ext/oci8/config.m4
@@ -124,7 +124,7 @@ dnl overwritten (Bug 61268).
 PHP_EXT_SRCDIR([oci8])/$ac_provsrc:;
 
 $ac_bdir[$]ac_hdrobj: $ac_srcdir[$]ac_provsrc
-	CFLAGS="\$(CFLAGS_CLEAN)" dtrace -h -C -s $ac_srcdir[$]ac_provsrc -o \$[]@.bak && \$(SED) -e 's,PHPOCI_,DTRACE_,g' \$[]@.bak > \$[]@
+	CPP="\$(CPP)" CC="\$(CC)" CFLAGS="\$(CFLAGS_CLEAN)" dtrace -h -C -s $ac_srcdir[$]ac_provsrc -o \$[]@.bak && \$(SED) -e 's,PHPOCI_,DTRACE_,g' \$[]@.bak > \$[]@
 
 \$(OCI8_DTRACE_OBJS): $ac_bdir[$]ac_hdrobj
 
@@ -145,12 +145,12 @@ EOF
 $ac_bdir[$]ac_provsrc.lo: \$(OCI8_DTRACE_OBJS)
 	echo "[#] Generated by Makefile for libtool" > \$[]@
 	@test -d "$dtrace_lib_dir" || mkdir $dtrace_lib_dir
-	if CFLAGS="\$(CFLAGS_CLEAN)" dtrace -G -o $dtrace_d_obj -s $ac_srcdir[$]ac_provsrc $dtrace_oci8_lib_objs 2> /dev/null && test -f "$dtrace_d_obj"; then [\\]
+	if CPP="\$(CPP)" CC="\$(CC)" CFLAGS="\$(CFLAGS_CLEAN)" dtrace -G -o $dtrace_d_obj -s $ac_srcdir[$]ac_provsrc $dtrace_oci8_lib_objs 2> /dev/null && test -f "$dtrace_d_obj"; then [\\]
 	  echo "pic_object=['].libs/$dtrace_prov_name[']" >> \$[]@ [;\\]
 	else [\\]
 	  echo "pic_object='none'" >> \$[]@ [;\\]
 	fi
-	if CFLAGS="\$(CFLAGS_CLEAN)" dtrace -G -o $ac_bdir[$]ac_provsrc.o -s $ac_srcdir[$]ac_provsrc $dtrace_nolib_objs 2> /dev/null && test -f "$ac_bdir[$]ac_provsrc.o"; then [\\]
+	if CPP="\$(CPP)" CC="\$(CC)" CFLAGS="\$(CFLAGS_CLEAN)" dtrace -G -o $ac_bdir[$]ac_provsrc.o -s $ac_srcdir[$]ac_provsrc $dtrace_nolib_objs 2> /dev/null && test -f "$ac_bdir[$]ac_provsrc.o"; then [\\]
 	  echo "non_pic_object=[']$dtrace_prov_name[']" >> \$[]@ [;\\]
 	else [\\]
 	  echo "non_pic_object='none'" >> \$[]@ [;\\]
@@ -162,7 +162,7 @@ EOF
     AC_MSG_WARN([OCI8 extension: OCI8 DTrace support is not confirmed on this platform])
 cat>>Makefile.objects<<EOF
 $ac_bdir[$]ac_provsrc.o: \$(OCI8_DTRACE_OBJS)
-	CFLAGS="\$(CFLAGS_CLEAN)" dtrace -G -o \$[]@ -s $ac_srcdir[$]ac_provsrc $dtrace_oci8_objs
+	CPP="\$(CPP)" CC="\$(CC)" CFLAGS="\$(CFLAGS_CLEAN)" dtrace -G -o \$[]@ -s $ac_srcdir[$]ac_provsrc $dtrace_oci8_objs
 
 EOF
     ;;

cjbj · 2023-07-17T07:56:27Z

Assuming the PR doesn't break OCI8 building I don't think this PR should be blocked waiting for OCI8 to add the equivalent.
Having said that, if the comment patch by @petk builds OCI8 cleanly, it can be added now to the master branch.

petk · 2023-07-19T13:32:23Z

This has been merged to master (upcoming PHP-8.3). I've modified also oci8/config.m4 accordingly. Thanks @f4z4on and @cjbj

We are experiencing an issue when building PHP with DTrace enabled with SystemTap (see phpGH-11847).† The issue is caused by inappropriate use C preprocessor detected by GNU Autoconf in our “configure” script. C preprocessor configuration found by AC_PROG_CPP macro is portable only to run on files with “.c” extension.‡ However, statically-defined tracing is described by D programs with “.d” extension which causes the issue. We experience this even on typical Linux distribution with GNU Compiler Collection (GCC) unless we override the defaults detected by our “configure” script. Many major Linux distributions use SystemTap to provide “dtrace” utility. It relies on both external C preprocessor and external C compiler. C preprocessor can be customized via CPP environment variable. Similarly, C compiler can be customized via CC environment variable. It also allows customization of C compiler flags via CFLAGS environment variable. We have recently aligned both CPP and CC environment variable with C preprocessor and C compiler we use to build regular C source code as provided by our “configure” script (see phpGH-11643).* We wanted to allow cross-compilation on Linux for which this was the only blocker. C compiler flags from CFLAGS_CLEAN macro have already been in place since versions 5.4.20 and 5.5.4 from 2013-09-18. We had modified all “dtrace” invocations in the same way to make it look consistent. However, only the C compiler (CC environment variable) is necessary to for cross-compilation. There have never been any reported issue with the C preprocessor. We acknowledge it would be great to allow C preprocessor customization as well. However, the implementation would require a lot of effort to do correctly given the limitations of AC_PROG_CPP macro from GNU Autoconf. This would be further complicated by the fact that all DTrace implementations, not just SystemTap, allow C preprocessor customization but Oracle DTrace, Open DTrace, and their forks do it differently. Nevertheless, they all default to “cpp” utility and they all have or had been working fine. Therefore, we believe simply removing CPP stabilizes “dtrace” invocation on Linux systems with SystemTap and aligns it with other system configurations on other platforms, until someone comes with complete solution with custom “m4” and “make” macros, while our build system on Linux with SystemTap supports cross-compilation. Fixes phpGH-11847 † php#11847 ‡ https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/autoconf.html#index-AC_005fPROG_005fCPP-1 * php#11643

We are experiencing an issue when building PHP with DTrace enabled with SystemTap (see GH-11847).† The issue is caused by inappropriate use C preprocessor detected by GNU Autoconf in our “configure” script. C preprocessor configuration found by AC_PROG_CPP macro is portable only to run on files with “.c” extension.‡ However, statically-defined tracing is described by D programs with “.d” extension which causes the issue. We experience this even on typical Linux distribution with GNU Compiler Collection (GCC) unless we override the defaults detected by our “configure” script. Many major Linux distributions use SystemTap to provide “dtrace” utility. It relies on both external C preprocessor and external C compiler. C preprocessor can be customized via CPP environment variable. Similarly, C compiler can be customized via CC environment variable. It also allows customization of C compiler flags via CFLAGS environment variable. We have recently aligned both CPP and CC environment variable with C preprocessor and C compiler we use to build regular C source code as provided by our “configure” script (see GH-11643).* We wanted to allow cross-compilation on Linux for which this was the only blocker. C compiler flags from CFLAGS_CLEAN macro have already been in place since versions 5.4.20 and 5.5.4 from 2013-09-18. We had modified all “dtrace” invocations in the same way to make it look consistent. However, only the C compiler (CC environment variable) is necessary to for cross-compilation. There have never been any reported issue with the C preprocessor. We acknowledge it would be great to allow C preprocessor customization as well. However, the implementation would require a lot of effort to do correctly given the limitations of AC_PROG_CPP macro from GNU Autoconf. This would be further complicated by the fact that all DTrace implementations, not just SystemTap, allow C preprocessor customization but Oracle DTrace, Open DTrace, and their forks do it differently. Nevertheless, they all default to “cpp” utility and they all have or had been working fine. Therefore, we believe simply removing CPP stabilizes “dtrace” invocation on Linux systems with SystemTap and aligns it with other system configurations on other platforms, until someone comes with complete solution with custom “m4” and “make” macros, while our build system on Linux with SystemTap supports cross-compilation. Fixes GH-11847 Closes GH-12083 † #11847 ‡ https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/autoconf.html#index-AC_005fPROG_005fCPP-1 * #11643

github-actions Bot added the Category: Build System label Jul 8, 2023

f4z4on changed the title ~~Improve DTrace probe generation /w non-default compiler~~ Improve DTrace probe generation with non-default compiler Jul 8, 2023

f4z4on mentioned this pull request Jul 9, 2023

DTrace & SystemTap support on all architectures oerdnj/deb.sury.org#1960

Closed

Girgias requested a review from petk July 10, 2023 15:41

petk closed this in 475fd29 Jul 19, 2023

This was referenced Aug 2, 2023

DTrace enabled build is broken #11847

Closed

Set DTrace preprocessor via the DTRACE_CPP variable #11853

Closed

f4z4on deleted the dtrace-cross-compile branch August 2, 2023 13:05

f4z4on mentioned this pull request Aug 2, 2023

Fix DTrace build with SystemTap and GCC defaults #11858

Closed

f4z4on mentioned this pull request Aug 30, 2023

Remove CPP when invoking dtrace utility #12083

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve DTrace probe generation with non-default compiler#11643

Improve DTrace probe generation with non-default compiler#11643
f4z4on wants to merge 1 commit into
php:masterfrom
f4z4on:dtrace-cross-compile

f4z4on commented Jul 8, 2023 •

edited

Loading

Uh oh!

petk commented Jul 14, 2023

Uh oh!

cjbj commented Jul 17, 2023 •

edited

Loading

Uh oh!

petk commented Jul 19, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

f4z4on commented Jul 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petk commented Jul 14, 2023

Uh oh!

cjbj commented Jul 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petk commented Jul 19, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

f4z4on commented Jul 8, 2023 •

edited

Loading

cjbj commented Jul 17, 2023 •

edited

Loading