build: give the presym preprocess the header dependencies of its object - #7243

The presym preprocess of a source, the .pi file the symbol table is scanned
from, depends on the source and the build config only. A header that changes
which symbols a source uses leaves its .pi as it was, and the table misses
the change. Switching to a branch that renames the symbol
MRB_EXC_EXIT_STATUS reads in include/mruby/error.h:

-#define MRB_EXC_EXIT_STATUS(mrb,e) ((int)mrb_as_int((mrb),mrb_obj_iv_get((mrb),(e),MRB_IVSYM(status))))
+#define MRB_EXC_EXIT_STATUS(mrb,e) ((int)mrb_as_int((mrb),mrb_obj_iv_get((mrb),(e),MRB_IVSYM(status_probe))))

and running rake in a build directory that was built before the switch:

$ rake
include/mruby/presym.h:38:25: error: ‘MRB_IVSYM__status_probe’ undeclared (first use in this function); did you mean ‘MRB_IVSYM__status’?
include/mruby/error.h:29:84: note: in expansion of macro ‘MRB_IVSYM’
include/mruby/error.h:31:67: note: in expansion of macro ‘MRB_EXC_EXIT_STATUS’
rake aborted!
Command failed with status (1): [gcc -MMD -c ... mrbgems/mruby-bin-mirb/tools/mirb/mirb.c]

Switching back fails the same way, with MRB_IVSYM__status undeclared, until
the .pi files are deleted by hand. A header that only drops a use of a
symbol does not fail; it leaves the entry in the table, so the table and
presym/id.h differ from a clean build's until one is made.

Why

Compiler#get_dependencies reads the .d file the compile writes only for
the object:

      dep_file = file.ext(".d")
      return deps unless object_ext?(file) && File.exist?(dep_file)

so a .pi depends on its .c, MRUBY_CONFIG and the gem's mrbgem.rake,
and on no header. The object does depend on the header, through the same
.d, so it is recompiled against a table that was scanned from a stale
.pi.

The .pi used to have header dependencies of its own: 456878b wrote a
.i.d beside the .o.d, on the reasoning that sharing the .d would make
the .i depend on the presym include and circulate. d95ffb0 stopped
writing the .pi.d, and d90abc6 folded the .o.d back into .d with the
object_ext? guard that is there now.

What this does

The compile of the object and the presym preprocess of the same source run
the preprocessor over the same source with the same include paths, so the
.d the compile writes lists the headers the preprocess reads too. The
guard is dropped and both read it.

The presym headers under build/<name>/include/mruby/presym are left out
of the preprocess's dependencies. They are made from the preprocessed
files, and the scan does not include them (mruby/presym.h skips them
under MRB_PRESYM_SCANNING), so keeping them would only preprocess every
source once more on the run after the table changed: with them in, adding
one MRB_SYM() to src/array.c and running rake twice does 154 CPP on
the first run and 153 on the second; with them out, the second run does
nothing.

On a clean build no .d exists when the .pi rules are resolved, so the
first build is unchanged; the dependency takes effect from the second run.

A .d that names a header that no longer exists

Reading the .d for the .pi meets a case the object already had. When
a header the .d names is deleted or renamed afterwards,
Rake::TaskManager#attempt_rule gives up on the rule, since the source
neither exists nor has a task, and says nothing: the object keeps only the
plain file task tasks/presym.rake defines for it, whose one dependency is
the presym proxy, so its .c changing does not rebuild it either, and the
object of the previous build is linked for as long as the .d stays. With
the .pi reading the same .d, its rule gives up too and the .pi drops
out of the scan: the table loses the symbols only that source uses, while
the stale object keeps the numbers of the old table.

The two commits before it close that:

• The presym task writes presym/id.h or presym/table.h again when it
  is gone and the list did not change. The list is the file of the task,
  so Rake runs it only when a .pi is newer than the list; a header that
  is gone marks the task as needed too, and the list itself is not
  touched, since its timestamp is what every object depends on. Only the
  header that is gone is written, since a new id.h recompiles every
  object that includes it. Deleting the generated headers used to leave
  every object with a dependency Rake could not resolve, and the link went
  through with the objects of the previous build; with the next commit
  alone it would fail the compile instead, on the missing id.h.
• get_dependencies removes an output whose .d names a header that no
  longer exists, and leaves that header out of the dependencies, so the
  rule builds the output again and writes a .d that matches the sources
  of now. The .d describes a compile that read that header, so the
  output is stale by its own record.

Verified

build_config/default.rb, rake -m -j16, CCACHE_DISABLE=1, the build
directory built once on master and then switched between the two states.

Build time, wall / user seconds, two runs each where a range is given:

The cost is one -E -P per source that includes the touched header, about
16 ms each in serial, in the run that recompiles that source's object
anyway.

Environment

gcc -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DMRB_USE_COMPLEX -DMRB_USE_BIGINT -DMRB_USE_DEBUG_HOOK src/array.c
gcc -E -P -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DMRB_USE_COMPLEX -DMRB_USE_BIGINT -DMRB_USE_DEBUG_HOOK -DMRB_PRESYM_SCANNING src/array.c

No C source changes, so .text is unaffected in every build.

Summary by CodeRabbit

• Bug Fixes

  • Improved dependency tracking for generated build outputs.
  • Automatically removes stale or missing dependencies and rebuilds affected outputs.
  • Correctly excludes internal pre-symbol header dependencies from preprocessing results.

• Build Improvements

  • Missing generated headers are now recreated automatically, even when the pre-symbol list is unchanged.
  • Only missing headers are regenerated, avoiding unnecessary updates.