[linux headers] Update build to support >= 6.3 kernels. Add 6.6 headers to fix ARM Container OS issue #2036
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ddelnano
commented
Oct 2, 2024
This was referenced Oct 2, 2024
ddelnano
added a commit
that referenced
this pull request
Oct 11, 2024
…er kernels (#2041) Summary: Upgrade bcc and libbpf to fix BPF program compilation on 6.10 and later kernels Bcc provides some "[virtual](https://github.com/iovisor/bcc/blob/cb1ba20f4800f556dc940682ba7016c50bd0a3ac/src/cc/exported_files.cc#L28-L48)" includes to BPF programs. The `compat/linux/virtual_bpf.h` file in particular needs to be kept in sync with libbpf and matches the [header guard](https://github.com/iovisor/bcc/blob/cb1ba20f4800f556dc940682ba7016c50bd0a3ac/src/cc/compat/linux/virtual_bpf.h#L9) of the `include/uapi/linux/bpf.h` file. This means that while our linux headers were updated, our older bcc install was inserting an older copy of the `uapi/linux/bpf.h` file -- one that didn't contain the `bpf_wq` declaration. ``` include/linux/bpf.h:348:10: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_wq' return sizeof(struct bpf_wq); ^ ~~~~~~~~~~~~~~~ include/linux/bpf.h:348:24: note: forward declaration of 'struct bpf_wq' return sizeof(struct bpf_wq); ^ include/linux/bpf.h:377:10: error: invalid application of '__alignof' to an incomplete type 'struct bpf_wq' return __alignof__(struct bpf_wq); ^ ~~~~~~~~~~~~~~~ include/linux/bpf.h:377:29: note: forward declaration of 'struct bpf_wq' return __alignof__(struct bpf_wq); ``` Note: while this fixes the 6.10 compilation issue, our 6.10 qemu build fails without disabling [this logic](https://github.com/pixie-io/pixie/blob/3c41d554215528e688328aef94192e696db617dc/src/stirling/source_connectors/socket_tracer/socket_trace_connector.cc#L464-L472). 6.10 kernels added BPF token support. This changes the BPF permission model slightly and causes the BPF instruction limit to be dependent on the permissions of the BPF syscall caller ([linux source](https://elixir.bootlin.com/linux/v6.11.1/source/kernel/bpf/syscall.c#L2757)). This new BPF token logic coupled with our qemu setup, causes our 6.10 build to fallback to the 4096 instruction limit. I'll be addressing this in #2040 and #2042. Those issues shouldn't block this change since that loop limit code can be bypasses at runtime with our current cli flags. Relevant Issues: Closes #2035 Type of change: /kind bugfix Test Plan: Built 6.10 and 6.11 kernels and the associated linux headers from #2036 and verified that a local qemu build passes - [x] Verify `#ci:bpf-build-all-kernels` build passes Changelog Message: Upgraded bcc and libbpf to support kernels 6.10 and later --------- Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
ddelnano
force-pushed
the
ddelnano/add-newer-linux-kernel-headers
branch
from
147d58d to
a66b431
Compare
ddelnano
changed the title
ddelnano
changed the title
…kernel versions to header builder list Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
… arm Container OS Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
ddelnano
force-pushed
the
ddelnano/add-newer-linux-kernel-headers
branch
from
4720824 to
f485f95
Compare
vihangm
reviewed
May 6, 2025
Member
vihangm
left a comment
vihangm
left a comment
There was a problem hiding this comment.
can the older kernels be built with the 20.04 base image?
in that case, I wonder if it's cleaner to move the header build into a shell script and not fork the Dockerfile.
…mmands Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
Member
Author
|
@vihangm unfortunately older kernel versions fail to build in the focal image. Here's a log of the error when building the 4.15.x: Kernel build failure |
vihangm
approved these changes
May 6, 2025
oazizi000
reviewed
May 12, 2025
Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
…multiple Dockerfiles Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
Member
Author
|
@vihangm I ended up consolidating the docker images by refactoring the build logic in a shell script. Highlighting this in case you want to look over it again. |
…y build args in favor of pure env var usage Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
ddelnano
force-pushed
the
ddelnano/add-newer-linux-kernel-headers
branch
from
5a5ac3f to
925f5f2
Compare
vihangm
approved these changes
May 14, 2025
oazizi000
approved these changes
May 15, 2025
ddelnano
added a commit
to ddelnano/pixie
that referenced
this pull request
Aug 6, 2025
…er kernels (pixie-io#2041) Summary: Upgrade bcc and libbpf to fix BPF program compilation on 6.10 and later kernels Bcc provides some "[virtual](https://github.com/iovisor/bcc/blob/cb1ba20f4800f556dc940682ba7016c50bd0a3ac/src/cc/exported_files.cc#L28-L48)" includes to BPF programs. The `compat/linux/virtual_bpf.h` file in particular needs to be kept in sync with libbpf and matches the [header guard](https://github.com/iovisor/bcc/blob/cb1ba20f4800f556dc940682ba7016c50bd0a3ac/src/cc/compat/linux/virtual_bpf.h#L9) of the `include/uapi/linux/bpf.h` file. This means that while our linux headers were updated, our older bcc install was inserting an older copy of the `uapi/linux/bpf.h` file -- one that didn't contain the `bpf_wq` declaration. ``` include/linux/bpf.h:348:10: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_wq' return sizeof(struct bpf_wq); ^ ~~~~~~~~~~~~~~~ include/linux/bpf.h:348:24: note: forward declaration of 'struct bpf_wq' return sizeof(struct bpf_wq); ^ include/linux/bpf.h:377:10: error: invalid application of '__alignof' to an incomplete type 'struct bpf_wq' return __alignof__(struct bpf_wq); ^ ~~~~~~~~~~~~~~~ include/linux/bpf.h:377:29: note: forward declaration of 'struct bpf_wq' return __alignof__(struct bpf_wq); ``` Note: while this fixes the 6.10 compilation issue, our 6.10 qemu build fails without disabling [this logic](https://github.com/pixie-io/pixie/blob/3c41d554215528e688328aef94192e696db617dc/src/stirling/source_connectors/socket_tracer/socket_trace_connector.cc#L464-L472). 6.10 kernels added BPF token support. This changes the BPF permission model slightly and causes the BPF instruction limit to be dependent on the permissions of the BPF syscall caller ([linux source](https://elixir.bootlin.com/linux/v6.11.1/source/kernel/bpf/syscall.c#L2757)). This new BPF token logic coupled with our qemu setup, causes our 6.10 build to fallback to the 4096 instruction limit. I'll be addressing this in pixie-io#2040 and pixie-io#2042. Those issues shouldn't block this change since that loop limit code can be bypasses at runtime with our current cli flags. Relevant Issues: Closes pixie-io#2035 Type of change: /kind bugfix Test Plan: Built 6.10 and 6.11 kernels and the associated linux headers from pixie-io#2036 and verified that a local qemu build passes - [x] Verify `#ci:bpf-build-all-kernels` build passes Changelog Message: Upgraded bcc and libbpf to support kernels 6.10 and later --------- Signed-off-by: Dom Del Nano <ddelnano@gmail.com> GitOrigin-RevId: 738111f
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: Update linux_headers_image to support kernels >= 6.3. Add 6.6 headers to fix ARM Container OS issue
#2035 reported an issue where the socket tracer fails to run on a openSUSE MicroOS instance running a 6.11 kernel. This change updates our header building process to support >= 6.3 kernels since upstream broke the source building process from non git trees (binary builds still work outside of git trees).
This also adds 6.6.x linux headers to the build. A recent bug report identified that recent ARM Container OS clusters fail to start the socket tracer without the 6.6.x headers. COS doesn't have a distro provided linux headers package, which means these installs are dependent on the pre-packaged headers.
Relevant Issues: #2035
Type of change: /kind compatibility
Test Plan: Verified the following