Support for __ksym and __kfunc

Breaking changes

This release requires at least Go 1.19.

github.com/cilium/ebpf/btf:

• Handle.Spec() now takes a base *Spec argument.
  nil is accepted if the Handle is for vmlinux. If Handle is for a (split BTF) kernel module, pass the result of LoadKernelSpec().
• NewHandle() now takes a *Builder instead of a *Spec.
  Loading BTF into the kernel now goes via a new Builder type. See the 'Additions' section below.

github.com/cilium/ebpf/link:

• KprobeMultiOptions.Addresses: changed from []uint64 to []uintptr
  The old API didn't take 32-bit architectures into account.

Fixes

Work around the kernel rejecting some Datasec

The kernel erroneously rejects Datasec where a Typedef, Volatile, Const, Restrict or typeTag follows a Pointer, Struct, Union or Array. There is now a workaround in place, see #954.

Marshaling Map keys and values is more efficient

The marshaling code in the library now uses sync.Pool to re-use bytes.Buffer, which makes common map operations cheaper, see #1053.

CO-RE relocations are a lot more efficient

The CO-RE code now does much less copying of BTF types, which makes CO-RE relocation a lot faster, especially against large types such as sk_buff. See #1084.

Additions

__ksym (kfunc) support

It's now possible to use new-style BPF helpers aka kfunc with the library. Going forward, all new BPF 'helper' functionality in the kernel will be exposed as kfuncs, and new helpers won't be added. See #966 and #996.

__kconfig support

Tracing programs often need to vary their behaviour based on kernel configuration, such as CONFIG_HZ. Such references to __kconfig variables are now automatically populated from a variety of sources such as /proc/config.gz. Note that if you run your application implementing ebpf-go in a container, it will need access to the host's /boot on some distributions that don't ship /proc/config(.gz). (notably, Debian-based distros)

As a special mention, the LINUX_HAS_SYSCALL_WRAPPER kconfig is also supported. This allows writing portable kprobes using the BPF_KSYSCALL macro from bpf_tracing.h.

See #951, #960 and #995.

Overwritable perf buffer support

The perf reader now allows creating "overwritable" perf buffers, which always contain the most recent events in case the buffer ever gets full. This is in contrast to regular perf buffers which drop recent events if there is no more space. This is useful to implement "flight recorder" type functionality for events sourced from BPF.

See #953.

btf.Builder

Constructing custom BTF type blobs is now possible through the new btf.Builder type. Call btf.NewBuilder() to obtain one, Builder.Add(t btf.Type) to add any types you need, followed by btf.NewHandle(b btf.Builder) to load the types into the kernel. Builder also has a Marshal() method for serializing the type collection into the canonical BTF format so it can be stored for later use.

What's Changed

• cs.RewriteConstants: define error MissingConstantsError by @alban in #904
• internal: add Memoize by @lmb in #909
• Fix ProgramInfo.MapIDs for programs without maps on old kernels by @lmb in #905
• link: make KprobeMultiOptions.Addresses a []uintptr by @lmb in #913
• btf: export API to create BTF from scratch by @lmb in #859
• Update CI dependencies by @lmb in #916
• map: replace hacky unsafe.Pointer conversion with PROT_NONE page by @lmb in #915
• link: use strings.Map in sanitizeSymbol by @lmb in #919
• btf: skip .data..percpu in TestRoundtripVMlinux by @lmb in #922
• link: close perfEventLink.fd before perf event in Close() by @ti-mo in #918
• map: ignore BPF_F_RDONLY_PROG for DevMap compatibility check by @aibor in #930
• Remove deprecated +build Go build tags by @ti-mo in #888
• link: use /sys/kernel/tracing if available by @spikat in #906
• collection: work around excessive copying of kernel BTF by @alban in #920
• sys: add fd leak tracing instrumentation by @ti-mo in #732
• program: set name from object info for pinned file by @aibor in #932
• Allow specifying the tracefs prefix in options by @brycekahle in #842
• Fixed broken links pointing to Cilium doc by @PhilipSchmid in #942
• link: use statfs for tracefs mount detection by @brycekahle in #944
• btf: ensure that TypesIterator is not affected by Spec.Add by @lmb in #937
• run-tests: allow using local kernel by @lmb in #952
• require Go 1.19 by @lmb in #950
• Support __kconfig macro for LINUX_KERNEL_VERSION by @eiffel-fl in #951
• btf: work around kernel Datasec bug by @lmb in #954
• link: always include requested symbol in kprobe creation errors by @ti-mo in #959
• Fix expected fs type for /sys/kernel/debug/tracing by @brycekahle in #958
• collection: ensure LINUX_KERNEL_VERSION variable is a btf.Int. by @eiffel-fl in #961
• add tests for FSType and getTracefsPath by @lmb in #962
• Link to maintainers file by @xmulligan in #957
• docs: update various bits of documentation by @lmb in #936
• fix some comments by @cuishuang in #972
• link: new link from fd by @Asphaltt in #971
• Add support for loong64 by @zhaixiaojuan in #975
• Allow debugfs fstype for /sys/kernel/debug/tracing by @brycekahle in #976
• Add error case for creating unspecified map type. by @tommyp1ckles in #974
• perf: Add support for overwritable buffer. by @eiffel-fl in #953
• update platformPrefix list with libbpf values by @paulcacheux in #982
• link: document AttachCgroup and flags by @ti-mo in #980
• perf: clean up tests by @lmb in #979
• btf: remove Spec.firstTypeID() by @lmb in #990
• elf_reader, linker: add kfunc support by @rgo3 in #966
• CI: enable gofmt linter by @lmb in #992
• link: use BPF links to attach Tracing and LSM prog types by @mmat11 in #837
• bpf2go: spring cleaning by @lmb in #910
• btf: fix off-by-one in Spec.TypeByID by @lmb in #993
• link/executable: lazy load symbol table by @mmat11 in #991
• btf: remove type alias for []Type by @lmb in #989
• sys: introduce TypeID by @lmb in #912
• elf: freeze .kconfig map by @lmb in #1000
• btf: clean up handling of split BTF in loadRawSpec and inflateRawTypes by @lmb in #999
• elf: use per-instruction metadata for .kconfig references by @lmb in #994
• btf: avoid repeated kernel BTF copies in Handle.Spec by @lmb in #1001
• docs: document API stability by @lmb in #1003
• bpf2go: Fall back to default module name when debug.ReadBuildInfo is not available by @folbricht in #1004
• update github.com/frankban/quicktest by @lmb in #1010
• asm: update eBPF built-in functions by @florianl in #1011
• ProgramInfo provides CreatedByUid by @zachcheu in #1006
• btf: optimize Spec.Copy by @lmb in #1002
• Add support for LINUX_HAS_SYSCALL_WRAPPER kconfig by @paulcacheux in #995
• program: improve error handling in haveSyscallWrapper by @lmb in #1022
• fix container-all make invocation by @paulcacheux in #1021
• make: allow interrupting container-all with ctrl-c by @lmb in #1024
• refactor tracefs event handling into an object by @lmb in #1023
• Print out the relocation name in the error message by @kuroa-me in #1026
• allow skipping kernel version check for feature tests if test skipped via env variable by @abhipranay in #1025
• replace deprecated rand.Seed calls by @kwakubiney in #978
• elf: support kernel ...

v0.10.0 - BTF Marshaling

We're pleased to announce the first ebpf-go release of 2023! There is one breaking change, so please read through the release notes carefully. All users are encouraged to upgrade.

Breaking Changes

Support for BTF marshaling

So far, the library would only load BTF into the kernel that was originally generated by clang or pahole. As of this release, the library will instead marshal btf.Types directly into BTF blobs. For example, it's now possible to create an ebpf.Map from Go code by populating the MapSpec.Key and .Value fields with the map's k/v types to make bpftool map dump nicely pretty-print its contents for you.

As expected, there are a few important side effects and breaking changes to be aware of:

• Setting MapSpec.BTF and ProgramSpec.BTF to nil can no longer be used to disable BTF during loading, which was the fields' last remaining purpose. The fields have been removed to make sure this doesn't slip through library upgrades unnoticed.
• If your userspace program replaces individual instructions in ProgramSpec.Instructions before loading a program, make sure to copy the old Instruction's Metadata using Instruction.WithMetadata(old.Metadata). Not doing so may cause you to discard the original instruction's func_info or line_info, leading to the verifier rejecting your program with e.g. invalid argument: missing bpf_line_info for func#0. (see cilium/cilium#21933 for how this can go wrong)
• Disabling BTF for program loads now requires calling btf.WithFuncMetadata(ins, nil) on the first instruction of each (sub)program, as well as Instruction.WithSource(nil) on each Instruction containing lineinfo.
• Disabling BTF for map loads now requires MapSpec.Key and .Value being set to nil.

We're interested in hearing your use cases for explicitly disabling BTF during map/prog loads. We hope the need for disabling BTF altogether should arise less frequently, given the library falls back gracefully when the underlying kernel doesn't support BTF. In case you do believe this to be necessary, please raise an issue to discuss this further.

Fixes

NewProgram returns an unwrapped VerifierError

Since commit 148c76c ("internal: make VerifierError more succinct") the library has defaulted to omitting most of the verifier log when loading a program failed. The intention was that callers would use formatting with the %+v verb to output as much context as necessary. Due to how error wrapping with fmt.Errorf works this meant that the error had to be unwrapped with errors.As, which led to confusion.

NewProgram and friends now return an unwrapped VerifierError so that callers can format the error more easily.

Reliable VerifierError.Truncate field

Prior to this release, the VerifierError.Truncate field would only be set in case ProgramOptions.LogLevel was left to the default value of 0, among a few other quirks. This has now been resolved, making the VerifierError.Truncate field a reliable driver for a retry loop that automatically grows the verifier log buffer and retries loading the program(s). See cilium/cilium@934bccf for an example implementation.

Additions

Expanded and overhauled feature probes

features.HaveProgramType() can now conclusively probe for the program types ebpf.LSM, ebpf.Tracing and ebpf.Extension without relying on recognizing specific error return values.

Package features has been refactored to use internal.FeatureTest, unifying error wrapping and result caching with the library's internal machinery. ErrNotSupported returned from features now includes the minimum required kernel version and a feature name.

Kretprobes allow setting maxactive

The kernel's kretprobe implementation has a limitation where only a fixed number of concurrent calls to a probed function are handled. Usually the kernel chooses a sufficient default value, but for very busy functions this default is too low. This leads to missed kretprobe events.

There is an (unfortunate) workaround for this: the user can specify how many concurrent calls they want to support via a maxactive parameter. This comes with a lot of drawbacks however. maxactive is only supported when using an obsolete interface to kretprobes, and it's not at all clear how to arrive at the correct maxactive setting. link.KprobeOptions now exposes this setting to the user, since it is the only partial fix.

The authors of the library recommend to not use maxactive unless absolutely necessary. Incorrect use will make your application more brittle and may have system-wide performance impact.

What's Changed

• map, prog: fix broken links by @boratanrikulu in #816
• program: clarify how to use VerifierError by @lmb in #819
• link: Allow kprobe multi to be disabled in kernel by @arthurfabre in #812
• Add IsPinned() to RawLink by @boratanrikulu in #817
• fix bad link to GitHub Discussions by @dmitris in #824
• internal: detect if /proc/self/auxv is not readable due to file caps by @lmb in #825
• btf: add support for marshaling Type and use it for Program and Map by @lmb in #796
• btf: support enum64 by @willfindlay in #820
• Add stringer to ebpf-builder by @lmb in #827
• asm: add .WithMetadata() for conveniently replacing individual Instructions by @ti-mo in #832
• prog: populate VerifierError.Truncate when LogLevel > 0 by @ti-mo in #834
• elf_reader: check if ELF is for BPF data by @florianl in #830
• map_test: clean up after tests and close maps by @florianl in #841
• btf: check for compatibility first when searching for a CO-RE field by @lmb in #852
• Set Program.name when constructing from file descriptor by @aibor in #849
• btf: do Datasec fixup on inflated types by @lmb in #860
• btf: distinguish 'map' and 'program' BTF by @ti-mo in #855
• program: return unwrapped VerifierError by @lmb in #851
• map: export MapSpec.Compatible by @olsajiri in #858
• btf: Remove deprecated {Map,Program}Spec.BTF field by @ti-mo in #864
• btf: fix some split BTF shortcomings by @lmb in #861
• Update dependencies for current Go versions by @thaJeztah in #866
• map: include generated pin path in newMapWithOptions error by @ti-mo in #870
• features: reuse internal.FeatureTest instead of open coding probes by @lmb in #776
• feat: support LSM prog type by @daemon1024 in #885
• bpf2go: write dependencies to temporary file to support Windows by @junjiexing in #865
• program: block SIGPROF during BPF_PROG_RUN by @lmb in #887
• btf: fix function doc typo by @rgo3 in #889
• link: add QueryPrograms API by @rgo3 in #867
• link: add maxactive for kretprobe by @alahaiyo in #755
• Add missing program type feature probes by @rgo3 in #890

New Contributors

• @boratanrikulu made their first contribution in #816
• @willfindlay made their first contribution in #820
• @aibor made their first contribution in #849
• @daemon1024 made their first contribution in #885
• @junjiexing made their first contribution in #865
• @alahaiyo made their first contribution in #755

Full Changelog: v0.9.3...v0.10.0

v0.9.3 - Prevent livelocks loading BPF programs while profiling

This is a bugfix release for an endless loop that could occur when running a pprof session while loading a BPF program.

@danobi published an article with a deep-dive into the problem: https://dxuuu.xyz/bpf-go-pprof.html.

If a thread receives a signal while blocked in BPF_PROG_LOAD, the verifier can cooperatively interrupt itself by checking pending signals for the thread and return -EAGAIN from the syscall to request userspace to retry.

During a Go pprof session, threads are routinely sent a SIGPROF to make them dump profiling information, which can lead to a runaway reaction if the program takes longer to verify than the interrupt frequency. To prevent this, the SIGPROF signal is now masked during BPF_PROG_LOAD.

What's Changed

• CI: test on 5.19 by @lmb in #793
• prep work for BTF marshaling by @lmb in #791
• cmd/bpf2go: test against clang-14 by default by @lmb in #794
• btf: fix IntEncoding by @lmb in #797
• bpf2go: use [16]byte instead of uint128 by @Benjamin-Yim in #799
• Semaphore: remove manual Go installation, expedite 'cache restore' by @ti-mo in #803
• Prevent pprof from causing BPF verifier livelocks by @ti-mo in #805
• bpf2go: add flag for alternative filename stem by @MarcusWichelmann in #770
• README: update to reflect the project's situation in H2 2022 by @ti-mo in #804
• sys: use SIG_BLOCK and SIG_UNBLOCK from x/sys/unix by @lmb in #807

New Contributors

• @Benjamin-Yim made their first contribution in #799
• @MarcusWichelmann made their first contribution in #770

Full Changelog: v0.9.2...v0.9.3

v0.9.2

This release contains an important bugfix for users of Program.Test() and .Benchmark(). A kernel change was made that disallows empty packet buffers, with knock-on effects to BPF_PROG_RUN and, as a result, ebpf-go's detection routine for said feature. Users are strongly encouraged to upgrade. See #788 for more details.

The new features.HaveMapFlag() API was merged in this release, congrats @paulcacheux!

Likewise, the link.K(ret)probeMulti() API was added by @mmat11, bringing blazingly-fast bulk kprobe attachments to kernels 5.18 and newer.

Some improvements were made to verifier log handling. The VerifierError.Truncated flag can now reliably be used to determine if a program load should be retried due to the provided buffer size being too small. Use this to retry with incrementally growing log buffers for large or complex programs. The Log* fields in ProgramOptions are now thoroughly documented to this effect.

It also bumps the minimum Go version to 1.18, since 1.19 is now out.

What's Changed

• run-tests: show curl error messages by @lmb in #738
• elf_reader: only read data from PROGBITS sections, ignore NOBITS by @ti-mo in #740
• program: support tracing of kernel modules by @lmb in #737
• CI: Test against kernel 5.18 by @ti-mo in #668
• internal: fix VerifierError output with empty log by @lmb in #743
• elf_reader: Allow strings read-only global data sections by @dylandreimerink in #742
• CI: expose individual test results by @lmb in #748
• Fix arm64 testsuite failures by @lmb in #745
• link: (u|k)probe: don't treat EINVAL as os.ErrNotExist by @ti-mo in #751
• features: remove EPERM wrapping exception, automatically wrap errors by @ti-mo in #749
• features: implement probes for StructOps maps and programs by @ti-mo in #746
• perf add timeout argument by @xpu22 in #736
• epoll deadline fixups by @lmb in #758
• prog: unconditionally enable verifier retries, add consts, document LogLevel by @ti-mo in #754
• program: fix comment typo by @wozniakjan in #764
• bpf2go: replace deprecated io/ioutil function by @florianl in #762
• btf: minimal support for BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG by @lmb in #747
• features: add HaveMapFlag API by @paulcacheux in #673
• Build, test and format with Go 1.19, minimum 1.18 by @ti-mo in #765
• errors: remove verifier truncation heuristic, rely on ENOSPC by @ti-mo in #756
• examples: clarify how to rebuild by @lmb in #767
• feature: define sys.MapFlags and re-export it for map flags feature detection by @paulcacheux in #766
• cmd/bpf2go: clarify -target usage with bpf_tracing.h by @lmb in #773
• examples: add sockops example by @BigMikes in #771
• features: fix incorrect lock in probeMisc by @lmb in #775
• link: memoize PMU event types read from sysfs by @lmb in #774
• btf: handle signed enums and fix default signedness by @lmb in #781
• iter: Add support to load sleepable iterator program type by @olsajiri in #777
• Correct misleading unpin error in progAttachCgroup by @weiqiangt in #787
• Fix haveProgTestRun on kernel version 5.15.65 by @player-two in #788
• Provide 15-byte context inputs to Program.{Run,Benchmark,Test} by @ti-mo in #792
• link: implement kprobe.multi link by @mmat11 in #716

New Contributors

• @wozniakjan made their first contribution in #764
• @weiqiangt made their first contribution in #787
• @player-two made their first contribution in #788

Full Changelog: v0.9.1...v0.9.2

v0.9.1

This is mainly a bugfix release, but a few long-deprecated pieces of API were finally removed after cutting v0.9.0.

These should be trivial to fix up, since they were simply moved or replaced by similar interfaces.

Enjoy!

Breaking Changes

• Deprecations:
  • ebpf.ErrNotExist was removed, as it was an alias to os.ErrNotExist
  • Map.ID() in favor of Map.Info().ID()
  • Program.ID() in favor of Program.Info().ID()
  • Program.Attach() and .Detach() in favor of link.RawAttachProgram() and link.RawDetachProgram() respectively
  • link.LoadPinned{Cgroup,Iter,RawLink,NetNs,Freplace} in favor of a single link.LoadPinnedLink()
  • link.RawLinkInfo was removed, as it was an alias to link.Info
• UprobeOptions.Offset is now a relative field, with the new .Address being an absolute offset.

What's Changed

• Remove API deprecated in v0.8.0 by @lmb in #684
• program: Add TestWithOptions() API by @danobi in #647
• prog: Avoid allocating for ctx if ctx not provided by @danobi in #687
• btf: add Array.Index by @lmb in #689
• btf: add btfInt by @lmb in #690
• Add RelativeOffset field to UprobeOptions struct by @mertyildiran in #682
• program: remove references from ProgramSpec by @lmb in #679
• map: suggest incorrect MaxEntries on EINVAL by @markpash in #697
• internal/sys: regenerate types by @florianl in #700
• link: close tracefs event if perf event cannot be opened by @nashuiliang in #699
• Revert ".semaphore - workaround to update ca-certificates before apt-get update" by @lmb in #703
• run-tests: fix shebang to find bash more reliably by @markpash in #702
• examples: add map in map example by @markpash in #696
• btf: fix NewHandleFromID and add test by @lmb in #704
• sycall should be syscall. by @shepf in #707
• CI: load bpf_testmod in VM by @lmb in #709
• internal: make VerifierError more succinct by @lmb in #688
• examples: add example for XDP by @wedaly in #706
• btf: simplify loadRawSpec by @lmb in #708
• Remove 4.4 from CI by @lmb in #710
• map: Do not chec maxEntries for PerfEventArray map by @olsajiri in #714
• link/kprobe: skip offset test on kernel version > 5.15 by @mmat11 in #719
• Avoid leaking fds in test suite and NewMapWithOptions() by @ti-mo in #725
• btf: Add support for loading split BTF by @brb in #691
• prog: Allow empty data for prog.Test and prog.Run by @danobi in #685
• internal: avoid sign extension of Statfs_t.Type on 32 bit arches by @lmb in #727
• link: unify KprobeOptions.Offset and UprobeOptions.Offset by @lmb in #694
• collection: close all unassigned resources before returning from LoadAndAssign by @ti-mo in #724
• btf: add Enum.Size by @lmb in #735
• btf: use ci-kernels to source kernel BTF by @lmb in #731
• asm: update BPF built-in functions by @rgo3 in #734
• program: add getter for btf.Handle by @lmb in #730
• btf: fix NewHandleFromID for kernel modules by @lmb in #729
• internal: include previous line on BPF context access error by @lmb in #733

New Contributors

• @danobi made their first contribution in #647
• @mertyildiran made their first contribution in #682
• @nashuiliang made their first contribution in #699
• @shepf made their first contribution in #707
• @wedaly made their first contribution in #706
• @olsajiri made their first contribution in #714

Full Changelog: v0.9.0...v0.9.1

Releasing the BTF package!

This release makes package btf public, allowing type information embedded in eBPF ELF objects and vmlinux to be read programmatically from Go. This enables use cases like searching and inspecting the kernel's types and function signatures at runtime, or ensuring alignment between structures defined in Go and C. Package btf stands on its own and does not require adopting other subpackages, potentially making it useful beyond the domain of eBPF. Additionally, parsing BTF was made significantly faster and now consumes less memory.

Note: Modifications to btf.Types are not (yet) reflected in the BTF info loaded into the kernel. This will be implemented in the near future, and will provide the flexibility of, for example, creating maps with arbitrarily crafted key/value type information for pretty-printing map dumps, among many other exciting use cases.

Users of the ringbuf and perf packages can now avoid allocations when reading samples with the addition of ReadInto().

Feature probes for program helpers (features.HaveProgramHelper()) were added, as well as a few miscellaneous probes for large instruction limit, bounded loops and the supported eBPF ISA. This now brings the library mostly on par with bpftool's probing capabilities, except for a few program types that require BTF.

There was also one important bugfix where CO-RE relocations in bpf2bpf subprograms were not applied. Users are strongly encouraged to upgrade.

Enjoy!

Breaking changes

• Deprecations:
  • CollectionSpec.RewriteMaps is deprecated in favour of CollectionOptions.MapReplacements
  • features.HaveProgType is deprecated in favour of features.HaveProgramType
• CollectionSpec.BTF is now called CollectionSpec.Types
• link.K(ret)probe and link.Tracepoint signatures have changed, they now take an additional options parameter. Pass nil to retain the old behaviour.

Features

• features: add misc probes by @florianl in #541
• bpf2go: generate Go types from BTF by @lmb in #552
• features: add LinuxVersionCode function to obtain LINUX_VERSION_CODE by @ti-mo in #585
• link: support bpf perf event link, add bpf_cookies by @mmat11 in #565
• link/kprobe: Allow . in symbol name by @chenhengqi in #600
• btf: Implement support for bitfield relocations by @joamaki in #573
• kprobe: fall back to tracefs if kernel doesn't support dots in PMU symbol by @chenhengqi in #605
• link/kprobe: specify symbol offset by @xpu22 in #618
• collection: Add ReplaceMaps to CollectionOptions by @mauriciovasquezbernal in #646
• fix: support bpf_core_type_exists() by @holyspectral in #655
• perf, ringbuf: add ReadBuffer mehods by @lmb in #663
• Make BTF for data sections optional by @ti-mo in #675
• btf: Add spec types iterator by @brb in #678
• features: add HaveProgramHelper API by @rgo3 in #375
• Probe kernel bpf2bpf support by @ti-mo in #657

Bug fixes

• elf_reader: Fix cgroup/sock_{create,release} program type by @joamaki in #580
• Execute and test CO-RE relocations of bpf2bpf subprogs by @ti-mo in #590
• syscalls: omit AttachType field in haveProgAttach by @ti-mo in #652
• prog: recognize ENOTSUPP in testRun() by @ti-mo in #653
• Assorted fixes for kernel 4.14 by @ti-mo in #658

Miscellaneous

• collection: Rename CollectionSpec.BTF to .Types, copy in Copy() by @ti-mo in #581
• collection: document garbage collection behaviour of *Map and *Program by @ti-mo in #579
• prog_test: compare BTF program tags instead of raw insns by @ti-mo in #583
• collection: don't copy btf.Spec in CollectionSpec.Copy() by @ti-mo in #584
• Moved Instruction.Symbol and Instruction.Reference into per-instruction metadata by @dylandreimerink in #582
• examples: generate types via bpf2go by @lmb in #575
• Add Maps to Instruction metadata by @dylandreimerink in #567
• CI: use Go 1.18 by @lmb in #599
• examples: include bpf_helpers.h from common.h by @ti-mo in #614
• examples: add example for CO-RE kernel struct (tcp_sock) by @BigMikes in #602
• examples: fentry - add comments to illustrate difference with tcprtt by @ti-mo in #615
• btf: reduce unnecessary copies when parsing BTF by @lmb in #623
• asm: add Instruction Source metadata by @lmb in #609
• Memory improvements to loadRawSpec by @paulcacheux in #634
• btf: split string table ahead of time by @lmb in #637
• sys: update types by @florianl in #630
• btf: fix up inflated types immediately if possible by @lmb in #638
• Go 1.17 by @lmb in #640
• btf: stop relying on TypeID when printing Types by @lmb in #633
• link: do not use regexp pkg by @kolyshkin in #642
• btf: make copying types infallible by @lmb in #631
• Deprecate CollectionSpec.RewriteMaps() by @ti-mo in #651
• CI: retry two times on error 42 by @lmb in #654
• Use instruction metadata to simplify linking and CO-RE relocation by @lmb in #606
• btf: allow users to obtain COREFixups by @lmb in #661
• CI: run test suite against LTS kernel 4.14 by @ti-mo in #670
• btf: introduce Bits type and remove Int.Bits and Int.Offset by @lmb in #660
• ringbuf: reduce allocations and syscalls by @lmb in #667
• Export package btf by @lmb in #665
• btf: don't cache kernel BTF in LoadKernelSpec by @lmb in #676

New Contributors

• @ningmingxiao made their first contribution in #568
• @dylandreimerink made their first contribution in #582
• @xmulligan made their first contribution in #589
• @BigMikes made their first contribution in #602
• @gujun4990 made their first contribution in #620
• @xpu22 made their first contribution in #618
• @kolyshkin made their first contribution in #642
• @mauriciovasquezbernal made their first contribution in #646
• @holyspectral made their first contribution in #655
• @brb made their first contribution in #678

Full Changelog: v0.8.1...v0.9.0

v0.8.1

Fixes

• elf_reader: Fix non-determinism in determining prog type by @joamaki in #549
• Look for Linux note in all elf note sections instead of just the first one by @paulcacheux in #560
• btf: remove symbol value bounds check to support vmlinux ELFs by @ti-mo in #555

New Contributors

• @joamaki made their first contribution in #549
• @paulcacheux made their first contribution in #560

Full Changelog: v0.8.0...v0.8.1

v0.8.0

We're happy to announce our first release of 2022!

Breaking Changes

• Deprecations
  • Deprecate StructOpts and perf.IsClosed() by @ti-mo in #547

• cmd/bpf2go: strip DWARF info by @lmb in #481
  DWARF info is now stripped from compiled BPF objects by default when using bpf2go, resulting in significant savings in object size. It can be re-enabled if desired.

• elf_reader: emit UnspecifiedPrograms into CollectionSpec by @ti-mo in #529
  Previously, ProgramSpecs were not emitted to CollectionSpec.Programs if their ELF section names were not recognized.
  This limitation has been lifted: programs that are explicitly assigned to an ELF section will now always appear in the CollectionSpec, though they will all be of type UnspecifiedProgram. The SectionName field was added to help the user determine program types manually.

• map: make Map.Extra field a pointer to a bytes.Reader by @ti-mo in #546
  A last-minute addition, no longer allocate an empty bytes.Reader for every Map, as this is generally only used when loading legacy programs with iproute2 map definitions. If you're using this field, check for nil before accessing.

Features

• automatically generate syscall wrappers by @lmb in #425
• asm: update eBPF built-in functions by @florianl in #459
• program: add program type syscall by @florianl in #458
• Add v5.14 to CI builds by @lmb in #462
• Improve kernel 4.4.x compatibility by @nplanel in #436
• Add BTFIDLink to support fentry/fexit/fmod_ret/tp_raw/lsm BPF programs by @mehrdadrad in #460
• link/uprobe: support ref_ctr_offset by @mmat11 in #463
• Allows user space to update/lookup map elements under bpf_spin_lock by @mehrdadrad in #473
• Makefile: prefer podman for container engine by @syscll in #480
• asm: add Instructions.SizeBytes() for convenience and readability by @ti-mo in #492
• Link: add XDP support by @mehrdadrad in #484
• link: add LoadPinnedLink by @lmb in #491
• implement BPF_PROG_BIND_MAP by @florianl in #485
• use clang 13 by @lmb in #488
• Read LINUX_VERSION_CODE from vDSO ELF header instead of uname by @brycekahle in #500
• cmd/bpf2go: infer llvm-strip suffix from clang binary by @lmb in #504
• Add support for function pointers by @pippolo84 in #499
• prog: update program types from libbpf by @mehrdadrad in #496
• btf: rename FindType to TypeByName, add TypesByName and TypeByID methods by @ti-mo in #503
• btf: support raw BTF in LoadSpecFromReader by @lmb in #520
• cmd/bpf2go: allow to specify native target by @tklauser in #526
• asm: Add Jump32 support by @arthurfabre in #514
• link: expose link info with type-specific information by @mehrdadrad in #509
• Adopt Go 1.18 fuzzing and fix btf crashes by @lmb in #530
• elf_reader,btf: support multiple programs per ELF section by @ti-mo in #508
• Eliminate binary.Read in instruction (un)marshaler by @ti-mo in #532
• program: retrieve xlated Instructions from Program by @ti-mo in #538
• Support attaching SocketFilter programs to raw sockets by @folbricht in #540

Bug Fixes

• CI: add make clean before testdata rebuild by @ti-mo in #450
• sys: avoid zero fd by @lmb in #454
• run-tests: don't require CI_MAX_KERNEL_VERSION to be set by @lmb in #470
• features: fix HaveProgType(ebpf.Syscall) by @rgo3 in #471
• elf: fix some typos in getProgType by @alxn in #493
• CI: work around hung agents by @lmb in #516
• link: switch tests from printk to vprintk symbol by @ti-mo in #518
• map: allow empty ProgramArray to not be assigned by @lmb in #513
• prog: don't panic when prog load EPERMs with empty verifier log by @ti-mo in #519
• map, prog: fix typo in MEMLOCK error string by @tklauser in #527
• btf: only return exact matches from AnyTypesByName by @lmb in #525
• Remove BTF type IDs on map creation if not supported by @chenhengqi in #534
• rlimit: avoid lowering Max rlimit by @ti-mo in #533
• linker: Allow calling into unimplemented forward function declarations by @ti-mo in #544

Documentation

• cmd/bpf2go: move doc to README by @lmb in #446

Examples

• reproducible builds for examples by @lmb in #417
• examples: added cgroup_skb example by @mingleiw in #465
• Add fentry example by @mehrdadrad in #482
• examples: add $BPF_STRIP to bpf2go invocation by @lmb in #498
• tidy up examples by @lmb in #521

New Contributors

During this release cycle, we've welcomed the following new contributors:

• @nplanel made their first contribution in #436
• @mingleiw made their first contribution in #465
• @syscll made their first contribution in #478
• @alxn made their first contribution in #493
• @chenhengqi made their first contribution in #534

Thanks to all contributors for your support!

Full Changelog: v0.7.0...v0.8.0

v0.7.0: Honeygopher Edition

The cilium/ebpf project now has a mascot, the Honeygopher! Thank you, Vadim Shchekoldin!

New Packages

• Package ringbuf allows reading from a BPF_MAP_TYPE_RINGBUF map.
• Package features implements the equivalent of bpftool feature probe for discovering BPF-related kernel features using native Go. Note that support for probing availability of BPF helpers is pending.
• Package rlimit provides a convenient API to lift the RLIMIT_MEMLOCK constraint on kernels before 5.11.

Breaking Changes

• Deprecations
  • StructOpts has been changed to alias the new StructOpsMap and will be removed in 0.8.0
    by @ti-mo in #381
  • perf.IsClosed() has been deprecated and will be removed in 0.8.0
    by @lmb in #397
  • Collection.Assign() has been removed after a sunset period
    by @ti-mo in #371

Features

• elf: support cgroup getsockname/getpeername prog by @zf1575192187 in #350
• MapType probe API by @rgo3 in #321
• link/uprobe: support filtering by pid by @mmat11 in #342
• Add support for freplace programs by @zeffron in #288
• Export CollectionSpec.ByteOrder by @ti-mo in #361
• Support ELF map definitions larger than bpf_map_def by @ti-mo in #362
• Auto-populate tail-call and map-in-map maps from ELF/BTF by @ti-mo in #363
• features: add HaveProgType API by @rgo3 in #355
• feature: add call fixups for bpf_probe_read_kernel, etc by @xh4n3 in #367
• cmd/bpf2go: use Go 1.16 embed to include BPF elfs by @lmb in #327
• bpf2go: add support for bpf_tracing.h by @lmb in #369
• features: support get map ids from program. by @arikachen in #391
• Recognize sk_reuseport programs by @lmb in #393
• btf: handle float relocation for CO-RE by @lmb in #403
• Utility function to remove memlock rlimit by @folbricht in #392
• BPF ring buffer by @mythi in #318

Bug Fixes

• link/uprobe: handle ENOTSUPP from the kernel's prepare_uprobe() by @ti-mo in #351
• Mark PerCPUCGroupStorage as per-cpu by @yadutaf in #341
• link/kprobe: don't redeclare a new variable sp by @guodongzz in #365
• Fix invalid usage of reflect.SliceHeader by @cuonglm in #376
• program: don't retrieve map ids by default by @lmb in #395
• link/uprobe: fix offsets for statically linked binaries by @mmat11 in #385
• Add sync.Pool to reduce allocations for the default case in unmarshalBytes() by @rockdaboot in #420
• Fix btf.FindType to avoid copy by @pippolo84 in #424
• Fix missing close of fds from BPF{ProgLoad,MapCreate} by @pippolo84 in #428
• Directly handle int32 and int64 types in unmarshalBytes() by @rockdaboot in #427

Examples

• Add kprobepin example showing how to re-use a pinned map in a BPF prog by @ungureanuvladvictor in #364
• examples: adapt uretprobe example to the new library version by @mmat11 in #389
• CI: build examples by @lmb in #390
• Add a script to update vendored libbpf headers for examples by @markpash in #394
• Add an example for the RINGBUF map type by @pippolo84 in #412
• Add a tracepoint example and a percpu example by @hao-lee in #402
• examples - convert to new rlimit package for memlock removal by @ti-mo in #441

New Contributors

• @zf1575192187 made their first contribution in #350
• @sharjeelaziz made their first contribution in #349
• @zeffron made their first contribution in #352
• @yadutaf made their first contribution in #341
• @guodongzz made their first contribution in #365
• @xh4n3 made their first contribution in #367
• @cuonglm made their first contribution in #376
• @mabis made their first contribution in #378
• @arikachen made their first contribution in #391
• @markpash made their first contribution in #394
• @pippolo84 made their first contribution in #401
• @folbricht made their first contribution in #392
• @hao-lee made their first contribution in #414
• @rockdaboot made their first contribution in #420
• @mehrdadrad made their first contribution in #439
• @lizrice made their first contribution in #442

This release saw a record amount of new contributors! We'd like to wish everyone a warm welcome and thank you all for your support!

Full Changelog: v0.6.2...v0.7.0

Wrap SYS_BPF errors, user-provided uprobe offsets, support new map, program and flag types

Breaking changes

• None

Features

• Return friendly error when creating maps with BPF_F_MMAPABLE or BPF_F_INNER_MAP and the kernel doesn't support them.
• Add support for StructOps, RingBuf, InodeStorage and TaskStorage map types.
• Support many new BPF program types added over the course of 2019/2020 in the assembler.
• Detect fentry, fmod_ret and fexit programs by their ELF section names.
• link: support user-provided offsets for uprobe locations by specifying UprobeOptions.
• link: support creating bpf_map_elem iterators
• Wrap and propagate all errors generated by SYS_BPF, allowing the caller to retrieve

Bug Fixes

• Fix flaky TestMapPin test on Go 1.17beta1

Deprecations

• ebpf.ErrNotExist is now an alias to os.ErrNotExist and will be removed in a future release.

Examples

• Docstring fixup in kprobe example

Contributors

Chris Tarazi
Lorenz Bauer
Mattia Meleleo
Mikko Ylinen
Robin Gögge
Tobias Klauser

Thank you for your work!