Add support to monitor bpf programs.#88
Open
liu-song-6 wants to merge 1 commit intoAtoptool:masterfrom
Open
Conversation
Author
|
|
Contributor
|
@liu-song-6 not directly related, but I wonder if it would be possible to implement the netatop module using eBPF? (and no longer require a custom kernel module to make use of that atop functionality) |
Author
|
Changes v1 => v2:
For #1, we need Linux kernel 5.8-rc1 or newer, and libbpf-0.0.9 or newer. |
danobi
reviewed
Jul 9, 2020
photobpf.c
Outdated
Comment on lines
104
to
108
| err = bpf_obj_get_info_by_fd(fd, &info, &len); | ||
| if (err) { | ||
| close(fd); | ||
| break; | ||
| } |
There was a problem hiding this comment.
Aren't you leaking fd if it doesn't go on the error path?
BPF is very important component for modern Linux systems, and getting more
features and adoptions. This commit enables atop to monitor BPF programs.
The output looks like:
ATOP - kerneltest002 2020/06/16 17:01:12 -------------- 10s elapsed
PRC | sys 2.72s | user 4.85s | #proc 761 | #zombie 0 | #exit 250 |
CPU | sys 29% | user 50% | irq 0% | idle 7915% | wait 8% |
CPL | avg1 1.68 | avg5 1.05 | avg15 0.72 | csw 160979 | intr 66341 |
[...]
BPF_PROG_ID NAME TOTAL_TIME_NS RUN_CNT CPU AVG_TIME_NS
894 tracepoint__sch 83882 11 0% 7625.64
893 tracepoint__sch 43231 5 0% 8646.20
892 tracepoint__tas 34818 4 0% 8704.50
PID SYSCPU USRCPU VGROW RGROW RDDSK WRDSK EXC THR S CPUNR CPU CMD 1/113
2669644 0.45s 1.08s 603.1M 23100K 0K 0K - 10 S 59 15% squashfuse_ll
To build atop with BPF monitoring, we need pass in option to make as:
ATOP_BPF_SUPPORT=1 make -j
Atop periodically enables monitoring of BPF programs calling:
bpf_enable_stats(BPF_STATS_RUN_TIME);
Since monitoring of BPF program has non-trivial overhead to the bpf
programs, the following options are added to only monitor BPF program
less often:
bpfsamplerate, default 1
bpfsampleinterval, default 1
bpf stats is enabled for bpfsampleinterval seconds every bpfsamplerate
atop intervals. bpfsampleinterval must be smaller than atop interval.
Changes v1 => v2:
1. Instead of using unsafe sysctl, using a safe new API to enable BPF
runtime stats.
2. Change output columns: remove "TYPE", add "CPU" for cpu %.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BPF is very important component for modern Linux system, and getting more
features and adoptions. This commit enables atop to monitor BPF programs.
The output looks like:
PRC | sys 9h36m | user 11h45m | #proc 759 | #tslpu 0 | #zombie 0 | #exit 5 |
CPU | sys 14% | user 18% | irq 0% | idle 7967% | wait 1% | ipc initial |
CPL | avg1 0.26 | avg5 0.44 | avg15 0.48 | csw 113066e5 | intr 61268e5 | numcpu 80 |
NET | lo ---- | pcki 18145e4 | pcko 18145e4 | sp 0 Mbps | si 2365 Kbps | so 2365 Kbps |
BPF_PROG_ID TYPE NAME TOTAL_TIME_NS RUN_CNT AVG_TIME_NS
39 sched_cls fbflow_egress 475443 235 2023.16
175 tracepoint tracepoint__sch 89347 10 8934.70
40 sched_cls fbflow_ingress 53494 227 235.66
PID SYSCPU USRCPU VGROW RGROW RDDSK WRDSK ST EXC THR S CPUNR CPU CMD 1/382
2377 81m34s 2h17m 2.2G 176.1M 319.1M 31.4G N- - 270 S 11 1% configerator_p
To build atop with BPF monitoring, we need pass in option to make as:
Atop periodically enables monitoring of BPF programs by writing to
/proc/sys/kernel/bpf_stats_enabled
This part is not 100% multi-process safe.
Since monitoring of BPF program has non-trivial overhead to the bpf
programs, the following options are added to only monitor BPF program
less often:
bpf stats is enabled for bpfsampleinterval seconds every bpfsamplerate
atop intervals. bpfsampleinterval must be smaller than atop interval.