Changes in task-analyzer-record

7ae9fb1b7ecbb5d85d07857943f677fd1a559b18 - Merge branch 'next' into for-linus

Tue, 21 Feb 2023 19:19:49 +0000

Merge branch 'next' into for-linusPrepare input updates for 6.3 merge window. List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

6f84981772535e670e4e2df051a672af229b6694 - Merge drm/drm-next into drm-misc-next

Thu, 19 Jan 2023 09:56:12 +0000

Merge drm/drm-next into drm-misc-nextBackmerging into drm-misc-next to get DRM accelerator infrastructure,which is required by ipuv driver.Signed-off-by: Thomas Zimmermann List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

d0e99511834b6828c960e978d9a8cb6e5731250d - Merge wireless into wireless-next

Tue, 17 Jan 2023 11:36:25 +0000

Merge wireless into wireless-nextDue to the two cherry picked commits from wireless to wireless-next we haveseveral conflicts in mt76. To avoid any bugs with conflicts merge wireless intowireless-next.96f134dc1964 wifi: mt76: handle possible mt76_rx_token_consume failuresfe13dad8992b wifi: mt76: dma: do not increment queue head if mt76_dma_add_buf fails List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

407da561244b9d51e6a794d6305ba38ec2c9d907 - Merge tag 'v6.2-rc3' into next

Tue, 10 Jan 2023 04:28:19 +0000

Merge tag 'v6.2-rc3' into nextMerge with mainline to bring in timer_shutdown_sync() API. List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

2c55d703391acf7e9101da596d0c15ee03b318a3 - Merge drm/drm-fixes into drm-misc-fixes

Tue, 03 Jan 2023 07:32:12 +0000

Merge drm/drm-fixes into drm-misc-fixesLet's start the fixes cycle.Signed-off-by: Maxime Ripard List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

0d8eae7b124e2ddaee00f186fe922450faad0ed7 - Merge drm/drm-next into drm-intel-next

Mon, 02 Jan 2023 09:31:03 +0000

Merge drm/drm-next into drm-intel-nextSync up with v6.2-rc1.Signed-off-by: Jani Nikula List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

b501d4dc83aa3940189b68045cadc8b3eac73988 - Merge drm/drm-next into drm-intel-gt-next

Fri, 30 Dec 2022 09:09:09 +0000

Merge drm/drm-next into drm-intel-gt-nextSync after v6.2-rc1 landed in drm-next.We need to get some dependencies in place before we can mergethe fixes series from Gwan-gyeong and Chris.References: https://lore.kernel.org/all/Y6x5JCDnh2rvh4lA@intel.com/Signed-off-by: Rodrigo Vivi List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

6599e683db1bf22fee74302c47e31b9a42a1c3d2 - Merge tag 'v6.2-rc1' into media_tree

Wed, 28 Dec 2022 16:07:44 +0000

Merge tag 'v6.2-rc1' into media_treeLinux 6.2-rc1* tag 'v6.2-rc1': (14398 commits) Linux 6.2-rc1 treewide: Convert del_timer*() to timer_shutdown*() pstore: Properly assign mem_type property pstore: Make sure CONFIG_PSTORE_PMSG selects CONFIG_RT_MUTEXES cfi: Fix CFI failure with KASAN perf python: Fix splitting CC into compiler and options afs: Stop implementing ->writepage() afs: remove afs_cache_netfs and afs_zap_permits() declarations afs: remove variable nr_servers afs: Fix lost servers_outstanding count ALSA: usb-audio: Add new quirk FIXED_RATE for JBL Quantum810 Wireless ALSA: azt3328: Remove the unused function snd_azf3328_codec_outl() gcov: add support for checksum field test_maple_tree: add test for mas_spanning_rebalance() on insufficient data maple_tree: fix mas_spanning_rebalance() on insufficient data hugetlb: really allocate vma lock for all sharable vmas kmsan: export kmsan_handle_urb kmsan: include linux/vmalloc.h mm/mempolicy: fix memory leak in set_mempolicy_home_node system call mm, mremap: fix mremap() expanding vma with addr inside vma ... List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

c183e6c3ec342624c43269c099050d01eeb67e63 - Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net

Wed, 21 Dec 2022 20:15:21 +0000

Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/netNo conflicts.Signed-off-by: Jakub Kicinski List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

563a5423b7b75e1bee8b91caddf3181cca2a74bd - Merge branch 'master' into mm-nonmm-stable

Fri, 16 Dec 2022 22:45:03 +0000

Merge branch 'master' into mm-nonmm-stable List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

bcfbff2ea3165d07775097df710cb4241fcd2567 - Merge branch 'master' into mm-stable

Fri, 16 Dec 2022 22:44:49 +0000

Merge branch 'master' into mm-stable List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

aa4800e31c547ed00681318335ca2298c4bca33a - Merge tag 'perf-tools-for-v6.2-1-2022-12-16' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux

Fri, 16 Dec 2022 19:21:20 +0000

Merge tag 'perf-tools-for-v6.2-1-2022-12-16' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linuxPull perf tools updates from Arnaldo Carvalho de Melo: "Libraries: - Drop the old copy of libtraceevent in tools/lib/traceevent/ now that all major distros ship it from its external repository. This is now just another feature detection, emitting a warning when the libtraceevent-dev[el] package isn't installed, disabling the build of perf features and tools that strictly require parsing things from tracefs while keeping the core functionality present and working with a subset of the events, the most used ones like CPU cycles, hardware cache and also vendor events, etc. This was tested with lots of containers for Fedora, Debian, OpenSUSE, Alpine Linux, Ubuntu, with cross builds, etc. Build: - Update to C standard to gnu11, like was done for the kernel. - Install the tools/lib/ libraries locally instead of having headers searched directly from the source code directories, to help the cases where we can build either from in-kernel source libraries or from the same library shipped as a distro package, as is the case with libbpf and was the case with libtraceevent. perf stat: - Do not delay the workload with --delay, the delay is just for starting to count the events, to skip noise at workload startup. - When we have events for each cgroup, the metric should be printed for each cgroup separately. $ perf stat -a --for-each-cgroup system.slice,user.slice --metric-only sleep 1 Performance counter stats for 'system wide': GHz insn per cycle branch-misses of all branches system.slice 3.792 0.61 3.24% user.slice 3.661 2.32 0.37% - Fix printing field separator in CSV metrics output. - Fix --metric-only --json output. - Fix summary output in CSV with --metric-only. - Update event group check for support of uncore event. perf test: - Stop requiring a C toolchain in shell tests, instead add a workload option that has all the previously C snippets built as part of 'perf test -w' that then get used in the 'perf test' shell scripts. - Add event group test for events in multiple PMUs - The "kernel lock contention analysis" test should not print warnings in quiet mode. - Add attr tests for ARM64's new VG register. - Fix record test on KVM guests, as using precise flag with the br_inst_retired.near_call event causes the test fail on KVM guests, even when the guests have PMU forwarding enabled and the event itself is supported, so just remove the precise flag from the event. - Add mechanism for skipping attr tests on specific kernel versions where it is known that these checks will fail. - Skip watchpoint tests if no watchpoints available. - Add more Intel PT 'perf test' entries: hybrid CPUs, split the packet decoder into a suite of subtests. perf script: - Introduce task analyzer python script, where one first records some events: Recording can be done in two ways: $ perf script record tasks-analyzer -- sleep 10 $ perf record -e sched:sched_switch -a -- sleep 10 The script can parse any perf.data files, as long as it has sched:sched_switch events, other events will be ignored. The most simple report use case is to just call the script without arguments. Runtime is the time the task was running on the CPU, Time Out-In is the time between the process being scheduled *out* and scheduled back *in*. So the last time span between two executions: $ perf script report tasks-analyzer Switched-In Switched-Out CPU PID TID Comm Runtime Time Out-In 15576.658891407 15576.659156086 4 2412 2428 gdbus 265 1949 15576.659111320 15576.659455410 0 2412 2412 gnome-shell 344 2267 15576.659491326 15576.659506173 2 74 74 kworker/2:1 15 13145 15576.659506173 15576.659825748 2 2858 2858 gnome-terminal- 320 63263 15576.659871270 15576.659902872 6 20932 20932 kworker/u16:0 32 2314582 15576.659909951 15576.659945501 3 27264 27264 sh 36 -1 15576.659853285 15576.659971052 7 27265 27265 perf 118 5050741 [...] perf lock: - Allow concurrent record and report to support live monitoring of kernel lock contention without BPF: # perf lock record -a -o- sleep 1 | perf lock contention -i- contended total wait max wait avg wait type caller 2 10.27 us 6.17 us 5.13 us spinlock load_balance+0xc03 1 5.29 us 5.29 us 5.29 us rwlock:W ep_scan_ready_list+0x54 1 4.12 us 4.12 us 4.12 us spinlock smpboot_thread_fn+0x116 1 3.28 us 3.28 us 3.28 us mutex pipe_read+0x50 - Implement -t/--threads option when using BPF: $ sudo ./perf lock contention -abt -E 5 sleep 1 contended total wait max wait avg wait pid comm 1 740.66 ms 740.66 ms 740.66 ms 1950 nv_queue 3 305.50 ms 298.19 ms 101.83 ms 1884 nvidia-modeset/ 1 25.14 us 25.14 us 25.14 us 2725038 EventManager_De 12 23.09 us 9.30 us 1.92 us 0 swapper 1 20.18 us 20.18 us 20.18 us 2725033 EventManager_De - Add -l/--lock-addr to aggregate per-lock-instance contention: $ sudo ./perf lock contention -abl sleep 1 contended total wait max wait avg wait address symbol 1 36.28 us 36.28 us 36.28 us ffff92615d6448b8 9 10.91 us 1.84 us 1.21 us ffffffffbaed50c0 rcu_state 1 10.49 us 10.49 us 10.49 us ffff9262ac4f0c80 8 4.68 us 1.67 us 585 ns ffffffffbae07a40 jiffies_lock 3 3.03 us 1.45 us 1.01 us ffff9262277861e0 1 924 ns 924 ns 924 ns ffff926095ba9d20 1 436 ns 436 ns 436 ns ffff9260bfda4f60 perf record: - Add remaining branch filters: "no_cycles", "no_flags" & "hw_index", to be used with hardware such as Intel's LBR that allows things like stitching stacks of two samples to overcome the limits of the number of LBR registers. Symbol resolution: - Handle .debug files created with 'objcopy --only-keep-debug', where program headers are zeroed and thus can't be used for adjustments, use the info in the runtime_ss (runtime ELF) instead. perf trace: - Add BPF based augmenter for the 'perf_event_open's 'struct perf_event_attr' argument. - Add BPF based augmenter for the 'clock_gettime's 'struct timespec' argument. - In both cases the syscall tracepoint has just the pointer value, we need to hook a BPF program to collect the pointer contents, and then, in userspace, pretty print it in 'perf trace'. perf list: - Introduce JSON output of events. - Streamline how the expression specifying what events should be shown is handled, fixing several corner cases, such as the metric filter that is specified as a glob but was using strstr(). perf probe: - Fix to avoid crashing if DW_AT_decl_file is NULL, coping with clang generating DWARF5 like that. - Use dwarf_attr_integrate() as generic DWARF attr accessor as it supersedes dwarf_attr(), supporting abstact origin DIEs. perf inject: - Set PERF_RECORD_MISC_BUILD_ID_SIZE in the PERF_RECORD_HEADER_BUILD_ID so that perf.data readers can get the real build-id size and avoid trailing zeroes. perf data: - Add tracepoint fields when converting a perf.data file to JSON. arm64: - Fix mksyscalltbl, don't lose syscalls due to sort -nu. - Add Arm Neoverse V2 PMU events. riscv: - Add riscv sbi firmware std event files. - Add Sifive U74 vendor events (JSON) file. - Add some more events and metrics for Alderlake/Alderlake-N. Documentation: - Add data documentation for the PMU structs in the C source code. Miscellaneous: - Periodic sanitization of headers, adding missing includes, removing needless ones, creating new ones, etc. - Use sig_atomic_t for signal handlers to avoid undefined behaviour in all perf tools. - Fixes for libbpf 1.0+ compatibility (maps, etc) on 'perf trace' BPF examples. - Remove some old perf bpf examples, leave the best ones that demonstrate how to associate BPF functions to points in the kernel. - Make quiet mode consistent between tools. - Use dedicated non-atomic clear/set bit helpers. - Use "grep -E" instead of "egrep" as recommended by warning emitted by GNU grep since at least version 3.8. - Complete list of supported subcommands in the 'perf daemon' help message. - Update John Garry's email address for arm64 perf tooling on the MAINTAINERS file, he moved from Huawei to Oracle"* tag 'perf-tools-for-v6.2-1-2022-12-16' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux: (239 commits) libperf: Fix install_pkgconfig target perf tools: Use "grep -E" instead of "egrep" perf stat: Do not delay the workload with --delay perf evlist: Remove group option. perf build: Fix python/perf.so library's name perf test arm64: Add attr tests for new VG register perf test: Add mechanism for skipping attr tests on kernel versions perf test: Add mechanism for skipping attr tests on auxiliary vector values perf test: Add ability to test exit code for attr tests perf test: add new task-analyzer tests perf script: task-analyzer add csv support perf script: Introduce task analyzer python script perf cs-etm: Print auxtrace info even if OpenCSD isn't linked perf cs-etm: Cleanup cs_etm__process_auxtrace_info() perf cs-etm: Tidy up auxtrace info header printing perf cs-etm: Remove unused stub methods perf cs-etm: Print unknown header version as an error perf test: Update perf lock contention test perf lock contention: Add -l/--lock-addr option perf lock contention: Implement -t/--threads option for BPF ... List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record

e76aff0523f7d3393f967bc13b10bb04b759abfa - perf script: Introduce task analyzer python script

Tue, 06 Dec 2022 15:44:04 +0000

perf script: Introduce task analyzer python scriptIntroduce a new 'perf script' to analyze task scheduling behavior.During the task analysis, some data is always needed - which goes beyondthe simple time of switching on and off a task (process/thread). Thisconcerns for example the runtime of a process or the frequency withwhich the process was called. This script serves to simplify thisrecurring analyze process. It immediately provides the user with helpfultask characteristic information about the tasks runtimes.Usage:Recorded can be in two ways: $ perf script record tasks-analyzer -- sleep 10 $ perf record -e sched:sched_switch -a -- sleep 10The script can parse all perf.data files, most important: sched:sched_switchevents are mandatory, other events will be ignored.Most simple report use case is to just call the script without arguments: $ perf script report tasks-analyzer Switched-In Switched-Out CPU PID TID Comm Runtime Time Out-In 15576.658891407 15576.659156086 4 2412 2428 gdbus 265 1949 15576.659111320 15576.659455410 0 2412 2412 gnome-shell 344 2267 15576.659491326 15576.659506173 2 74 74 kworker/2:1 15 13145 15576.659506173 15576.659825748 2 2858 2858 gnome-terminal- 320 63263 15576.659871270 15576.659902872 6 20932 20932 kworker/u16:0 32 2314582 15576.659909951 15576.659945501 3 27264 27264 sh 36 -1 15576.659853285 15576.659971052 7 27265 27265 perf 118 5050741 [...]What is not shown here are the ASCII color sequences. For example, ifthe task consists of only one thread, the TID is grayed out.Runtime is the time the task was running on the CPU, Time Out-In is thetime between the process being scheduled *out* and scheduled back *in*.So the last time span between two executions. If -1 is printed, then thetask simply ran the first time in the measurements - a Out-In deltacould not be calculated.In addition to the chronological representation, there is a summary ontask level. This output can be additionally switched on via the--summary option and provides information such as max, min & averageruntime per process. The maximum runtime is often important fordebugging. The call looks like this: $ perf script report tasks-analyzer --summary Summary Task Information Runtime Information PID TID Comm Runs Accumulated Mean Median Min Max Max At 14 14 ksoftirqd/0 13 334 26 15 9 127 15571.621211956 15 15 rcu_preempt 133 1778 13 13 2 33 15572.581176024 16 16 migration/0 3 49 16 13 12 24 15571.608915425 20 20 migration/1 3 34 11 13 8 13 15571.639101555 25 25 migration/2 3 32 11 12 9 12 15575.639239896 [...]Besides these two options, there are a number of other options that change theoutput and behavior. This can be queried via --help. Options worth mentioning include:- filter-tasks - filter out unneeded tasks, --filter-task 1337,/sbin/init- highlight-tasks - more pleasant focusing, --highlight-tasks 1:red,mutt:yellow- extended-times - show combinations of elapsed times between schedule in/schedule out- summary-extended - summary with additional information, like maximum delta time statistics- rename-comms-by-tids - handy for inexpressive processnames like python, --rename 1337:my-python-app- ms - show timestamps in milliseconds, nanoseconds is also possible (--ns)- time-limit - limit the analyzer to a time range, --time-limit 15576.0:15576.1Script is tested and prime time ready for python2 & python3:- make PYTHON=python3 prefix=/usr/local install- make PYTHON=python2 prefix=/usr/local installSigned-off-by: Hagen Paul Pfeifer Cc: Andi Kleen Cc: Ian Rogers Cc: Jiri Olsa Cc: Namhyung Kim Link: https://lore.kernel.org/r/20221206154406.41941-2-petar.gligor@gmail.comSigned-off-by: Petar Gligoric Signed-off-by: Arnaldo Carvalho de Melo List of files: /linux/tools/perf/scripts/python/bin/task-analyzer-record