Type inference in combination with a compact C-like syntax means scripts can be written and edited in seconds, directly in your favorite shell. The language is heavily inspired by, though not nearly as powerful as, awk(1) and dtrace(1).
Designed with embedded systems in mind. Written in C, all ply needs to run is libc and a modern kernel with Linux BPF support. No external kernel modules, no LLVM, no python. Works on aarch64, arm, powerpc, and x86_64 today — adding an ISA is easy.
Data gathering and aggregation is done in the kernel using Linux BPF programs that are JIT compiled to native instructions on most common architectures. Thus, ply runs with very low overhead, allowing it to probe even the hottest code paths.
"yeah yeah whatever, so what can I actually do with it?!"
- well I'm glad you asked, dear reader, here are a few examples
#!/usr/bin/env ply kprobe:SyS_* { @syscalls[caller] = count(); }
This probe will be attached to all functions whose name
starts with SyS_
, i.e. all syscalls. On each
syscall, the probe will fire and index into the user-defined
aggregation @syscalls
using the program counter
caller
as the key and bump a counter.
ply will compile the script, attach it to the matching probes and start collecting data. On exit, ply will dump the values of all user-defined maps and aggregations.
~$ ply count-syscalls.ply ply: active ^Cply: deactivating @syscalls: { sys_sigaltstack+1 }: 1 { sys_set_tid_address+1 }: 2 { sys_sysinfo+1 }: 2 { sys_alarm+1 }: 2 { sys_mincore+1 }: 2 { sys_pread64+1 }: 2 { sys_newlstat+1 }: 2 { sys_unlink+1 }: 2 [REDACTED LINES] { sys_newfstat+1 }: 586 { sys_mmap_pgoff+1 }: 599 { sys_mmap+1 }: 600 { sys_rt_sigprocmask+1 }: 859 { sys_close+1 }: 863 { sys_ppoll+1 }: 1315 { sys_epoll_wait+1 }: 1731 { sys_poll+1 }: 2644 { sys_write+1 }: 3262 { sys_recvmsg+1 }: 3624 { sys_ioctl+1 }: 4423 { sys_read+1 }: 5059 { sys_futex+1 }: 7918
#!/usr/bin/env ply kretprobe:SyS_read { @["retsize"] = quantize(retval); }
This example shows a very simple script that instruments the
return of the read(2)
syscall and records the
distribution of the return argument.
User-defined aggregations in ply always start
with @
. When writing small scripts, only using
one aggregation, it is common and convenient to simply call
it @
.
~$ sudo ./read-dist.ply ply: active ^Cply: deactivating @: { retsize }: < 0 861 ┤████████▏ │ [ 0, 1] 865 ┤████████▏ │ [ 2, 3] 372 ┤███▌ │ [ 4, 7] 1 ┤ │ [ 8, 15] 1090 ┤██████████▎ │ [ 16, 31] 122 ┤█▏ │ [ 32, 63] 22 ┤▎ │ [ 64, 127] 25 ┤▎ │ [ 128, 255] 1 ┤ │ ... [ 512, 1k) 23 ┤▎ │ [ 1k, 2k) 5 ┤ │ ... [ 4k, 8k) 2 ┤ │ ... [ 16k, 32k) 11 ┤▏ │
#!/usr/bin/env ply kprobe:i2c_transfer { print(stack); }
Sometimes it can be useful to know how a particular location
is reached. kprobes
can get the current stack
trace via the stack
variable.
In this example, the stack trace is simply printed to
stdout
, but it can also be used as a map key in
an aggregation. I.e. it is possible to do frequency counting
based on how a function was reached.
root@chaos:~ $ ply ./i2c-stack.ply & root@chaos:~ $ ply: active root@chaos:~ $ hwclock -r i2c_transfer i2c_smbus_read_i2c_block_data+0x58 ds1307_native_smbus_read_block_data+0x88 ds1307_get_time+0x38 __rtc_read_time+0x54 rtc_read_time+0x3c rtc_dev_ioctl+0x318 do_vfs_ioctl+0xa0 sys_ioctl+0x44 __sys_trace_return Mon Feb 20 18:33:33 2017 0.000000 seconds root@chaos:~ $ fg ply ./i2c_stack.ply ^Cply: deactivating root@chaos:~ $
#!/usr/bin/env ply kprobe:do_sys_open { printf("%v(%v): %s\n", comm, pid, str(arg1)); }
Every time a process calls open(2)
print the calling process's
comm
, i.e. executable name, PID and the
filename by extracting a 128-byte string from the address of
the first argument using the str()
function.
~$ sudo ./opensoop.ply ply: active ply (28836): /sys/kernel/debug/tracing/events/enable SimpleCacheWork ( 5818): /home/wkz/.cache/google-chrome/Default/Cache/37586f4b9464a393_0 irqbalance ( 1083): /proc/interrupts irqbalance ( 1083): /proc/stat irqbalance ( 1083): /proc/irq/18/smp_affinity irqbalance ( 1083): /proc/irq/126/smp_affinity irqbalance ( 1083): /proc/irq/128/smp_affinity irqbalance ( 1083): /proc/irq/122/smp_affinity irqbalance ( 1083): /proc/irq/11/smp_affinity irqbalance ( 1083): /proc/irq/124/smp_affinity irqbalance ( 1083): /proc/irq/16/smp_affinity irqbalance ( 1083): /proc/irq/1/smp_affinity irqbalance ( 1083): /proc/irq/8/smp_affinity irqbalance ( 1083): /proc/irq/9/smp_affinity irqbalance ( 1083): /proc/irq/12/smp_affinity irqbalance ( 1083): /proc/irq/120/smp_affinity irqbalance ( 1083): /proc/irq/121/smp_affinity Chrome_IOThread ( 5361): /dev/shm/.org.chromium.Chromium.59XkZF SimpleCacheWork ( 5818): /home/wkz/.cache/google-chrome/Default/Cache/37586f4b9464a393_0 Core Thread ( 5368): /home/wkz/.config/spotify/Users/wkz-user/pending-messages.tmp Core Thread ( 5368): /home/wkz/.config/spotify/Users/wkz-user/pending-messages.tmp ^Cply: deactivating