input.sysstat always returns pct_pct_Fields in pct_util, so a "pct_" too much

[StefanSa]

Relevant telegraf.conf:

[[inputs.sysstat]]
sadc_path = "/usr/lib/sysstat/sadc" # required
activities = ["DISK"]
[inputs.sysstat.options]
-C = "cpu"
-B = "paging"
-b = "io"
"-n ALL" = "network"
"-P ALL" = "per_cpu"
-q = "queue"
-r = "mem_util"
-S = "swap_util"
-u = "cpu_util"
-d = "disk"
-v = "inode"
-W = "swap"
-w = "task"

System info:

Telegraf v1.7.0-1
Ubuntu Xenial

Steps to reproduce:

telegraf --config telegraf.conf --input-filter sysstat --test

Expected behavior:

Fields in cpu_util, for example, pct_idle

Actual behavior:

Instead of pct_idle, pct_pct_idle so a "pct_" too much

Additional info:

> cpu_util,device=all,host=vpn,hostgroup=IT,role=VPN-Access pct_pct_idle=99.49,pct_pct_iowait=0,pct_pct_nice=0,pct_pct_steal=0,pct_pct_system=0,pct_pct_user=0.51 1530183608000000000

[glinton]

Thanks for submitting this, can you by any chance see if sysstat is outputting metrics with two percent signs (%%idle, %%iowait, etc...), if you're familiar (I'm trying to replicate and find that command myself.)

[StefanSa]

As i see it, only "one" percent sign.
mpstat output:
root@openvpnas2:~# mpstat -P ALL Linux 4.4.0-128-generic (VPN) 06/29/2018 _x86_64_ (2 CPU) 10:44:48 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 10:44:48 AM all 0.36 0.00 0.25 0.02 0.00 0.04 0.00 0.00 0.00 99.33 10:44:48 AM 0 0.37 0.00 0.26 0.02 0.00 0.06 0.00 0.00 0.00 99.29 10:44:48 AM 1 0.35 0.00 0.25 0.02 0.00 0.02 0.00 0.00 0.00 99.37

sar output:
root@openvpnas2:~# sar -u -o sarfile 2 5 Linux 4.4.0-128-generic (VPN) 06/29/2018 _x86_64_ (2 CPU) 10:45:49 AM CPU %user %nice %system %iowait %steal %idle 10:45:51 AM all 0.76 0.00 0.76 0.00 0.00 98.49 10:45:53 AM all 0.25 0.00 0.25 0.00 0.00 99.50 10:45:55 AM all 0.00 0.00 0.25 0.00 0.00 99.75 10:45:57 AM all 0.00 0.00 0.00 0.00 0.00 100.00 10:45:59 AM all 0.50 0.00 1.00 0.00 0.00 98.50 Average: all 0.30 0.00 0.45 0.00 0.00 99.24

[danielnelson]

With the default settings my cpu_util measurements have a single pct_:

cpu_util,device=all pct_idle=97.24,pct_iowait=0.25,pct_nice=0,pct_steal=0,pct_system=0.5,pct_user=2.01 1530565084000000000

@StefanSa Can you run these commands, adjusting the path to sadc/sadf as needed:

sadc -S DISK 1 2 /tmp/telegraf-sysstat
sadf -p -- -p -u /tmp/telegraf-sysstat

[StefanSa]

@danielnelson
sadc -S DISK 1 2 /tmp/telegraf-sysstat
sadf -p -- -p -u /tmp/telegraf-sysstat

sadf -p -- -p -u /tmp/telegraf-sysstat
vpn	1	2018-07-04 08:30:27 UTC	all	%%user	0.00
vpn	1	2018-07-04 08:30:27 UTC	all	%%nice	0.00
vpn	1	2018-07-04 08:30:27 UTC	all	%%system	0.50
vpn	1	2018-07-04 08:30:27 UTC	all	%%iowait	0.00
vpn	1	2018-07-04 08:30:27 UTC	all	%%steal	0.00
vpn	1	2018-07-04 08:30:27 UTC	all	%%idle	99.50

the same on a centos machine:

sadf -p -- -p -u /tmp/telegraf-sysstat
logger	1	2018-07-04 08:35:16 UTC	all	%user	0,13
logger	1	2018-07-04 08:35:16 UTC	all	%nice	1,82
logger	1	2018-07-04 08:35:16 UTC	all	%system	0,44
logger	1	2018-07-04 08:35:16 UTC	all	%iowait	0,00
logger	1	2018-07-04 08:35:16 UTC	all	%steal	0,00
logger	1	2018-07-04 08:35:16 UTC	all	%idle	97,62

[pulord]

the same problem! How to fix it @danielnelson . thanks

[pulord]

the same problem! How to fix it @danielnelson . thanks

sdf cmd result contains two %, so after replaced, the field prefix with pct_pct

sdf output

prod-assetdb-d001 300 2019-06-15 16:05:01 UTC all %%user 0.12
prod-assetdb-d001 300 2019-06-15 16:05:01 UTC all %%nice 0.00
prod-assetdb-d001 300 2019-06-15 16:05:01 UTC all %%system 0.12
prod-assetdb-d001 300 2019-06-15 16:05:01 UTC all %%iowait 0.01
prod-assetdb-d001 300 2019-06-15 16:05:01 UTC all %%steal 0.00
prod-assetdb-d001 300 2019-06-15 16:05:01 UTC all %%idle 99.75
prod-assetdb-d001 600 2019-06-15 16:15:01 UTC all %%user 0.12
prod-assetdb-d001 600 2019-06-15 16:15:01 UTC all %%nice 0.00

System info:

LSB Version: core-9.20170808ubuntu1-noarch:security-9.20170808ubuntu1-noarch
Distributor ID: Ubuntu
Description: Ubuntu 18.04.2 LTS
Release: 18.04
Codename: bionic

Bug

telegraf/plugins/inputs/sysstat/sysstat.go

Line 338 in d260437

`%`, "pct_",

[pythonwood]

why it still happen now? (linux x64.tar.gz)

`$ usr/bin/telegraf --config etc/telegraf/telegraf.conf --input-filter sysstat --test
2019-07-21T12:55:02Z I! Starting Telegraf 1.11.2

sysstat_task,host=ubuntu18 cswch_per_s=4624,proc_per_s=0 1563713703000000000
sysstat_mem_util,host=ubuntu18 kbactive=5029420,kbavail=2497120,kbbuffers=140268,kbcached=1981484,kbcommit=18556172,kbdirty=3996,kbinact=1308492,kbmemfree=1296148,kbmemused=6806824,pct_pct_commit=229,pct_pct_memused=84 1563713703000000000
sysstat_io,host=ubuntu18 bread_per_s=0,bwrtn_per_s=8,rtps=0,tps=1,wtps=1 1563713703000000000
sysstat_disk,device=sda,host=ubuntu18 aqu-sz=0,areq-sz=4,await=0,pct_util=0,rkB_per_s=0,svctm=0,tps=1,wkB_per_s=4 1563713703000000000
sysstat_inode,host=ubuntu18 dentunusd=31126,file-nr=14816,inode-nr=50133,pty-nr=10 1563713703000000000
sysstat_queue,host=ubuntu18 blocked=0,ldavg-1=0.52,ldavg-15=0.62,ldavg-5=0.58,plist-sz=963,runq-sz=0 1563713703000000000
sysstat_swap_util,host=ubuntu18 kbswpcad=0,kbswpfree=0,kbswpused=0,pct_pct_swpcad=0,pct_pct_swpused=0 1563713703000000000
sysstat_per_cpu,device=all,host=ubuntu18 pct_pct_idle=87.94,pct_pct_iowait=0,pct_pct_nice=0,pct_pct_steal=0,pct_pct_system=3.77,pct_pct_user=8.29 1563713703000000000
sysstat_per_cpu,device=cpu0,host=ubuntu18 pct_idle=88.12,pct_iowait=0,pct_nice=0,pct_steal=0,pct_system=3.96,pct_user=7.92 1563713703000000000
sysstat_per_cpu,device=cpu1,host=ubuntu18 pct_idle=87.88,pct_iowait=0,pct_nice=0,pct_steal=0,pct_system=2.02,pct_user=10.1 1563713703000000000
sysstat_per_cpu,device=cpu2,host=ubuntu18 pct_idle=87.13,pct_iowait=0,pct_nice=0,pct_steal=0,pct_system=5.94,pct_user=6.93 1563713703000000000
sysstat_per_cpu,device=cpu3,host=ubuntu18 pct_idle=87,pct_iowait=0,pct_nice=0,pct_steal=0,pct_system=3,pct_user=10 1563713703000000000
sysstat_cpu_util,device=all,host=ubuntu18 pct_pct_idle=87.94,pct_pct_iowait=0,pct_pct_nice=0,pct_pct_steal=0,pct_pct_system=3.77,pct_pct_user=8.29 1563713703000000000
sysstat_paging,host=ubuntu18 fault_per_s=47,majflt_per_s=0,pct_pct_vmeff=0,pgfree_per_s=271,pgpgin_per_s=0,pgpgout_per_s=4,pgscand_per_s=0,pgscank_per_s=0,pgsteal_per_s=0 1563713703000000000
sysstat_cpu,device=all,host=ubuntu18 pct_pct_idle=87.94,pct_pct_iowait=0,pct_pct_nice=0,pct_pct_steal=0,pct_pct_system=3.77,pct_pct_user=8.29 1563713703000000000
sysstat_swap,host=ubuntu18 pswpin_per_s=0,pswpout_per_s=0 1563713703000000000
sysstat_network,device=all,host=ubuntu18 dropd_per_s=0,flw_lim_per_s=0,rx_rps_per_s=0,squeezd_per_s=0,total_per_s=6 1563713703000000000
sysstat_network,device=enp8s0,host=ubuntu18 coll_per_s=0,pct_ifutil=0,rxcmp_per_s=0,rxdrop_per_s=0,rxerr_per_s=0,rxfifo_per_s=0,rxfram_per_s=0,rxkB_per_s=0,rxmcst_per_s=0,rxpck_per_s=0,txcarr_per_s=0,txcmp_per_s=0,txdrop_per_s=0,txerr_per_s=0,txfifo_per_s=0,txkB_per_s=0,txpck_per_s=0 1563713703000000000
sysstat_network,device=virbr0,host=ubuntu18 coll_per_s=0,pct_ifutil=0,rxcmp_per_s=0,rxdrop_per_s=0,rxerr_per_s=0,rxfifo_per_s=0,rxfram_per_s=0,rxkB_per_s=0,rxmcst_per_s=0,rxpck_per_s=0,txcarr_per_s=0,txcmp_per_s=0,txdrop_per_s=0,txerr_per_s=0,txfifo_per_s=0,txkB_per_s=0,txpck_per_s=0 1563713703000000000
sysstat_network,device=wlo1,host=ubuntu18 coll_per_s=0,pct_ifutil=0,rxcmp_per_s=0,rxdrop_per_s=0,rxerr_per_s=0,rxfifo_per_s=0,rxfram_per_s=0,rxkB_per_s=0.32,rxmcst_per_s=0,rxpck_per_s=4,txcarr_per_s=0,txcmp_per_s=0,txdrop_per_s=0,txerr_per_s=0,txfifo_per_s=0,txkB_per_s=0.41,txpck_per_s=4 1563713703000000000
sysstat_network,device=lo,host=ubuntu18 coll_per_s=0,pct_ifutil=0,rxcmp_per_s=0,rxdrop_per_s=0,rxerr_per_s=0,rxfifo_per_s=0,rxfram_per_s=0,rxkB_per_s=0.1,rxmcst_per_s=0,rxpck_per_s=2,txcarr_per_s=0,txcmp_per_s=0,txdrop_per_s=0,txerr_per_s=0,txfifo_per_s=0,txkB_per_s=0.1,txpck_per_s=2 1563713703000000000
sysstat_network,device=virbr0-nic,host=ubuntu18 coll_per_s=0,pct_ifutil=0,rxcmp_per_s=0,rxdrop_per_s=0,rxerr_per_s=0,rxfifo_per_s=0,rxfram_per_s=0,rxkB_per_s=0,rxmcst_per_s=0,rxpck_per_s=0,txcarr_per_s=0,txcmp_per_s=0,txdrop_per_s=0,txerr_per_s=0,txfifo_per_s=0,txkB_per_s=0,txpck_per_s=0 1563713703000000000
sysstat_network,host=ubuntu18 access_per_s=0,badcall_per_s=0,call_per_s=0,getatt_per_s=0,hit_per_s=0,ip-frag=0,miss_per_s=0,packet_per_s=0,rawsck=0,read_per_s=0,retrans_per_s=0,saccess_per_s=0,scall_per_s=0,sgetatt_per_s=0,sread_per_s=0,swrite_per_s=0,tcp-tw=2,tcp_per_s=0,tcpsck=57,totsck=1451,udp_per_s=0,udpsck=10,write_per_s=0 1563713703000000000

`

[glinton]

@pythonwood try using a nightly build the fix has been merged, but is scheduled for release in 1.12