Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add docker input server version #4035

Merged
merged 3 commits into from
Apr 23, 2018
Merged

Add docker input server version #4035

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
11bd6df
Add docker server version tag.
danielnelson Apr 17, 2018
00136c7
Update docker README
danielnelson Apr 17, 2018
862038f
Use docker API 1.24
danielnelson Apr 17, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
204 changes: 96 additions & 108 deletions plugins/inputs/docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,11 @@ The docker plugin uses the Docker Engine API to gather metrics on running
docker containers.

The docker plugin uses the [Official Docker Client](https://github.com/moby/moby/tree/master/client)
to gather stats from the [Engine API](https://docs.docker.com/engine/api/v1.20/).
[Library Documentation](https://godoc.org/github.com/moby/moby/client)
to gather stats from the [Engine API](https://docs.docker.com/engine/api/v1.24/).

### Configuration:

```
```toml
# Read metrics about docker containers
[[inputs.docker]]
## Docker Endpoint
Expand Down Expand Up @@ -76,15 +75,57 @@ may prefer to exclude them:
```


### Measurements & Fields:
### Metrics:

Every effort was made to preserve the names based on the JSON response from the
docker API.

Note that the docker_container_cpu metric may appear multiple times per collection,
based on the availability of per-cpu stats on your system.
- docker
- tags:
- unit
- engine_host
- server_version
- fields:
- n_used_file_descriptors
- n_cpus
- n_containers
- n_containers_running
- n_containers_stopped
- n_containers_paused
- n_images
- n_goroutines
- n_listener_events
- memory_total
- pool_blocksize

- docker_data
- tags:
- unit
- engine_host
- server_version
- fields:
- available
- total
- used

- docker_metadata
- tags:
- unit
- engine_host
- server_version
- fields:
- available
- total
- used

- docker_container_mem
- tags:
- engine_host
- server_version
- container_image
- container_name
- container_version
- fields:
- total_pgmafault
- cache
- mapped_file
Expand Down Expand Up @@ -119,7 +160,16 @@ based on the availability of per-cpu stats on your system.
- failcnt
- limit
- container_id

- docker_container_cpu
- tags:
- engine_host
- server_version
- container_image
- container_name
- container_version
- cpu
- fields:
- throttling_periods
- throttling_throttled_periods
- throttling_throttled_time
Expand All @@ -129,7 +179,16 @@ based on the availability of per-cpu stats on your system.
- usage_total
- usage_percent
- container_id

- docker_container_net
- tags:
- engine_host
- server_version
- container_image
- container_name
- container_version
- network
- fields:
- rx_dropped
- rx_bytes
- rx_errors
Expand All @@ -139,7 +198,16 @@ based on the availability of per-cpu stats on your system.
- tx_errors
- tx_bytes
- container_id

- docker_container_blkio
- tags:
- engine_host
- server_version
- container_image
- container_name
- container_version
- device
- fields:
- io_service_bytes_recursive_async
- io_service_bytes_recursive_read
- io_service_bytes_recursive_sync
Expand All @@ -151,118 +219,38 @@ based on the availability of per-cpu stats on your system.
- io_serviced_recursive_total
- io_serviced_recursive_write
- container_id
- docker_
- n_used_file_descriptors
- n_cpus
- n_containers
- n_containers_running
- n_containers_stopped
- n_containers_paused
- n_images
- n_goroutines
- n_listener_events
- memory_total
- pool_blocksize
- docker_data
- available
- total
- used
- docker_metadata
- available
- total
- used
- docker_swarm
- tasks_desired
- tasks_running


### Tags:
#### Docker Engine tags
- docker (memory_total)
- unit=bytes
- engine_host
- docker (pool_blocksize)
- unit=bytes
- engine_host
- docker_data
- unit=bytes
- engine_host
- docker_metadata
- unit=bytes
- engine_host

#### Docker Container tags
- Tags on all containers:
- docker_container_health
- tags:
- engine_host
- server_version
- container_image
- container_name
- container_version
- docker_container_mem specific:
- docker_container_cpu specific:
- cpu
- docker_container_net specific:
- network
- docker_container_blkio specific:
- device
- docker_container_health specific:
- health_status
- failing_streak
- docker_swarm specific:
- fields:
- health_status (string)
- failing_streak (integer)

- docker_swarm
- tags:
- service_id
- service_name
- service_mode
- fields:
- tasks_desired
- tasks_running

### Example Output:

```
% ./telegraf --config ~/ws/telegraf.conf --input-filter docker --test
* Plugin: docker, Collection 1
> docker n_cpus=8i 1456926671065383978
> docker n_used_file_descriptors=15i 1456926671065383978
> docker n_containers=7i 1456926671065383978
> docker n_containers_running=7i 1456926671065383978
> docker n_containers_stopped=3i 1456926671065383978
> docker n_containers_paused=0i 1456926671065383978
> docker n_images=152i 1456926671065383978
> docker n_goroutines=36i 1456926671065383978
> docker n_listener_events=0i 1456926671065383978
> docker,unit=bytes memory_total=18935443456i 1456926671065383978
> docker,unit=bytes pool_blocksize=65540i 1456926671065383978
> docker_data,unit=bytes available=24340000000i,total=107400000000i,used=14820000000i 1456926671065383978
> docker_metadata,unit=bytes available=2126999999i,total=2146999999i,used=20420000i 145692667106538
> docker_container_mem,
container_image=spotify/kafka,container_name=kafka \
active_anon=52568064i,active_file=6926336i,cache=12038144i,fail_count=0i,\
hierarchical_memory_limit=9223372036854771712i,inactive_anon=52707328i,\
inactive_file=5111808i,limit=1044578304i,mapped_file=10301440i,\
max_usage=140656640i,pgfault=63762i,pgmajfault=2837i,pgpgin=73355i,\
pgpgout=45736i,rss=105275392i,rss_huge=4194304i,total_active_anon=52568064i,\
total_active_file=6926336i,total_cache=12038144i,total_inactive_anon=52707328i,\
total_inactive_file=5111808i,total_mapped_file=10301440i,total_pgfault=63762i,\
total_pgmafault=0i,total_pgpgin=73355i,total_pgpgout=45736i,\
total_rss=105275392i,total_rss_huge=4194304i,total_unevictable=0i,\
total_writeback=0i,unevictable=0i,usage=117440512i,writeback=0i 1453409536840126713
> docker_container_cpu,
container_image=spotify/kafka,container_name=kafka,cpu=cpu-total \
throttling_periods=0i,throttling_throttled_periods=0i,\
throttling_throttled_time=0i,usage_in_kernelmode=440000000i,\
usage_in_usermode=2290000000i,usage_system=84795360000000i,\
usage_total=6628208865i 1453409536840126713
> docker_container_cpu,
container_image=spotify/kafka,container_name=kafka,cpu=cpu0 \
usage_total=6628208865i 1453409536840126713
> docker_container_net,\
container_image=spotify/kafka,container_name=kafka,network=eth0 \
rx_bytes=7468i,rx_dropped=0i,rx_errors=0i,rx_packets=94i,tx_bytes=946i,\
tx_dropped=0i,tx_errors=0i,tx_packets=13i 1453409536840126713
> docker_container_blkio,
container_image=spotify/kafka,container_name=kafka,device=8:0 \
io_service_bytes_recursive_async=80216064i,io_service_bytes_recursive_read=79925248i,\
io_service_bytes_recursive_sync=77824i,io_service_bytes_recursive_total=80293888i,\
io_service_bytes_recursive_write=368640i,io_serviced_recursive_async=6562i,\
io_serviced_recursive_read=6492i,io_serviced_recursive_sync=37i,\
io_serviced_recursive_total=6599i,io_serviced_recursive_write=107i 1453409536840126713
>docker_swarm,
service_id=xaup2o9krw36j2dy1mjx1arjw,service_mode=replicated,service_name=test,\
tasks_desired=3,tasks_running=3 1508968160000000000
docker,engine_host=debian-stretch-docker,server_version=17.09.0-ce n_containers=6i,n_containers_paused=0i,n_containers_running=1i,n_containers_stopped=5i,n_cpus=2i,n_goroutines=41i,n_images=2i,n_listener_events=0i,n_used_file_descriptors=27i 1524002041000000000
docker,engine_host=debian-stretch-docker,server_version=17.09.0-ce,unit=bytes memory_total=2101661696i 1524002041000000000
docker_container_mem,container_image=telegraf,container_name=zen_ritchie,container_version=unknown,engine_host=debian-stretch-docker,server_version=17.09.0-ce active_anon=8327168i,active_file=2314240i,cache=27402240i,container_id="adc4ba9593871bf2ab95f3ffde70d1b638b897bb225d21c2c9c84226a10a8cf4",hierarchical_memory_limit=9223372036854771712i,inactive_anon=0i,inactive_file=25088000i,limit=2101661696i,mapped_file=20582400i,max_usage=36646912i,pgfault=4193i,pgmajfault=214i,pgpgin=9243i,pgpgout=520i,rss=8327168i,rss_huge=0i,total_active_anon=8327168i,total_active_file=2314240i,total_cache=27402240i,total_inactive_anon=0i,total_inactive_file=25088000i,total_mapped_file=20582400i,total_pgfault=4193i,total_pgmajfault=214i,total_pgpgin=9243i,total_pgpgout=520i,total_rss=8327168i,total_rss_huge=0i,total_unevictable=0i,total_writeback=0i,unevictable=0i,usage=36528128i,usage_percent=0.4342225020025297,writeback=0i 1524002042000000000
docker_container_cpu,container_image=telegraf,container_name=zen_ritchie,container_version=unknown,cpu=cpu-total,engine_host=debian-stretch-docker,server_version=17.09.0-ce container_id="adc4ba9593871bf2ab95f3ffde70d1b638b897bb225d21c2c9c84226a10a8cf4",throttling_periods=0i,throttling_throttled_periods=0i,throttling_throttled_time=0i,usage_in_kernelmode=40000000i,usage_in_usermode=100000000i,usage_percent=0,usage_system=6394210000000i,usage_total=117319068i 1524002042000000000
docker_container_cpu,container_image=telegraf,container_name=zen_ritchie,container_version=unknown,cpu=cpu0,engine_host=debian-stretch-docker,server_version=17.09.0-ce container_id="adc4ba9593871bf2ab95f3ffde70d1b638b897bb225d21c2c9c84226a10a8cf4",usage_total=20825265i 1524002042000000000
docker_container_cpu,container_image=telegraf,container_name=zen_ritchie,container_version=unknown,cpu=cpu1,engine_host=debian-stretch-docker,server_version=17.09.0-ce container_id="adc4ba9593871bf2ab95f3ffde70d1b638b897bb225d21c2c9c84226a10a8cf4",usage_total=96493803i 1524002042000000000
docker_container_net,container_image=telegraf,container_name=zen_ritchie,container_version=unknown,engine_host=debian-stretch-docker,network=eth0,server_version=17.09.0-ce container_id="adc4ba9593871bf2ab95f3ffde70d1b638b897bb225d21c2c9c84226a10a8cf4",rx_bytes=1576i,rx_dropped=0i,rx_errors=0i,rx_packets=20i,tx_bytes=0i,tx_dropped=0i,tx_errors=0i,tx_packets=0i 1524002042000000000
docker_container_blkio,container_image=telegraf,container_name=zen_ritchie,container_version=unknown,device=254:0,engine_host=debian-stretch-docker,server_version=17.09.0-ce container_id="adc4ba9593871bf2ab95f3ffde70d1b638b897bb225d21c2c9c84226a10a8cf4",io_service_bytes_recursive_async=27398144i,io_service_bytes_recursive_read=27398144i,io_service_bytes_recursive_sync=0i,io_service_bytes_recursive_total=27398144i,io_service_bytes_recursive_write=0i,io_serviced_recursive_async=529i,io_serviced_recursive_read=529i,io_serviced_recursive_sync=0i,io_serviced_recursive_total=529i,io_serviced_recursive_write=0i 1524002042000000000
docker_container_health,container_image=telegraf,container_name=zen_ritchie,container_version=unknown,engine_host=debian-stretch-docker,server_version=17.09.0-ce failing_streak=0i,health_status="healthy" 1524007529000000000
docker_swarm,service_id=xaup2o9krw36j2dy1mjx1arjw,service_mode=replicated,service_name=test tasks_desired=3,tasks_running=3 1508968160000000000
```
2 changes: 1 addition & 1 deletion plugins/inputs/docker/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import (
)

var (
version string
version = "1.24"
defaultHeaders = map[string]string{"User-Agent": "engine-api-cli-1.0"}
)

Expand Down
29 changes: 15 additions & 14 deletions plugins/inputs/docker/docker.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ type Docker struct {
client Client
httpClient *http.Client
engine_host string
serverVersion string
filtersCreated bool
labelFilter filter.Filter
containerFilter filter.Filter
Expand Down Expand Up @@ -301,7 +302,14 @@ func (d *Docker) gatherInfo(acc telegraf.Accumulator) error {
if err != nil {
return err
}

d.engine_host = info.Name
d.serverVersion = info.ServerVersion

tags := map[string]string{
"engine_host": d.engine_host,
"server_version": d.serverVersion,
}

fields := map[string]interface{}{
"n_cpus": info.NCPU,
Expand All @@ -315,15 +323,13 @@ func (d *Docker) gatherInfo(acc telegraf.Accumulator) error {
"n_listener_events": info.NEventsListener,
}
// Add metrics
acc.AddFields("docker",
fields,
map[string]string{"engine_host": d.engine_host},
now)
acc.AddFields("docker", fields, tags, now)
acc.AddFields("docker",
map[string]interface{}{"memory_total": info.MemTotal},
map[string]string{"unit": "bytes", "engine_host": d.engine_host},
tags,
now)
// Get storage metrics
tags["unit"] = "bytes"
for _, rawData := range info.DriverStatus {
// Try to convert string to int (bytes)
value, err := parseSize(rawData[1])
Expand All @@ -335,7 +341,7 @@ func (d *Docker) gatherInfo(acc telegraf.Accumulator) error {
// pool blocksize
acc.AddFields("docker",
map[string]interface{}{"pool_blocksize": value},
map[string]string{"unit": "bytes", "engine_host": d.engine_host},
tags,
now)
} else if strings.HasPrefix(name, "data_space_") {
// data space
Expand All @@ -348,16 +354,10 @@ func (d *Docker) gatherInfo(acc telegraf.Accumulator) error {
}
}
if len(dataFields) > 0 {
acc.AddFields("docker_data",
dataFields,
map[string]string{"unit": "bytes", "engine_host": d.engine_host},
now)
acc.AddFields("docker_data", dataFields, tags, now)
}
if len(metadataFields) > 0 {
acc.AddFields("docker_metadata",
metadataFields,
map[string]string{"unit": "bytes", "engine_host": d.engine_host},
now)
acc.AddFields("docker_metadata", metadataFields, tags, now)
}
return nil
}
Expand Down Expand Up @@ -388,6 +388,7 @@ func (d *Docker) gatherContainer(

tags := map[string]string{
"engine_host": d.engine_host,
"server_version": d.serverVersion,
"container_name": cname,
"container_image": imageName,
"container_version": imageVersion,
Expand Down
12 changes: 9 additions & 3 deletions plugins/inputs/docker/docker_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -615,7 +615,10 @@ func TestDockerGatherInfo(t *testing.T) {
"n_images": int(199),
"n_goroutines": int(39),
},
map[string]string{"engine_host": "absol"},
map[string]string{
"engine_host": "absol",
"server_version": "17.09.0-ce",
},
)

acc.AssertContainsTaggedFields(t,
Expand All @@ -626,8 +629,9 @@ func TestDockerGatherInfo(t *testing.T) {
"available": int64(36530000000),
},
map[string]string{
"unit": "bytes",
"engine_host": "absol",
"unit": "bytes",
"engine_host": "absol",
"server_version": "17.09.0-ce",
},
)
acc.AssertContainsTaggedFields(t,
Expand All @@ -648,6 +652,7 @@ func TestDockerGatherInfo(t *testing.T) {
"ENVVAR7": "ENVVAR8=ENVVAR9",
"label1": "test_value_1",
"label2": "test_value_2",
"server_version": "17.09.0-ce",
},
)
acc.AssertContainsTaggedFields(t,
Expand All @@ -670,6 +675,7 @@ func TestDockerGatherInfo(t *testing.T) {
"ENVVAR7": "ENVVAR8=ENVVAR9",
"label1": "test_value_1",
"label2": "test_value_2",
"server_version": "17.09.0-ce",
},
)
}
Expand Down
Loading