Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: support multi nodes config for es #3568

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 26 additions & 7 deletions cmd/internal/storage/elasticsearch/elasticsearch.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import (
"flag"
"fmt"
"os"
"strings"
"sync"
"time"

Expand Down Expand Up @@ -51,6 +52,8 @@ var (
argIndexName = flag.String("storage_driver_es_index", "cadvisor", "ElasticSearch index name")
argTypeName = flag.String("storage_driver_es_type", "stats", "ElasticSearch type name")
argEnableSniffer = flag.Bool("storage_driver_es_enable_sniffer", false, "ElasticSearch uses a sniffing process to find all nodes of your cluster by default, automatically")
argUserName = flag.String("storage_driver_es_username", "", "ElasticSearch basic auth username")
argPassword = flag.String("storage_driver_es_password", "", "ElasticSearch basic auth password")
)

func new() (storage.StorageDriver, error) {
Expand All @@ -64,6 +67,8 @@ func new() (storage.StorageDriver, error) {
*argTypeName,
*argElasticHost,
*argEnableSniffer,
*argUserName,
*argPassword,
)
}

Expand Down Expand Up @@ -124,29 +129,43 @@ func newStorage(
typeName,
elasticHost string,
enableSniffer bool,
username string,
password string,
) (storage.StorageDriver, error) {
// Remove all spaces to help user to configure
elasticHost = strings.ReplaceAll(elasticHost, " ", "")
hosts := strings.Split(elasticHost, ",")

// Obtain a client and connect to the default Elasticsearch installation
// on 127.0.0.1:9200. Of course you can configure your client to connect
// to other hosts and configure it in various other ways.
client, err := elastic.NewClient(
elastic.SetHealthcheck(true),
elastic.SetSniff(enableSniffer),
elastic.SetHealthcheckInterval(30*time.Second),
elastic.SetURL(elasticHost),
elastic.SetURL(hosts...),
elastic.SetBasicAuth(username, password),
)
if err != nil {
// Handle error
return nil, fmt.Errorf("failed to create the elasticsearch client - %s", err)
}

// Ping the Elasticsearch server to get e.g. the version number
info, code, err := client.Ping().URL(elasticHost).Do()
if err != nil {
// Handle error
return nil, fmt.Errorf("failed to ping the elasticsearch - %s", err)

// Just ping anyone of hosts successfully will be ok
var res *elastic.PingResult
var code int
for _, host := range hosts {
res, code, err = client.Ping().URL(host).Do()
if err == nil {
break
}
fmt.Printf("ping host %s failed, code: %d, err: %s", host, code, err)
}
if res == nil {
return nil, fmt.Errorf("failed to ping any host of the elasticsearch")
}
fmt.Printf("Elasticsearch returned with code %d and version %s", code, info.Version.Number)
fmt.Printf("Elasticsearch returned with code %d and version %s", code, res.Version.Number)

ret := &elasticStorage{
client: client,
Expand Down
8 changes: 7 additions & 1 deletion docs/storage/elasticsearch.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Exporting cAdvisor Stats to ElasticSearch

cAdvisor supports exporting stats to [ElasticSearch](https://www.elastic.co/). To use ES, you need to provide the additional flags to cAdvisor:
cAdvisor supports exporting stats to [ElasticSearch](https://www.elastic.co/). To use ES, you need to provide the
additional flags to cAdvisor:

Set the storage driver as ES:

Expand All @@ -12,6 +13,8 @@ Specify ES host address:

```
-storage_driver_es_host="http://elasticsearch:9200"
# If you has several hosts, just use comma to separate it.
-storage_driver_es_host="http://elasticsearch1:9200,http://elasticsearch2:9200,http://elasticsearch3:9200"
```

There are also optional flags:
Expand All @@ -21,6 +24,9 @@ There are also optional flags:
-storage_driver_es_type="stats"
# ElasticSearch can use a sniffing process to find all nodes of your cluster automatically. False by default.
-storage_driver_es_enable_sniffer=false
# ElasticSearch basic auth for http request, only works when any one of them is not empty
-storage_driver_es_username="xxx"
-storage_driver_es_password="xxx"
```

# Examples
Expand Down
Loading