diff --git a/CHANGELOG.md b/CHANGELOG.md index f10c712f34be..778da9cd499a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,13 +18,23 @@ * Postgres/MySQL/MSSQL datasources now per default uses `max open connections` = `unlimited` (earlier 10), `max idle connections` = `2` (earlier 10) and `connection max lifetime` = `4` hours (earlier unlimited) -# 5.3.2 (unreleased) +# 5.3.2 (2018-10-24) * **InfluxDB/Graphite/Postgres**: Prevent cross site scripting (XSS) in query editor [#13667](https://github.com/grafana/grafana/issues/13667), thx [@svenklemm](https://github.com/svenklemm) * **Postgres**: Fix template variables error [#13692](https://github.com/grafana/grafana/issues/13692), thx [@svenklemm](https://github.com/svenklemm) * **Cloudwatch**: Fix service panic because of race conditions [#13674](https://github.com/grafana/grafana/issues/13674), thx [@mtanda](https://github.com/mtanda) +* **Cloudwatch**: Fix check for invalid percentile statistics [#13633](https://github.com/grafana/grafana/issues/13633), thx [@apalaniuk](https://github.com/apalaniuk) * **Stackdriver/Cloudwatch**: Allow user to change unit in graph panel if cloudwatch/stackdriver datasource response doesn't include unit [#13718](https://github.com/grafana/grafana/issues/13718), thx [@mtanda](https://github.com/mtanda) +* **Stackdriver**: stackdriver user-metrics duplicated response when multiple resource types [#13691](https://github.com/grafana/grafana/issues/13691) +* **Variables**: Fix text box template variable doesn't work properly without a default value [#13666](https://github.com/grafana/grafana/issues/13666) +* **Variables**: Fix variable dependency check when using `${var}` format [#13600](https://github.com/grafana/grafana/issues/13600) +* **Dashboard**: Fix kiosk=1 url parameter should put dashboard in kiosk mode [#13764](https://github.com/grafana/grafana/pull/13764) * **LDAP**: Fix super admins can also be admins of orgs [#13710](https://github.com/grafana/grafana/issues/13710), thx [@adrien-f](https://github.com/adrien-f) +* **Provisioning**: Fix deleting provisioned dashboard folder should cleanup provisioning meta data [#13280](https://github.com/grafana/grafana/issues/13280) + +### Minor + +* **Docker**: adds curl back into the docker image for utility. [#13794](https://github.com/grafana/grafana/pull/13794) # 5.3.1 (2018-10-16) diff --git a/docs/sources/alerting/notifications.md b/docs/sources/alerting/notifications.md index 307af1ee15eb..b232ee78f270 100644 --- a/docs/sources/alerting/notifications.md +++ b/docs/sources/alerting/notifications.md @@ -140,7 +140,7 @@ In DingTalk PC Client: 6. There will be a Webhook URL in the panel, looks like this: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxx. Copy this URL to the grafana Dingtalk setting page and then click "finish". -Dingtalk supports the following "message type": `text`, `link` and `markdown`. Only the `text` message type is supported. +Dingtalk supports the following "message type": `text`, `link` and `markdown`. Only the `link` message type is supported. ### Kafka diff --git a/docs/sources/features/datasources/stackdriver.md b/docs/sources/features/datasources/stackdriver.md index 3ae2ed3df404..d19dbe4ea505 100644 --- a/docs/sources/features/datasources/stackdriver.md +++ b/docs/sources/features/datasources/stackdriver.md @@ -156,6 +156,16 @@ Example Alias By: `{{metric.type}} - {{metric.labels.instance_name}}` Example Result: `compute.googleapis.com/instance/cpu/usage_time - server1-prod` +It is also possible to resolve the name of the Monitored Resource Type. + +| Alias Pattern Format | Description | Example Result | +| ------------------------ | ------------------------------------------------| ---------------- | +| `{{resource.type}}` | returns the name of the monitored resource type | `gce_instance` | + +Example Alias By: `{{resource.type}} - {{metric.type}}` + +Example Result: `gce_instance - compute.googleapis.com/instance/cpu/usage_time` + ## Templating Instead of hard-coding things like server, application and sensor name in you metric queries you can use variables in their place. diff --git a/docs/sources/tutorials/index.md b/docs/sources/tutorials/index.md index cb11940c6dd3..90410e901d3e 100644 --- a/docs/sources/tutorials/index.md +++ b/docs/sources/tutorials/index.md @@ -1,5 +1,6 @@ +++ title = "Tutorials" +type = "docs" [menu.docs] identifier = "tutorials" weight = 6 @@ -11,7 +12,11 @@ This section of the docs contains a series for tutorials and stack setup guides. ## Articles -- [How to integrate Hubot with Grafana](hubot_howto.md) +- [Running Grafana behind a reverse proxy]({{< relref "behind_proxy.md" >}}) +- [API Tutorial: How To Create API Tokens And Dashboards For A Specific Organization]({{< relref "api_org_token_howto.md" >}}) +- [How to Use IIS with URL Rewrite as a Reverse Proxy for Grafana on Windows]({{< relref "iis.md" >}}) +- [How to integrate Hubot with Grafana]({{< relref "hubot_howto.md" >}}) +- [How to setup Grafana for high availability]({{< relref "ha_setup.md" >}}) ## External links diff --git a/latest.json b/latest.json index 4355e9a64b79..992af0a83366 100644 --- a/latest.json +++ b/latest.json @@ -1,4 +1,4 @@ { - "stable": "5.3.1", - "testing": "5.3.1" + "stable": "5.3.2", + "testing": "5.3.2" } diff --git a/packaging/docker/Dockerfile b/packaging/docker/Dockerfile index 890d6a4fb11a..dc8972b0ba04 100644 --- a/packaging/docker/Dockerfile +++ b/packaging/docker/Dockerfile @@ -25,7 +25,7 @@ ENV PATH=/usr/share/grafana/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bi WORKDIR $GF_PATHS_HOME -RUN apt-get update && apt-get install -qq -y libfontconfig ca-certificates && \ +RUN apt-get update && apt-get install -qq -y libfontconfig ca-certificates curl && \ apt-get autoremove -y && \ rm -rf /var/lib/apt/lists/* diff --git a/pkg/services/alerting/notifiers/dingding.go b/pkg/services/alerting/notifiers/dingding.go index 738e43af2d21..1ef085c82f18 100644 --- a/pkg/services/alerting/notifiers/dingding.go +++ b/pkg/services/alerting/notifiers/dingding.go @@ -57,6 +57,9 @@ func (this *DingDingNotifier) Notify(evalContext *alerting.EvalContext) error { message := evalContext.Rule.Message picUrl := evalContext.ImagePublicUrl title := evalContext.GetNotificationTitle() + if message == "" { + message = title + } bodyJSON, err := simplejson.NewJson([]byte(`{ "msgtype": "link", diff --git a/pkg/services/sqlstore/dashboard.go b/pkg/services/sqlstore/dashboard.go index e43279208e71..1b853d17b5f9 100644 --- a/pkg/services/sqlstore/dashboard.go +++ b/pkg/services/sqlstore/dashboard.go @@ -320,13 +320,18 @@ func DeleteDashboard(cmd *m.DeleteDashboardCommand) error { "DELETE FROM dashboard WHERE id = ?", "DELETE FROM playlist_item WHERE type = 'dashboard_by_id' AND value = ?", "DELETE FROM dashboard_version WHERE dashboard_id = ?", - "DELETE FROM dashboard WHERE folder_id = ?", "DELETE FROM annotation WHERE dashboard_id = ?", "DELETE FROM dashboard_provisioning WHERE dashboard_id = ?", } + if dashboard.IsFolder { + deletes = append(deletes, "DELETE FROM dashboard_provisioning WHERE dashboard_id in (select id from dashboard where folder_id = ?)") + deletes = append(deletes, "DELETE FROM dashboard WHERE folder_id = ?") + } + for _, sql := range deletes { _, err := sess.Exec(sql, dashboard.Id) + if err != nil { return err } diff --git a/pkg/services/sqlstore/dashboard_provisioning_test.go b/pkg/services/sqlstore/dashboard_provisioning_test.go index 7ef45df31529..1b7a39767279 100644 --- a/pkg/services/sqlstore/dashboard_provisioning_test.go +++ b/pkg/services/sqlstore/dashboard_provisioning_test.go @@ -13,17 +13,30 @@ func TestDashboardProvisioningTest(t *testing.T) { Convey("Testing Dashboard provisioning", t, func() { InitTestDB(t) - saveDashboardCmd := &models.SaveDashboardCommand{ + folderCmd := &models.SaveDashboardCommand{ OrgId: 1, FolderId: 0, + IsFolder: true, + Dashboard: simplejson.NewFromAny(map[string]interface{}{ + "id": nil, + "title": "test dashboard", + }), + } + + err := SaveDashboard(folderCmd) + So(err, ShouldBeNil) + + saveDashboardCmd := &models.SaveDashboardCommand{ + OrgId: 1, IsFolder: false, + FolderId: folderCmd.Result.Id, Dashboard: simplejson.NewFromAny(map[string]interface{}{ "id": nil, "title": "test dashboard", }), } - Convey("Saving dashboards with extras", func() { + Convey("Saving dashboards with provisioning meta data", func() { now := time.Now() cmd := &models.SaveProvisionedDashboardCommand{ @@ -67,6 +80,21 @@ func TestDashboardProvisioningTest(t *testing.T) { So(err, ShouldBeNil) So(query.Result, ShouldBeFalse) }) + + Convey("Deleteing folder should delete provision meta data", func() { + deleteCmd := &models.DeleteDashboardCommand{ + Id: folderCmd.Result.Id, + OrgId: 1, + } + + So(DeleteDashboard(deleteCmd), ShouldBeNil) + + query := &models.IsDashboardProvisionedQuery{DashboardId: cmd.Result.Id} + + err = GetProvisionedDataByDashboardId(query) + So(err, ShouldBeNil) + So(query.Result, ShouldBeFalse) + }) }) }) } diff --git a/pkg/tsdb/stackdriver/stackdriver.go b/pkg/tsdb/stackdriver/stackdriver.go index 8b903ba01139..b33d33fb41ce 100644 --- a/pkg/tsdb/stackdriver/stackdriver.go +++ b/pkg/tsdb/stackdriver/stackdriver.go @@ -355,11 +355,21 @@ func (e *StackdriverExecutor) unmarshalResponse(res *http.Response) (Stackdriver func (e *StackdriverExecutor) parseResponse(queryRes *tsdb.QueryResult, data StackdriverResponse, query *StackdriverQuery) error { metricLabels := make(map[string][]string) resourceLabels := make(map[string][]string) + var resourceTypes []string + + for _, series := range data.TimeSeries { + if !containsLabel(resourceTypes, series.Resource.Type) { + resourceTypes = append(resourceTypes, series.Resource.Type) + } + } for _, series := range data.TimeSeries { points := make([]tsdb.TimePoint, 0) defaultMetricName := series.Metric.Type + if len(resourceTypes) > 1 { + defaultMetricName += " " + series.Resource.Type + } for key, value := range series.Metric.Labels { if !containsLabel(metricLabels[key], value) { @@ -403,7 +413,7 @@ func (e *StackdriverExecutor) parseResponse(queryRes *tsdb.QueryResult, data Sta points = append(points, tsdb.NewTimePoint(null.FloatFrom(value), float64((point.Interval.EndTime).Unix())*1000)) } - metricName := formatLegendKeys(series.Metric.Type, defaultMetricName, series.Metric.Labels, series.Resource.Labels, make(map[string]string), query) + metricName := formatLegendKeys(series.Metric.Type, defaultMetricName, series.Resource.Type, series.Metric.Labels, series.Resource.Labels, make(map[string]string), query) queryRes.Series = append(queryRes.Series, &tsdb.TimeSeries{ Name: metricName, @@ -429,7 +439,7 @@ func (e *StackdriverExecutor) parseResponse(queryRes *tsdb.QueryResult, data Sta bucketBound := calcBucketBound(point.Value.DistributionValue.BucketOptions, i) additionalLabels := map[string]string{"bucket": bucketBound} buckets[i] = &tsdb.TimeSeries{ - Name: formatLegendKeys(series.Metric.Type, defaultMetricName, series.Metric.Labels, series.Resource.Labels, additionalLabels, query), + Name: formatLegendKeys(series.Metric.Type, defaultMetricName, series.Resource.Type, series.Metric.Labels, series.Resource.Labels, additionalLabels, query), Points: make([]tsdb.TimePoint, 0), } if maxKey < i { @@ -445,7 +455,7 @@ func (e *StackdriverExecutor) parseResponse(queryRes *tsdb.QueryResult, data Sta bucketBound := calcBucketBound(point.Value.DistributionValue.BucketOptions, i) additionalLabels := map[string]string{"bucket": bucketBound} buckets[i] = &tsdb.TimeSeries{ - Name: formatLegendKeys(series.Metric.Type, defaultMetricName, series.Metric.Labels, series.Resource.Labels, additionalLabels, query), + Name: formatLegendKeys(series.Metric.Type, defaultMetricName, series.Resource.Type, series.Metric.Labels, series.Resource.Labels, additionalLabels, query), Points: make([]tsdb.TimePoint, 0), } } @@ -460,6 +470,7 @@ func (e *StackdriverExecutor) parseResponse(queryRes *tsdb.QueryResult, data Sta queryRes.Meta.Set("resourceLabels", resourceLabels) queryRes.Meta.Set("metricLabels", metricLabels) queryRes.Meta.Set("groupBys", query.GroupBys) + queryRes.Meta.Set("resourceTypes", resourceTypes) return nil } @@ -473,7 +484,7 @@ func containsLabel(labels []string, newLabel string) bool { return false } -func formatLegendKeys(metricType string, defaultMetricName string, metricLabels map[string]string, resourceLabels map[string]string, additionalLabels map[string]string, query *StackdriverQuery) string { +func formatLegendKeys(metricType string, defaultMetricName string, resourceType string, metricLabels map[string]string, resourceLabels map[string]string, additionalLabels map[string]string, query *StackdriverQuery) string { if query.AliasBy == "" { return defaultMetricName } @@ -487,6 +498,10 @@ func formatLegendKeys(metricType string, defaultMetricName string, metricLabels return []byte(metricType) } + if metaPartName == "resource.type" && resourceType != "" { + return []byte(resourceType) + } + metricPart := replaceWithMetricPart(metaPartName, metricType) if metricPart != nil { diff --git a/public/app/core/components/help/help.ts b/public/app/core/components/help/help.ts index eac47b6e0a29..8e8a5ed45d2f 100644 --- a/public/app/core/components/help/help.ts +++ b/public/app/core/components/help/help.ts @@ -34,6 +34,7 @@ export class HelpCtrl { { keys: ['p', 's'], description: 'Open Panel Share Modal' }, { keys: ['p', 'd'], description: 'Duplicate Panel' }, { keys: ['p', 'r'], description: 'Remove Panel' }, + { keys: ['p', 'l'], description: 'Toggle panel legend' }, ], 'Time Range': [ { keys: ['t', 'z'], description: 'Zoom out time range' }, diff --git a/public/app/core/services/keybindingSrv.ts b/public/app/core/services/keybindingSrv.ts index f43dc96cd378..6fe57dfa77ae 100644 --- a/public/app/core/services/keybindingSrv.ts +++ b/public/app/core/services/keybindingSrv.ts @@ -242,6 +242,18 @@ export class KeybindingSrv { } }); + // toggle panel legend + this.bind('p l', () => { + if (dashboard.meta.focusPanelId) { + const panelInfo = dashboard.getPanelInfoById(dashboard.meta.focusPanelId); + if (panelInfo.panel.legend) { + const panelRef = dashboard.getPanelById(dashboard.meta.focusPanelId); + panelRef.legend.show = !panelRef.legend.show; + panelRef.refresh(); + } + } + }); + // collapse all rows this.bind('d shift+c', () => { dashboard.collapseRows(); diff --git a/public/app/core/specs/table_model.test.ts b/public/app/core/specs/table_model.test.ts index 990daaaa2da4..19b2a7543fb9 100644 --- a/public/app/core/specs/table_model.test.ts +++ b/public/app/core/specs/table_model.test.ts @@ -1,4 +1,4 @@ -import TableModel from 'app/core/table_model'; +import TableModel, { mergeTablesIntoModel } from 'app/core/table_model'; describe('when sorting table desc', () => { let table; @@ -79,3 +79,118 @@ describe('when sorting with nulls', () => { expect(values).toEqual([null, null, 'd', 'c', 'b', 'a', '', '']); }); }); + +describe('mergeTables', () => { + const time = new Date().getTime(); + + const singleTable = new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Value' }], + rows: [[time, 'Label Value 1', 42]], + }); + + const multipleTablesSameColumns = [ + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Label Key 2' }, { text: 'Value #A' }], + rows: [[time, 'Label Value 1', 'Label Value 2', 42]], + }), + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Label Key 2' }, { text: 'Value #B' }], + rows: [[time, 'Label Value 1', 'Label Value 2', 13]], + }), + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Label Key 2' }, { text: 'Value #C' }], + rows: [[time, 'Label Value 1', 'Label Value 2', 4]], + }), + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Label Key 2' }, { text: 'Value #C' }], + rows: [[time, 'Label Value 1', 'Label Value 2', 7]], + }), + ]; + + const multipleTablesDifferentColumns = [ + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Value #A' }], + rows: [[time, 'Label Value 1', 42]], + }), + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 2' }, { text: 'Value #B' }], + rows: [[time, 'Label Value 2', 13]], + }), + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Value #C' }], + rows: [[time, 'Label Value 3', 7]], + }), + ]; + + it('should return the single table as is', () => { + const table = mergeTablesIntoModel(new TableModel(), singleTable); + expect(table.columns.length).toBe(3); + expect(table.columns[0].text).toBe('Time'); + expect(table.columns[1].text).toBe('Label Key 1'); + expect(table.columns[2].text).toBe('Value'); + }); + + it('should return the union of columns for multiple tables', () => { + const table = mergeTablesIntoModel(new TableModel(), ...multipleTablesSameColumns); + expect(table.columns.length).toBe(6); + expect(table.columns[0].text).toBe('Time'); + expect(table.columns[1].text).toBe('Label Key 1'); + expect(table.columns[2].text).toBe('Label Key 2'); + expect(table.columns[3].text).toBe('Value #A'); + expect(table.columns[4].text).toBe('Value #B'); + expect(table.columns[5].text).toBe('Value #C'); + }); + + it('should return 1 row for a single table', () => { + const table = mergeTablesIntoModel(new TableModel(), singleTable); + expect(table.rows.length).toBe(1); + expect(table.rows[0][0]).toBe(time); + expect(table.rows[0][1]).toBe('Label Value 1'); + expect(table.rows[0][2]).toBe(42); + }); + + it('should return 2 rows for a multiple tables with same column values plus one extra row', () => { + const table = mergeTablesIntoModel(new TableModel(), ...multipleTablesSameColumns); + expect(table.rows.length).toBe(2); + expect(table.rows[0][0]).toBe(time); + expect(table.rows[0][1]).toBe('Label Value 1'); + expect(table.rows[0][2]).toBe('Label Value 2'); + expect(table.rows[0][3]).toBe(42); + expect(table.rows[0][4]).toBe(13); + expect(table.rows[0][5]).toBe(4); + expect(table.rows[1][0]).toBe(time); + expect(table.rows[1][1]).toBe('Label Value 1'); + expect(table.rows[1][2]).toBe('Label Value 2'); + expect(table.rows[1][3]).toBeUndefined(); + expect(table.rows[1][4]).toBeUndefined(); + expect(table.rows[1][5]).toBe(7); + }); + + it('should return 2 rows for multiple tables with different column values', () => { + const table = mergeTablesIntoModel(new TableModel(), ...multipleTablesDifferentColumns); + expect(table.rows.length).toBe(2); + expect(table.columns.length).toBe(6); + + expect(table.rows[0][0]).toBe(time); + expect(table.rows[0][1]).toBe('Label Value 1'); + expect(table.rows[0][2]).toBe(42); + expect(table.rows[0][3]).toBe('Label Value 2'); + expect(table.rows[0][4]).toBe(13); + expect(table.rows[0][5]).toBeUndefined(); + + expect(table.rows[1][0]).toBe(time); + expect(table.rows[1][1]).toBe('Label Value 3'); + expect(table.rows[1][2]).toBeUndefined(); + expect(table.rows[1][3]).toBeUndefined(); + expect(table.rows[1][4]).toBeUndefined(); + expect(table.rows[1][5]).toBe(7); + }); +}); diff --git a/public/app/core/table_model.ts b/public/app/core/table_model.ts index f8b96d0537b0..99395258ba30 100644 --- a/public/app/core/table_model.ts +++ b/public/app/core/table_model.ts @@ -1,3 +1,5 @@ +import _ from 'lodash'; + interface Column { text: string; title?: string; @@ -14,11 +16,20 @@ export default class TableModel { type: string; columnMap: any; - constructor() { + constructor(table?: any) { this.columns = []; this.columnMap = {}; this.rows = []; this.type = 'table'; + + if (table) { + if (table.columns) { + table.columns.forEach(col => this.addColumn(col)); + } + if (table.rows) { + table.rows.forEach(row => this.addRow(row)); + } + } } sort(options) { @@ -52,3 +63,100 @@ export default class TableModel { this.rows.push(row); } } + +// Returns true if both rows have matching non-empty fields as well as matching +// indexes where one field is empty and the other is not +function areRowsMatching(columns, row, otherRow) { + let foundFieldToMatch = false; + for (let columnIndex = 0; columnIndex < columns.length; columnIndex++) { + if (row[columnIndex] !== undefined && otherRow[columnIndex] !== undefined) { + if (row[columnIndex] !== otherRow[columnIndex]) { + return false; + } + } else if (row[columnIndex] === undefined || otherRow[columnIndex] === undefined) { + foundFieldToMatch = true; + } + } + return foundFieldToMatch; +} + +export function mergeTablesIntoModel(dst?: TableModel, ...tables: TableModel[]): TableModel { + const model = dst || new TableModel(); + + // Single query returns data columns and rows as is + if (arguments.length === 2) { + model.columns = [...tables[0].columns]; + model.rows = [...tables[0].rows]; + return model; + } + + // Track column indexes of union: name -> index + const columnNames = {}; + + // Union of all non-value columns + const columnsUnion = tables.slice().reduce((acc, series) => { + series.columns.forEach(col => { + const { text } = col; + if (columnNames[text] === undefined) { + columnNames[text] = acc.length; + acc.push(col); + } + }); + return acc; + }, []); + + // Map old column index to union index per series, e.g., + // given columnNames {A: 0, B: 1} and + // data [{columns: [{ text: 'A' }]}, {columns: [{ text: 'B' }]}] => [[0], [1]] + const columnIndexMapper = tables.map(series => series.columns.map(col => columnNames[col.text])); + + // Flatten rows of all series and adjust new column indexes + const flattenedRows = tables.reduce((acc, series, seriesIndex) => { + const mapper = columnIndexMapper[seriesIndex]; + series.rows.forEach(row => { + const alteredRow = []; + // Shifting entries according to index mapper + mapper.forEach((to, from) => { + alteredRow[to] = row[from]; + }); + acc.push(alteredRow); + }); + return acc; + }, []); + + // Merge rows that have same values for columns + const mergedRows = {}; + const compactedRows = flattenedRows.reduce((acc, row, rowIndex) => { + if (!mergedRows[rowIndex]) { + // Look from current row onwards + let offset = rowIndex + 1; + // More than one row can be merged into current row + while (offset < flattenedRows.length) { + // Find next row that could be merged + const match = _.findIndex(flattenedRows, otherRow => areRowsMatching(columnsUnion, row, otherRow), offset); + if (match > -1) { + const matchedRow = flattenedRows[match]; + // Merge values from match into current row if there is a gap in the current row + for (let columnIndex = 0; columnIndex < columnsUnion.length; columnIndex++) { + if (row[columnIndex] === undefined && matchedRow[columnIndex] !== undefined) { + row[columnIndex] = matchedRow[columnIndex]; + } + } + // Don't visit this row again + mergedRows[match] = matchedRow; + // Keep looking for more rows to merge + offset = match + 1; + } else { + // No match found, stop looking + break; + } + } + acc.push(row); + } + return acc; + }, []); + + model.columns = columnsUnion; + model.rows = compactedRows; + return model; +} diff --git a/public/app/core/utils/explore.test.ts b/public/app/core/utils/explore.test.ts index 915b47e14e20..4252730338dd 100644 --- a/public/app/core/utils/explore.test.ts +++ b/public/app/core/utils/explore.test.ts @@ -8,23 +8,17 @@ const DEFAULT_EXPLORE_STATE: ExploreState = { datasourceMissing: false, datasourceName: '', exploreDatasources: [], - graphResult: null, + graphRange: DEFAULT_RANGE, history: [], - latency: 0, - loading: false, - logsResult: null, queries: [], - queryErrors: [], - queryHints: [], + queryTransactions: [], range: DEFAULT_RANGE, - requestOptions: null, showingGraph: true, showingLogs: true, showingTable: true, supportsGraph: null, supportsLogs: null, supportsTable: null, - tableResult: null, }; describe('state functions', () => { diff --git a/public/app/features/explore/Explore.tsx b/public/app/features/explore/Explore.tsx index 4fe67d9d37b8..bac063116f1b 100644 --- a/public/app/features/explore/Explore.tsx +++ b/public/app/features/explore/Explore.tsx @@ -1,8 +1,17 @@ import React from 'react'; import { hot } from 'react-hot-loader'; import Select from 'react-select'; - -import { ExploreState, ExploreUrlState, Query } from 'app/types/explore'; +import _ from 'lodash'; + +import { + ExploreState, + ExploreUrlState, + HistoryItem, + Query, + QueryTransaction, + Range, + ResultType, +} from 'app/types/explore'; import kbn from 'app/core/utils/kbn'; import colors from 'app/core/utils/colors'; import store from 'app/core/store'; @@ -13,8 +22,8 @@ import ResetStyles from 'app/core/components/Picker/ResetStyles'; import PickerOption from 'app/core/components/Picker/PickerOption'; import IndicatorsContainer from 'app/core/components/Picker/IndicatorsContainer'; import NoOptionsMessage from 'app/core/components/Picker/NoOptionsMessage'; +import TableModel, { mergeTablesIntoModel } from 'app/core/table_model'; -import ElapsedTime from './ElapsedTime'; import QueryRows from './QueryRows'; import Graph from './Graph'; import Logs from './Logs'; @@ -24,16 +33,6 @@ import { ensureQueries, generateQueryKey, hasQuery } from './utils/query'; const MAX_HISTORY_ITEMS = 100; -function makeHints(hints) { - const hintsByIndex = []; - hints.forEach(hint => { - if (hint) { - hintsByIndex[hint.index] = hint; - } - }); - return hintsByIndex; -} - function makeTimeSeriesList(dataList, options) { return dataList.map((seriesData, index) => { const datapoints = seriesData.datapoints || []; @@ -52,6 +51,25 @@ function makeTimeSeriesList(dataList, options) { }); } +/** + * Update the query history. Side-effect: store history in local storage + */ +function updateHistory(history: HistoryItem[], datasourceId: string, queries: string[]): HistoryItem[] { + const ts = Date.now(); + queries.forEach(query => { + history = [{ query, ts }, ...history]; + }); + + if (history.length > MAX_HISTORY_ITEMS) { + history = history.slice(0, MAX_HISTORY_ITEMS); + } + + // Combine all queries of a datasource type into one history + const historyKey = `grafana.explore.history.${datasourceId}`; + store.setObject(historyKey, history); + return history; +} + interface ExploreProps { datasourceSrv: any; onChangeSplit: (split: boolean, state?: ExploreState) => void; @@ -82,6 +100,7 @@ export class Explore extends React.PureComponent { } else { const { datasource, queries, range } = props.urlState as ExploreUrlState; initialQueries = ensureQueries(queries); + const initialRange = range || { ...DEFAULT_RANGE }; this.state = { datasource: null, datasourceError: null, @@ -89,23 +108,17 @@ export class Explore extends React.PureComponent { datasourceMissing: false, datasourceName: datasource, exploreDatasources: [], - graphResult: null, + graphRange: initialRange, history: [], - latency: 0, - loading: false, - logsResult: null, queries: initialQueries, - queryErrors: [], - queryHints: [], - range: range || { ...DEFAULT_RANGE }, - requestOptions: null, + queryTransactions: [], + range: initialRange, showingGraph: true, showingLogs: true, showingTable: true, supportsGraph: null, supportsLogs: null, supportsTable: null, - tableResult: null, }; } this.queryExpressions = initialQueries.map(q => q.query); @@ -199,14 +212,32 @@ export class Explore extends React.PureComponent { }; onAddQueryRow = index => { - const { queries } = this.state; + // Local cache this.queryExpressions[index + 1] = ''; - const nextQueries = [ - ...queries.slice(0, index + 1), - { query: '', key: generateQueryKey() }, - ...queries.slice(index + 1), - ]; - this.setState({ queries: nextQueries }); + + this.setState(state => { + const { queries, queryTransactions } = state; + + // Add row by generating new react key + const nextQueries = [ + ...queries.slice(0, index + 1), + { query: '', key: generateQueryKey() }, + ...queries.slice(index + 1), + ]; + + // Ongoing transactions need to update their row indices + const nextQueryTransactions = queryTransactions.map(qt => { + if (qt.rowIndex > index) { + return { + ...qt, + rowIndex: qt.rowIndex + 1, + }; + } + return qt; + }); + + return { queries: nextQueries, queryTransactions: nextQueryTransactions }; + }); }; onChangeDatasource = async option => { @@ -214,12 +245,7 @@ export class Explore extends React.PureComponent { datasource: null, datasourceError: null, datasourceLoading: true, - graphResult: null, - latency: 0, - logsResult: null, - queryErrors: [], - queryHints: [], - tableResult: null, + queryTransactions: [], }); const datasourceName = option.value; const datasource = await this.props.datasourceSrv.get(datasourceName); @@ -230,24 +256,25 @@ export class Explore extends React.PureComponent { // Keep current value in local cache this.queryExpressions[index] = value; - // Replace query row on override if (override) { - const { queries } = this.state; - const nextQuery: Query = { - key: generateQueryKey(index), - query: value, - }; - const nextQueries = [...queries]; - nextQueries[index] = nextQuery; - - this.setState( - { - queryErrors: [], - queryHints: [], + this.setState(state => { + // Replace query row + const { queries, queryTransactions } = state; + const nextQuery: Query = { + key: generateQueryKey(index), + query: value, + }; + const nextQueries = [...queries]; + nextQueries[index] = nextQuery; + + // Discard ongoing transaction related to row query + const nextQueryTransactions = queryTransactions.filter(qt => qt.rowIndex !== index); + + return { queries: nextQueries, - }, - this.onSubmit - ); + queryTransactions: nextQueryTransactions, + }; + }, this.onSubmit); } }; @@ -263,13 +290,8 @@ export class Explore extends React.PureComponent { this.queryExpressions = ['']; this.setState( { - graphResult: null, - logsResult: null, - latency: 0, queries: ensureQueries(), - queryErrors: [], - queryHints: [], - tableResult: null, + queryTransactions: [], }, this.saveState ); @@ -283,11 +305,41 @@ export class Explore extends React.PureComponent { }; onClickGraphButton = () => { - this.setState(state => ({ showingGraph: !state.showingGraph })); + this.setState( + state => { + const showingGraph = !state.showingGraph; + let nextQueryTransactions = state.queryTransactions; + if (!showingGraph) { + // Discard transactions related to Graph query + nextQueryTransactions = state.queryTransactions.filter(qt => qt.resultType !== 'Graph'); + } + return { queryTransactions: nextQueryTransactions, showingGraph }; + }, + () => { + if (this.state.showingGraph) { + this.onSubmit(); + } + } + ); }; onClickLogsButton = () => { - this.setState(state => ({ showingLogs: !state.showingLogs })); + this.setState( + state => { + const showingLogs = !state.showingLogs; + let nextQueryTransactions = state.queryTransactions; + if (!showingLogs) { + // Discard transactions related to Logs query + nextQueryTransactions = state.queryTransactions.filter(qt => qt.resultType !== 'Logs'); + } + return { queryTransactions: nextQueryTransactions, showingLogs }; + }, + () => { + if (this.state.showingLogs) { + this.onSubmit(); + } + } + ); }; onClickSplit = () => { @@ -299,7 +351,22 @@ export class Explore extends React.PureComponent { }; onClickTableButton = () => { - this.setState(state => ({ showingTable: !state.showingTable })); + this.setState( + state => { + const showingTable = !state.showingTable; + let nextQueryTransactions = state.queryTransactions; + if (!showingTable) { + // Discard transactions related to Table query + nextQueryTransactions = state.queryTransactions.filter(qt => qt.resultType !== 'Table'); + } + return { queryTransactions: nextQueryTransactions, showingTable }; + }, + () => { + if (this.state.showingTable) { + this.onSubmit(); + } + } + ); }; onClickTableCell = (columnKey: string, rowValue: string) => { @@ -307,39 +374,68 @@ export class Explore extends React.PureComponent { }; onModifyQueries = (action: object, index?: number) => { - const { datasource, queries } = this.state; + const { datasource } = this.state; if (datasource && datasource.modifyQuery) { - let nextQueries; - if (index === undefined) { - // Modify all queries - nextQueries = queries.map((q, i) => ({ - key: generateQueryKey(i), - query: datasource.modifyQuery(this.queryExpressions[i], action), - })); - } else { - // Modify query only at index - nextQueries = [ - ...queries.slice(0, index), - { - key: generateQueryKey(index), - query: datasource.modifyQuery(this.queryExpressions[index], action), - }, - ...queries.slice(index + 1), - ]; - } - this.queryExpressions = nextQueries.map(q => q.query); - this.setState({ queries: nextQueries }, () => this.onSubmit()); + this.setState( + state => { + const { queries, queryTransactions } = state; + let nextQueries; + let nextQueryTransactions; + if (index === undefined) { + // Modify all queries + nextQueries = queries.map((q, i) => ({ + key: generateQueryKey(i), + query: datasource.modifyQuery(this.queryExpressions[i], action), + })); + // Discard all ongoing transactions + nextQueryTransactions = []; + } else { + // Modify query only at index + nextQueries = [ + ...queries.slice(0, index), + { + key: generateQueryKey(index), + query: datasource.modifyQuery(this.queryExpressions[index], action), + }, + ...queries.slice(index + 1), + ]; + // Discard transactions related to row query + nextQueryTransactions = queryTransactions.filter(qt => qt.rowIndex !== index); + } + this.queryExpressions = nextQueries.map(q => q.query); + return { + queries: nextQueries, + queryTransactions: nextQueryTransactions, + }; + }, + () => this.onSubmit() + ); } }; onRemoveQueryRow = index => { - const { queries } = this.state; - if (queries.length <= 1) { - return; - } - const nextQueries = [...queries.slice(0, index), ...queries.slice(index + 1)]; - this.queryExpressions = nextQueries.map(q => q.query); - this.setState({ queries: nextQueries }, () => this.onSubmit()); + // Remove from local cache + this.queryExpressions = [...this.queryExpressions.slice(0, index), ...this.queryExpressions.slice(index + 1)]; + + this.setState( + state => { + const { queries, queryTransactions } = state; + if (queries.length <= 1) { + return null; + } + // Remove row from react state + const nextQueries = [...queries.slice(0, index), ...queries.slice(index + 1)]; + + // Discard transactions related to row query + const nextQueryTransactions = queryTransactions.filter(qt => qt.rowIndex !== index); + + return { + queries: nextQueries, + queryTransactions: nextQueryTransactions, + }; + }, + () => this.onSubmit() + ); }; onSubmit = () => { @@ -348,7 +444,7 @@ export class Explore extends React.PureComponent { this.runTableQuery(); } if (showingGraph && supportsGraph) { - this.runGraphQuery(); + this.runGraphQueries(); } if (showingLogs && supportsLogs) { this.runLogsQuery(); @@ -356,32 +452,11 @@ export class Explore extends React.PureComponent { this.saveState(); }; - onQuerySuccess(datasourceId: string, queries: string[]): void { - // save queries to history - let { history } = this.state; - const { datasource } = this.state; - - if (datasource.meta.id !== datasourceId) { - // Navigated away, queries did not matter - return; - } - - const ts = Date.now(); - queries.forEach(query => { - history = [{ query, ts }, ...history]; - }); - - if (history.length > MAX_HISTORY_ITEMS) { - history = history.slice(0, MAX_HISTORY_ITEMS); - } - - // Combine all queries of a datasource type into one history - const historyKey = `grafana.explore.history.${datasourceId}`; - store.setObject(historyKey, history); - this.setState({ history }); - } - - buildQueryOptions(targetOptions: { format: string; hinting?: boolean; instant?: boolean }) { + buildQueryOptions( + query: string, + rowIndex: number, + targetOptions: { format: string; hinting?: boolean; instant?: boolean } + ) { const { datasource, range } = this.state; const resolution = this.el.offsetWidth; const absoluteRange = { @@ -389,88 +464,235 @@ export class Explore extends React.PureComponent { to: parseDate(range.to, true), }; const { interval } = kbn.calculateInterval(absoluteRange, resolution, datasource.interval); - const targets = this.queryExpressions.map(q => ({ - ...targetOptions, - expr: q, - })); + const targets = [ + { + ...targetOptions, + // Target identifier is needed for table transformations + refId: rowIndex + 1, + expr: query, + }, + ]; + + // Clone range for query request + const queryRange: Range = { ...range }; + return { interval, - range, targets, + range: queryRange, }; } - async runGraphQuery() { + startQueryTransaction(query: string, rowIndex: number, resultType: ResultType, options: any): QueryTransaction { + const queryOptions = this.buildQueryOptions(query, rowIndex, options); + const transaction: QueryTransaction = { + query, + resultType, + rowIndex, + id: generateQueryKey(), + done: false, + latency: 0, + options: queryOptions, + }; + + // Using updater style because we might be modifying queryTransactions in quick succession + this.setState(state => { + const { queryTransactions } = state; + // Discarding existing transactions of same type + const remainingTransactions = queryTransactions.filter( + qt => !(qt.resultType === resultType && qt.rowIndex === rowIndex) + ); + + // Append new transaction + const nextQueryTransactions = [...remainingTransactions, transaction]; + + return { + queryTransactions: nextQueryTransactions, + }; + }); + + return transaction; + } + + completeQueryTransaction( + transactionId: string, + result: any, + latency: number, + queries: string[], + datasourceId: string + ) { const { datasource } = this.state; + if (datasource.meta.id !== datasourceId) { + // Navigated away, queries did not matter + return; + } + + this.setState(state => { + const { history, queryTransactions } = state; + + // Transaction might have been discarded + const transaction = queryTransactions.find(qt => qt.id === transactionId); + if (!transaction) { + return null; + } + + // Get query hints + let hints; + if (datasource.getQueryHints) { + hints = datasource.getQueryHints(transaction.query, result); + } + + // Mark transactions as complete + const nextQueryTransactions = queryTransactions.map(qt => { + if (qt.id === transactionId) { + return { + ...qt, + hints, + latency, + result, + done: true, + }; + } + return qt; + }); + + const nextHistory = updateHistory(history, datasourceId, queries); + + return { + history: nextHistory, + queryTransactions: nextQueryTransactions, + }; + }); + } + + discardTransactions(rowIndex: number) { + this.setState(state => { + const remainingTransactions = state.queryTransactions.filter(qt => qt.rowIndex !== rowIndex); + return { queryTransactions: remainingTransactions }; + }); + } + + failQueryTransaction(transactionId: string, error: string, datasourceId: string) { + const { datasource } = this.state; + if (datasource.meta.id !== datasourceId) { + // Navigated away, queries did not matter + return; + } + + this.setState(state => { + // Transaction might have been discarded + if (!state.queryTransactions.find(qt => qt.id === transactionId)) { + return null; + } + + // Mark transactions as complete + const nextQueryTransactions = state.queryTransactions.map(qt => { + if (qt.id === transactionId) { + return { + ...qt, + error, + done: true, + }; + } + return qt; + }); + + return { + queryTransactions: nextQueryTransactions, + }; + }); + } + + async runGraphQueries() { const queries = [...this.queryExpressions]; if (!hasQuery(queries)) { return; } - this.setState({ latency: 0, loading: true, graphResult: null, queryErrors: [], queryHints: [] }); - const now = Date.now(); - const options = this.buildQueryOptions({ format: 'time_series', instant: false, hinting: true }); - try { - const res = await datasource.query(options); - const result = makeTimeSeriesList(res.data, options); - const queryHints = res.hints ? makeHints(res.hints) : []; - const latency = Date.now() - now; - this.setState({ latency, loading: false, graphResult: result, queryHints, requestOptions: options }); - this.onQuerySuccess(datasource.meta.id, queries); - } catch (response) { - console.error(response); - const queryError = response.data ? response.data.error : response; - this.setState({ loading: false, queryErrors: [queryError] }); - } + const { datasource } = this.state; + const datasourceId = datasource.meta.id; + // Run all queries concurrently + queries.forEach(async (query, rowIndex) => { + if (query) { + const transaction = this.startQueryTransaction(query, rowIndex, 'Graph', { + format: 'time_series', + instant: false, + }); + try { + const now = Date.now(); + const res = await datasource.query(transaction.options); + const latency = Date.now() - now; + const results = makeTimeSeriesList(res.data, transaction.options); + this.completeQueryTransaction(transaction.id, results, latency, queries, datasourceId); + this.setState({ graphRange: transaction.options.range }); + } catch (response) { + console.error(response); + const queryError = response.data ? response.data.error : response; + this.failQueryTransaction(transaction.id, queryError, datasourceId); + } + } else { + this.discardTransactions(rowIndex); + } + }); } async runTableQuery() { const queries = [...this.queryExpressions]; - const { datasource } = this.state; if (!hasQuery(queries)) { return; } - this.setState({ latency: 0, loading: true, queryErrors: [], queryHints: [], tableResult: null }); - const now = Date.now(); - const options = this.buildQueryOptions({ - format: 'table', - instant: true, + const { datasource } = this.state; + const datasourceId = datasource.meta.id; + // Run all queries concurrently + queries.forEach(async (query, rowIndex) => { + if (query) { + const transaction = this.startQueryTransaction(query, rowIndex, 'Table', { + format: 'table', + instant: true, + valueWithRefId: true, + }); + try { + const now = Date.now(); + const res = await datasource.query(transaction.options); + const latency = Date.now() - now; + const results = res.data[0]; + this.completeQueryTransaction(transaction.id, results, latency, queries, datasourceId); + } catch (response) { + console.error(response); + const queryError = response.data ? response.data.error : response; + this.failQueryTransaction(transaction.id, queryError, datasourceId); + } + } else { + this.discardTransactions(rowIndex); + } }); - try { - const res = await datasource.query(options); - const tableModel = res.data[0]; - const latency = Date.now() - now; - this.setState({ latency, loading: false, tableResult: tableModel, requestOptions: options }); - this.onQuerySuccess(datasource.meta.id, queries); - } catch (response) { - console.error(response); - const queryError = response.data ? response.data.error : response; - this.setState({ loading: false, queryErrors: [queryError] }); - } } async runLogsQuery() { const queries = [...this.queryExpressions]; - const { datasource } = this.state; if (!hasQuery(queries)) { return; } - this.setState({ latency: 0, loading: true, queryErrors: [], queryHints: [], logsResult: null }); - const now = Date.now(); - const options = this.buildQueryOptions({ - format: 'logs', + const { datasource } = this.state; + const datasourceId = datasource.meta.id; + // Run all queries concurrently + queries.forEach(async (query, rowIndex) => { + if (query) { + const transaction = this.startQueryTransaction(query, rowIndex, 'Logs', { format: 'logs' }); + try { + const now = Date.now(); + const res = await datasource.query(transaction.options); + const latency = Date.now() - now; + const results = res.data; + this.completeQueryTransaction(transaction.id, results, latency, queries, datasourceId); + } catch (response) { + console.error(response); + const queryError = response.data ? response.data.error : response; + this.failQueryTransaction(transaction.id, queryError, datasourceId); + } + } else { + this.discardTransactions(rowIndex); + } }); - - try { - const res = await datasource.query(options); - const logsData = res.data; - const latency = Date.now() - now; - this.setState({ latency, loading: false, logsResult: logsData, requestOptions: options }); - this.onQuerySuccess(datasource.meta.id, queries); - } catch (response) { - console.error(response); - const queryError = response.data ? response.data.error : response; - this.setState({ loading: false, queryErrors: [queryError] }); - } } request = url => { @@ -482,6 +704,7 @@ export class Explore extends React.PureComponent { // Copy state, but copy queries including modifications return { ...this.state, + queryTransactions: [], queries: ensureQueries(this.queryExpressions.map(query => ({ query }))), }; } @@ -499,23 +722,17 @@ export class Explore extends React.PureComponent { datasourceLoading, datasourceMissing, exploreDatasources, - graphResult, + graphRange, history, - latency, - loading, - logsResult, queries, - queryErrors, - queryHints, + queryTransactions, range, - requestOptions, showingGraph, showingLogs, showingTable, supportsGraph, supportsLogs, supportsTable, - tableResult, } = this.state; const showingBoth = showingGraph && showingTable; const graphHeight = showingBoth ? '200px' : '400px'; @@ -524,6 +741,20 @@ export class Explore extends React.PureComponent { const tableButtonActive = showingBoth || showingTable ? 'active' : ''; const exploreClass = split ? 'explore explore-split' : 'explore'; const selectedDatasource = datasource ? exploreDatasources.find(d => d.label === datasource.name) : undefined; + const graphLoading = queryTransactions.some(qt => qt.resultType === 'Graph' && !qt.done); + const tableLoading = queryTransactions.some(qt => qt.resultType === 'Table' && !qt.done); + const logsLoading = queryTransactions.some(qt => qt.resultType === 'Logs' && !qt.done); + const graphResult = _.flatten( + queryTransactions.filter(qt => qt.resultType === 'Graph' && qt.done && qt.result).map(qt => qt.result) + ); + const tableResult = mergeTablesIntoModel( + new TableModel(), + ...queryTransactions.filter(qt => qt.resultType === 'Table' && qt.done).map(qt => qt.result) + ); + const logsResult = _.flatten( + queryTransactions.filter(qt => qt.resultType === 'Logs' && qt.done).map(qt => qt.result) + ); + const loading = queryTransactions.some(qt => !qt.done); return (
@@ -581,9 +812,9 @@ export class Explore extends React.PureComponent {
- {loading || latency ? : null}
@@ -602,8 +833,6 @@ export class Explore extends React.PureComponent { { onExecuteQuery={this.onSubmit} onRemoveQueryRow={this.onRemoveQueryRow} supportsLogs={supportsLogs} + transactions={queryTransactions} />
{supportsGraph ? ( @@ -632,23 +862,22 @@ export class Explore extends React.PureComponent {
{supportsGraph && - showingGraph && - graphResult && ( + showingGraph && ( )} {supportsTable && showingTable ? ( -
- +
+
) : null} - {supportsLogs && showingLogs ? : null} + {supportsLogs && showingLogs ? : null} ) : null} diff --git a/public/app/features/explore/Graph.test.tsx b/public/app/features/explore/Graph.test.tsx index 043b843f0a51..fe4deaf17aa4 100644 --- a/public/app/features/explore/Graph.test.tsx +++ b/public/app/features/explore/Graph.test.tsx @@ -4,24 +4,11 @@ import { Graph } from './Graph'; import { mockData } from './__mocks__/mockData'; const setup = (propOverrides?: object) => { - const props = Object.assign( - { - data: mockData().slice(0, 19), - options: { - interval: '20s', - range: { from: 'now-6h', to: 'now' }, - targets: [ - { - format: 'time_series', - instant: false, - hinting: true, - expr: 'prometheus_http_request_duration_seconds_bucket', - }, - ], - }, - }, - propOverrides - ); + const props = { + data: mockData().slice(0, 19), + range: { from: 'now-6h', to: 'now' }, + ...propOverrides, + }; // Enzyme.shallow did not work well with jquery.flop. Mocking the draw function. Graph.prototype.draw = jest.fn(); diff --git a/public/app/features/explore/Graph.tsx b/public/app/features/explore/Graph.tsx index cb5def07faba..d57f8d49a438 100644 --- a/public/app/features/explore/Graph.tsx +++ b/public/app/features/explore/Graph.tsx @@ -5,6 +5,8 @@ import { withSize } from 'react-sizeme'; import 'vendor/flot/jquery.flot'; import 'vendor/flot/jquery.flot.time'; + +import { Range } from 'app/types/explore'; import * as dateMath from 'app/core/utils/datemath'; import TimeSeries from 'app/core/time_series2'; @@ -74,7 +76,7 @@ interface GraphProps { height?: string; // e.g., '200px' id?: string; loading?: boolean; - options: any; + range: Range; split?: boolean; size?: { width: number; height: number }; } @@ -101,7 +103,7 @@ export class Graph extends PureComponent { componentDidUpdate(prevProps: GraphProps) { if ( prevProps.data !== this.props.data || - prevProps.options !== this.props.options || + prevProps.range !== this.props.range || prevProps.split !== this.props.split || prevProps.height !== this.props.height || (prevProps.size && prevProps.size.width !== this.props.size.width) @@ -120,22 +122,22 @@ export class Graph extends PureComponent { }; draw() { - const { options: userOptions, size } = this.props; + const { range, size } = this.props; const data = this.getGraphData(); const $el = $(`#${this.props.id}`); - if (!data) { - $el.empty(); - return; + let series = [{ data: [[0, 0]] }]; + + if (data && data.length > 0) { + series = data.map((ts: TimeSeries) => ({ + color: ts.color, + label: ts.label, + data: ts.getFlotPairs('null'), + })); } - const series = data.map((ts: TimeSeries) => ({ - color: ts.color, - label: ts.label, - data: ts.getFlotPairs('null'), - })); const ticks = (size.width || 0) / 100; - let { from, to } = userOptions.range; + let { from, to } = range; if (!moment.isMoment(from)) { from = dateMath.parse(from, false); } @@ -157,7 +159,6 @@ export class Graph extends PureComponent { const options = { ...FLOT_OPTIONS, ...dynamicOptions, - ...userOptions, }; $.plot($el, series, options); } @@ -166,16 +167,11 @@ export class Graph extends PureComponent { const { height = '100px', id = 'graph', loading = false } = this.props; const data = this.getGraphData(); - if (!loading && data.length === 0) { - return ( -
-
The queries returned no time series to graph.
-
- ); - } return ( -
- {this.props.data.length > MAX_NUMBER_OF_TIME_SERIES && +
+ {loading &&
} + {this.props.data && + this.props.data.length > MAX_NUMBER_OF_TIME_SERIES && !this.state.showAllTimeSeries && (
@@ -185,10 +181,8 @@ export class Graph extends PureComponent { }`}
)} -
-
- -
+
+
); } diff --git a/public/app/features/explore/QueryField.tsx b/public/app/features/explore/QueryField.tsx index c3c41b7ab17f..ce0bcd71ed0e 100644 --- a/public/app/features/explore/QueryField.tsx +++ b/public/app/features/explore/QueryField.tsx @@ -198,7 +198,7 @@ class QueryField extends React.PureComponent { diff --git a/public/app/features/explore/QueryRows.tsx b/public/app/features/explore/QueryRows.tsx index 3d71c2f35667..0b0d7085d2dc 100644 --- a/public/app/features/explore/QueryRows.tsx +++ b/public/app/features/explore/QueryRows.tsx @@ -1,7 +1,18 @@ import React, { PureComponent } from 'react'; +import { QueryTransaction } from 'app/types/explore'; + // TODO make this datasource-plugin-dependent import QueryField from './PromQueryField'; +import QueryTransactions from './QueryTransactions'; + +function getFirstHintFromTransactions(transactions: QueryTransaction[]) { + const transaction = transactions.find(qt => qt.hints && qt.hints.length > 0); + if (transaction) { + return transaction.hints[0]; + } + return undefined; +} class QueryRow extends PureComponent { onChangeQuery = (value, override?: boolean) => { @@ -44,13 +55,19 @@ class QueryRow extends PureComponent { }; render() { - const { history, query, queryError, queryHint, request, supportsLogs } = this.props; + const { history, query, request, supportsLogs, transactions } = this.props; + const transactionWithError = transactions.find(t => t.error); + const hint = getFirstHintFromTransactions(transactions); + const queryError = transactionWithError ? transactionWithError.error : null; return (
+
+ +
{ export default class QueryRows extends PureComponent { render() { - const { className = '', queries, queryErrors, queryHints, ...handlers } = this.props; + const { className = '', queries, queryHints, transactions, ...handlers } = this.props; return (
{queries.map((q, index) => ( @@ -86,8 +103,7 @@ export default class QueryRows extends PureComponent { key={q.key} index={index} query={q.query} - queryError={queryErrors[index]} - queryHint={queryHints[index]} + transactions={transactions.filter(t => t.rowIndex === index)} {...handlers} /> ))} diff --git a/public/app/features/explore/QueryTransactions.tsx b/public/app/features/explore/QueryTransactions.tsx new file mode 100644 index 000000000000..0ce721f14e71 --- /dev/null +++ b/public/app/features/explore/QueryTransactions.tsx @@ -0,0 +1,42 @@ +import React, { PureComponent } from 'react'; + +import { QueryTransaction as QueryTransactionModel } from 'app/types/explore'; +import ElapsedTime from './ElapsedTime'; + +function formatLatency(value) { + return `${(value / 1000).toFixed(1)}s`; +} + +interface QueryTransactionProps { + transaction: QueryTransactionModel; +} + +class QueryTransaction extends PureComponent { + render() { + const { transaction } = this.props; + const className = transaction.done ? 'query-transaction' : 'query-transaction query-transaction--loading'; + return ( +
+
{transaction.resultType}:
+
+ {transaction.done ? formatLatency(transaction.latency) : } +
+
+ ); + } +} + +interface QueryTransactionsProps { + transactions: QueryTransactionModel[]; +} + +export default class QueryTransactions extends PureComponent { + render() { + const { transactions } = this.props; + return ( +
+ {transactions.map((t, i) => )} +
+ ); + } +} diff --git a/public/app/features/explore/Table.tsx b/public/app/features/explore/Table.tsx index e1c71fa55e42..0264bd3b4ccc 100644 --- a/public/app/features/explore/Table.tsx +++ b/public/app/features/explore/Table.tsx @@ -5,6 +5,8 @@ import ReactTable from 'react-table'; import TableModel from 'app/core/table_model'; const EMPTY_TABLE = new TableModel(); +// Identify columns that contain values +const VALUE_REGEX = /^[Vv]alue #\d+/; interface TableProps { data: TableModel; @@ -34,6 +36,7 @@ export default class Table extends PureComponent { const columns = tableModel.columns.map(({ filterable, text }) => ({ Header: text, accessor: text, + className: VALUE_REGEX.test(text) ? 'text-right' : '', show: text !== 'Time', Cell: row => {row.value}, })); @@ -48,7 +51,7 @@ export default class Table extends PureComponent { minRows={0} noDataText={noDataText} resolveData={data => prepareRows(data, columnNames)} - showPagination={data} + showPagination={Boolean(data)} /> ); } diff --git a/public/app/features/explore/__snapshots__/Graph.test.tsx.snap b/public/app/features/explore/__snapshots__/Graph.test.tsx.snap index d6760dff59cd..fd2010a76d31 100644 --- a/public/app/features/explore/__snapshots__/Graph.test.tsx.snap +++ b/public/app/features/explore/__snapshots__/Graph.test.tsx.snap @@ -1,468 +1,468 @@ // Jest Snapshot v1, https://goo.gl/fbAQLP exports[`Render should render component 1`] = ` -
+
-
- -
+ } + /> +
`; exports[`Render should render component with disclaimer 1`] = ` -
+
@@ -478,484 +478,480 @@ exports[`Render should render component with disclaimer 1`] = `
-
- -
+ } + /> +
`; @@ -964,9 +960,16 @@ exports[`Render should show query return no time series 1`] = ` className="panel-container" >
- The queries returned no time series to graph. -
+ className="explore-graph" + id="graph" + style={ + Object { + "height": "100px", + } + } + /> +
`; diff --git a/public/app/features/templating/partials/editor.html b/public/app/features/templating/partials/editor.html index ac4450c20a2a..c44639721770 100644 --- a/public/app/features/templating/partials/editor.html +++ b/public/app/features/templating/partials/editor.html @@ -115,7 +115,7 @@
Interval Options
Values - +
diff --git a/public/app/features/templating/specs/template_srv.test.ts b/public/app/features/templating/specs/template_srv.test.ts index 7f5ff959216a..7805341d1a26 100644 --- a/public/app/features/templating/specs/template_srv.test.ts +++ b/public/app/features/templating/specs/template_srv.test.ts @@ -286,10 +286,40 @@ describe('templateSrv', () => { initTemplateSrv([{ type: 'query', name: 'test', current: { value: 'oogle' } }]); }); - it('should return true if exists', () => { + it('should return true if $test exists', () => { const result = _templateSrv.variableExists('$test'); expect(result).toBe(true); }); + + it('should return true if $test exists in string', () => { + const result = _templateSrv.variableExists('something $test something'); + expect(result).toBe(true); + }); + + it('should return true if [[test]] exists in string', () => { + const result = _templateSrv.variableExists('something [[test]] something'); + expect(result).toBe(true); + }); + + it('should return true if [[test:csv]] exists in string', () => { + const result = _templateSrv.variableExists('something [[test:csv]] something'); + expect(result).toBe(true); + }); + + it('should return true if ${test} exists in string', () => { + const result = _templateSrv.variableExists('something ${test} something'); + expect(result).toBe(true); + }); + + it('should return true if ${test:raw} exists in string', () => { + const result = _templateSrv.variableExists('something ${test:raw} something'); + expect(result).toBe(true); + }); + + it('should return null if there are no variables in string', () => { + const result = _templateSrv.variableExists('string without variables'); + expect(result).toBe(null); + }); }); describe('can highlight variables in string', () => { @@ -429,6 +459,11 @@ describe('templateSrv', () => { name: 'period', current: { value: '$__auto_interval_interval', text: 'auto' }, }, + { + type: 'textbox', + name: 'empty_on_init', + current: { value: '', text: '' }, + }, ]); _templateSrv.setGrafanaVariable('$__auto_interval_interval', '13m'); _templateSrv.updateTemplateData(); @@ -438,6 +473,11 @@ describe('templateSrv', () => { const target = _templateSrv.replaceWithText('Server: $server, period: $period'); expect(target).toBe('Server: All, period: 13m'); }); + + it('should replace empty string-values with an empty string', () => { + const target = _templateSrv.replaceWithText('Hello $empty_on_init'); + expect(target).toBe('Hello '); + }); }); describe('built in interval variables', () => { diff --git a/public/app/features/templating/specs/variable.test.ts b/public/app/features/templating/specs/variable.test.ts index 6d5e88fa4bdb..83f4af8bca9f 100644 --- a/public/app/features/templating/specs/variable.test.ts +++ b/public/app/features/templating/specs/variable.test.ts @@ -22,6 +22,11 @@ describe('containsVariable', () => { expect(contains).toBe(true); }); + it('should find it with [[var:option]] syntax', () => { + const contains = containsVariable('this.[[test:csv]].filters', 'test'); + expect(contains).toBe(true); + }); + it('should find it when part of segment', () => { const contains = containsVariable('metrics.$env.$group-*', 'group'); expect(contains).toBe(true); @@ -36,6 +41,16 @@ describe('containsVariable', () => { const contains = containsVariable('asd', 'asd2.$env', 'env'); expect(contains).toBe(true); }); + + it('should find it with ${var} syntax', () => { + const contains = containsVariable('this.${test}.filters', 'test'); + expect(contains).toBe(true); + }); + + it('should find it with ${var:option} syntax', () => { + const contains = containsVariable('this.${test:csv}.filters', 'test'); + expect(contains).toBe(true); + }); }); }); diff --git a/public/app/features/templating/template_srv.ts b/public/app/features/templating/template_srv.ts index 70fd287402fe..0db7b8e77e05 100644 --- a/public/app/features/templating/template_srv.ts +++ b/public/app/features/templating/template_srv.ts @@ -1,5 +1,6 @@ import kbn from 'app/core/utils/kbn'; import _ from 'lodash'; +import { variableRegex } from 'app/features/templating/variable'; function luceneEscape(value) { return value.replace(/([\!\*\+\-\=<>\s\&\|\(\)\[\]\{\}\^\~\?\:\\/"])/g, '\\$1'); @@ -8,13 +9,7 @@ function luceneEscape(value) { export class TemplateSrv { variables: any[]; - /* - * This regex matches 3 types of variable reference with an optional format specifier - * \$(\w+) $var1 - * \[\[([\s\S]+?)(?::(\w+))?\]\] [[var2]] or [[var2:fmt2]] - * \${(\w+)(?::(\w+))?} ${var3} or ${var3:fmt3} - */ - private regex = /\$(\w+)|\[\[([\s\S]+?)(?::(\w+))?\]\]|\${(\w+)(?::(\w+))?}/g; + private regex = variableRegex; private index = {}; private grafanaVariables = {}; private builtIns = {}; @@ -30,17 +25,14 @@ export class TemplateSrv { } updateTemplateData() { - this.index = {}; + const existsOrEmpty = value => value || value === ''; - for (let i = 0; i < this.variables.length; i++) { - const variable = this.variables[i]; - - if (!variable.current || (!variable.current.isNone && !variable.current.value)) { - continue; + this.index = this.variables.reduce((acc, currentValue) => { + if (currentValue.current && !currentValue.current.isNone && existsOrEmpty(currentValue.current.value)) { + acc[currentValue.name] = currentValue; } - - this.index[variable.name] = variable; - } + return acc; + }, {}); } variableInitialized(variable) { @@ -144,7 +136,8 @@ export class TemplateSrv { if (!match) { return null; } - return match[1] || match[2]; + const variableName = match.slice(1).find(match => match !== undefined); + return variableName; } variableExists(expression) { diff --git a/public/app/features/templating/variable.ts b/public/app/features/templating/variable.ts index 412426fb2946..1994e86eff0f 100644 --- a/public/app/features/templating/variable.ts +++ b/public/app/features/templating/variable.ts @@ -1,6 +1,19 @@ -import kbn from 'app/core/utils/kbn'; import { assignModelProperties } from 'app/core/utils/model_utils'; +/* + * This regex matches 3 types of variable reference with an optional format specifier + * \$(\w+) $var1 + * \[\[([\s\S]+?)(?::(\w+))?\]\] [[var2]] or [[var2:fmt2]] + * \${(\w+)(?::(\w+))?} ${var3} or ${var3:fmt3} + */ +export const variableRegex = /\$(\w+)|\[\[([\s\S]+?)(?::(\w+))?\]\]|\${(\w+)(?::(\w+))?}/g; + +// Helper function since lastIndex is not reset +export const variableRegexExec = (variableString: string) => { + variableRegex.lastIndex = 0; + return variableRegex.exec(variableString); +}; + export interface Variable { setValue(option); updateOptions(); @@ -14,15 +27,16 @@ export let variableTypes = {}; export { assignModelProperties }; export function containsVariable(...args: any[]) { - let variableName = args[args.length - 1]; - let str = args[0] || ''; - - for (let i = 1; i < args.length - 1; i++) { - str += ' ' + args[i] || ''; - } + const variableName = args[args.length - 1]; + const variableString = args.slice(0, -1).join(' '); + const matches = variableString.match(variableRegex); + const isMatchingVariable = + matches !== null + ? matches.find(match => { + const varMatch = variableRegexExec(match); + return varMatch !== null && varMatch.indexOf(variableName) > -1; + }) + : false; - variableName = kbn.regexEscape(variableName); - const findVarRegex = new RegExp('\\$(' + variableName + ')(?:\\W|$)|\\[\\[(' + variableName + ')\\]\\]', 'g'); - const match = findVarRegex.exec(str); - return match !== null; + return !!isMatchingVariable; } diff --git a/public/app/plugins/datasource/prometheus/datasource.ts b/public/app/plugins/datasource/prometheus/datasource.ts index 856ab035ea05..89f88a946c23 100644 --- a/public/app/plugins/datasource/prometheus/datasource.ts +++ b/public/app/plugins/datasource/prometheus/datasource.ts @@ -176,7 +176,6 @@ export class PrometheusDatasource { return this.$q.all(allQueryPromise).then(responseList => { let result = []; - let hints = []; _.each(responseList, (response, index) => { if (response.status === 'error') { @@ -196,19 +195,14 @@ export class PrometheusDatasource { end: queries[index].end, query: queries[index].expr, responseListLength: responseList.length, - responseIndex: index, refId: activeTargets[index].refId, + valueWithRefId: activeTargets[index].valueWithRefId, }; const series = this.resultTransformer.transform(response, transformerOptions); result = [...result, ...series]; - - if (queries[index].hinting) { - const queryHints = getQueryHints(series, this); - hints = [...hints, ...queryHints]; - } }); - return { data: result, hints }; + return { data: result }; }); } @@ -437,6 +431,10 @@ export class PrometheusDatasource { return state; } + getQueryHints(query: string, result: any[]) { + return getQueryHints(query, result, this); + } + loadRules() { this.metadataRequest('/api/v1/rules') .then(res => res.data || res.json()) diff --git a/public/app/plugins/datasource/prometheus/query_hints.ts b/public/app/plugins/datasource/prometheus/query_hints.ts index 5efaadb77be1..cfd04c766ba0 100644 --- a/public/app/plugins/datasource/prometheus/query_hints.ts +++ b/public/app/plugins/datasource/prometheus/query_hints.ts @@ -1,100 +1,92 @@ import _ from 'lodash'; -export function getQueryHints(series: any[], datasource?: any): any[] { - const hints = series.map((s, i) => { - const query: string = s.query; - const index: number = s.responseIndex; - if (query === undefined || index === undefined) { - return null; - } +export function getQueryHints(query: string, series?: any[], datasource?: any): any[] { + const hints = []; - // ..._bucket metric needs a histogram_quantile() - const histogramMetric = query.trim().match(/^\w+_bucket$/); - if (histogramMetric) { - const label = 'Time series has buckets, you probably wanted a histogram.'; - return { - index, - label, - fix: { - label: 'Fix by adding histogram_quantile().', - action: { - type: 'ADD_HISTOGRAM_QUANTILE', - query, - index, - }, + // ..._bucket metric needs a histogram_quantile() + const histogramMetric = query.trim().match(/^\w+_bucket$/); + if (histogramMetric) { + const label = 'Time series has buckets, you probably wanted a histogram.'; + hints.push({ + type: 'HISTOGRAM_QUANTILE', + label, + fix: { + label: 'Fix by adding histogram_quantile().', + action: { + type: 'ADD_HISTOGRAM_QUANTILE', + query, }, - }; - } + }, + }); + } - // Check for monotony - const datapoints: number[][] = s.datapoints; - if (query.indexOf('rate(') === -1 && datapoints.length > 1) { - let increasing = false; - const nonNullData = datapoints.filter(dp => dp[0] !== null); - const monotonic = nonNullData.every((dp, index) => { - if (index === 0) { - return true; - } - increasing = increasing || dp[0] > nonNullData[index - 1][0]; - // monotonic? - return dp[0] >= nonNullData[index - 1][0]; - }); - if (increasing && monotonic) { - const simpleMetric = query.trim().match(/^\w+$/); - let label = 'Time series is monotonously increasing.'; - let fix; - if (simpleMetric) { - fix = { - label: 'Fix by adding rate().', - action: { - type: 'ADD_RATE', - query, - index, - }, - }; - } else { - label = `${label} Try applying a rate() function.`; + // Check for monotony on series (table results are being ignored here) + if (series && series.length > 0) { + series.forEach(s => { + const datapoints: number[][] = s.datapoints; + if (query.indexOf('rate(') === -1 && datapoints.length > 1) { + let increasing = false; + const nonNullData = datapoints.filter(dp => dp[0] !== null); + const monotonic = nonNullData.every((dp, index) => { + if (index === 0) { + return true; + } + increasing = increasing || dp[0] > nonNullData[index - 1][0]; + // monotonic? + return dp[0] >= nonNullData[index - 1][0]; + }); + if (increasing && monotonic) { + const simpleMetric = query.trim().match(/^\w+$/); + let label = 'Time series is monotonously increasing.'; + let fix; + if (simpleMetric) { + fix = { + label: 'Fix by adding rate().', + action: { + type: 'ADD_RATE', + query, + }, + }; + } else { + label = `${label} Try applying a rate() function.`; + } + hints.push({ + type: 'APPLY_RATE', + label, + fix, + }); } - return { - label, - index, - fix, - }; } - } + }); + } - // Check for recording rules expansion - if (datasource && datasource.ruleMappings) { - const mapping = datasource.ruleMappings; - const mappingForQuery = Object.keys(mapping).reduce((acc, ruleName) => { - if (query.search(ruleName) > -1) { - return { - ...acc, - [ruleName]: mapping[ruleName], - }; - } - return acc; - }, {}); - if (_.size(mappingForQuery) > 0) { - const label = 'Query contains recording rules.'; + // Check for recording rules expansion + if (datasource && datasource.ruleMappings) { + const mapping = datasource.ruleMappings; + const mappingForQuery = Object.keys(mapping).reduce((acc, ruleName) => { + if (query.search(ruleName) > -1) { return { - label, - index, - fix: { - label: 'Expand rules', - action: { - type: 'EXPAND_RULES', - query, - index, - mapping: mappingForQuery, - }, - }, + ...acc, + [ruleName]: mapping[ruleName], }; } + return acc; + }, {}); + if (_.size(mappingForQuery) > 0) { + const label = 'Query contains recording rules.'; + hints.push({ + type: 'EXPAND_RULES', + label, + fix: { + label: 'Expand rules', + action: { + type: 'EXPAND_RULES', + query, + mapping: mappingForQuery, + }, + }, + }); } - - // No hint found - return null; - }); - return hints; + } + return hints.length > 0 ? hints : null; } diff --git a/public/app/plugins/datasource/prometheus/result_transformer.ts b/public/app/plugins/datasource/prometheus/result_transformer.ts index bf916bebf04a..c9693eaf6572 100644 --- a/public/app/plugins/datasource/prometheus/result_transformer.ts +++ b/public/app/plugins/datasource/prometheus/result_transformer.ts @@ -8,7 +8,14 @@ export class ResultTransformer { const prometheusResult = response.data.data.result; if (options.format === 'table') { - return [this.transformMetricDataToTable(prometheusResult, options.responseListLength, options.refId)]; + return [ + this.transformMetricDataToTable( + prometheusResult, + options.responseListLength, + options.refId, + options.valueWithRefId + ), + ]; } else if (options.format === 'heatmap') { let seriesList = []; prometheusResult.sort(sortSeriesByLabel); @@ -66,12 +73,11 @@ export class ResultTransformer { return { datapoints: dps, query: options.query, - responseIndex: options.responseIndex, target: metricLabel, }; } - transformMetricDataToTable(md, resultCount: number, refId: string) { + transformMetricDataToTable(md, resultCount: number, refId: string, valueWithRefId?: boolean) { const table = new TableModel(); let i, j; const metricLabels = {}; @@ -96,7 +102,7 @@ export class ResultTransformer { metricLabels[label] = labelIndex + 1; table.columns.push({ text: label, filterable: !label.startsWith('__') }); }); - const valueText = resultCount > 1 ? `Value #${refId}` : 'Value'; + const valueText = resultCount > 1 || valueWithRefId ? `Value #${refId}` : 'Value'; table.columns.push({ text: valueText }); // Populate rows, set value to empty string when label not present. diff --git a/public/app/plugins/datasource/prometheus/specs/query_hints.test.ts b/public/app/plugins/datasource/prometheus/specs/query_hints.test.ts index 2c72203137d8..7eba54536fe1 100644 --- a/public/app/plugins/datasource/prometheus/specs/query_hints.test.ts +++ b/public/app/plugins/datasource/prometheus/specs/query_hints.test.ts @@ -2,34 +2,31 @@ import { getQueryHints } from '../query_hints'; describe('getQueryHints()', () => { it('returns no hints for no series', () => { - expect(getQueryHints([])).toEqual([]); + expect(getQueryHints('', [])).toEqual(null); }); it('returns no hints for empty series', () => { - expect(getQueryHints([{ datapoints: [], query: '' }])).toEqual([null]); + expect(getQueryHints('', [{ datapoints: [] }])).toEqual(null); }); it('returns no hint for a monotonously decreasing series', () => { - const series = [{ datapoints: [[23, 1000], [22, 1001]], query: 'metric', responseIndex: 0 }]; - const hints = getQueryHints(series); - expect(hints).toEqual([null]); + const series = [{ datapoints: [[23, 1000], [22, 1001]] }]; + const hints = getQueryHints('metric', series); + expect(hints).toEqual(null); }); it('returns no hint for a flat series', () => { - const series = [ - { datapoints: [[null, 1000], [23, 1001], [null, 1002], [23, 1003]], query: 'metric', responseIndex: 0 }, - ]; - const hints = getQueryHints(series); - expect(hints).toEqual([null]); + const series = [{ datapoints: [[null, 1000], [23, 1001], [null, 1002], [23, 1003]] }]; + const hints = getQueryHints('metric', series); + expect(hints).toEqual(null); }); it('returns a rate hint for a monotonously increasing series', () => { - const series = [{ datapoints: [[23, 1000], [24, 1001]], query: 'metric', responseIndex: 0 }]; - const hints = getQueryHints(series); + const series = [{ datapoints: [[23, 1000], [24, 1001]] }]; + const hints = getQueryHints('metric', series); expect(hints.length).toBe(1); expect(hints[0]).toMatchObject({ label: 'Time series is monotonously increasing.', - index: 0, fix: { action: { type: 'ADD_RATE', @@ -40,26 +37,25 @@ describe('getQueryHints()', () => { }); it('returns no rate hint for a monotonously increasing series that already has a rate', () => { - const series = [{ datapoints: [[23, 1000], [24, 1001]], query: 'rate(metric[1m])', responseIndex: 0 }]; - const hints = getQueryHints(series); - expect(hints).toEqual([null]); + const series = [{ datapoints: [[23, 1000], [24, 1001]] }]; + const hints = getQueryHints('rate(metric[1m])', series); + expect(hints).toEqual(null); }); it('returns a rate hint w/o action for a complex monotonously increasing series', () => { - const series = [{ datapoints: [[23, 1000], [24, 1001]], query: 'sum(metric)', responseIndex: 0 }]; - const hints = getQueryHints(series); + const series = [{ datapoints: [[23, 1000], [24, 1001]] }]; + const hints = getQueryHints('sum(metric)', series); expect(hints.length).toBe(1); expect(hints[0].label).toContain('rate()'); expect(hints[0].fix).toBeUndefined(); }); it('returns a rate hint for a monotonously increasing series with missing data', () => { - const series = [{ datapoints: [[23, 1000], [null, 1001], [24, 1002]], query: 'metric', responseIndex: 0 }]; - const hints = getQueryHints(series); + const series = [{ datapoints: [[23, 1000], [null, 1001], [24, 1002]] }]; + const hints = getQueryHints('metric', series); expect(hints.length).toBe(1); expect(hints[0]).toMatchObject({ label: 'Time series is monotonously increasing.', - index: 0, fix: { action: { type: 'ADD_RATE', @@ -70,12 +66,11 @@ describe('getQueryHints()', () => { }); it('returns a histogram hint for a bucket series', () => { - const series = [{ datapoints: [[23, 1000]], query: 'metric_bucket', responseIndex: 0 }]; - const hints = getQueryHints(series); + const series = [{ datapoints: [[23, 1000]] }]; + const hints = getQueryHints('metric_bucket', series); expect(hints.length).toBe(1); expect(hints[0]).toMatchObject({ label: 'Time series has buckets, you probably wanted a histogram.', - index: 0, fix: { action: { type: 'ADD_HISTOGRAM_QUANTILE', diff --git a/public/app/plugins/datasource/stackdriver/datasource.ts b/public/app/plugins/datasource/stackdriver/datasource.ts index cda952c23b96..4a81eb8a619a 100644 --- a/public/app/plugins/datasource/stackdriver/datasource.ts +++ b/public/app/plugins/datasource/stackdriver/datasource.ts @@ -107,34 +107,32 @@ export default class StackdriverDatasource { } async query(options) { - this.queryPromise = new Promise(async resolve => { - const result = []; - const data = await this.getTimeSeries(options); - if (data.results) { - Object['values'](data.results).forEach(queryRes => { - if (!queryRes.series) { - return; + const result = []; + const data = await this.getTimeSeries(options); + if (data.results) { + Object['values'](data.results).forEach(queryRes => { + if (!queryRes.series) { + return; + } + this.projectName = queryRes.meta.defaultProject; + const unit = this.resolvePanelUnitFromTargets(options.targets); + queryRes.series.forEach(series => { + let timeSerie: any = { + target: series.name, + datapoints: series.points, + refId: queryRes.refId, + meta: queryRes.meta, + }; + if (unit) { + timeSerie = { ...timeSerie, unit }; } - this.projectName = queryRes.meta.defaultProject; - const unit = this.resolvePanelUnitFromTargets(options.targets); - queryRes.series.forEach(series => { - let timeSerie: any = { - target: series.name, - datapoints: series.points, - refId: queryRes.refId, - meta: queryRes.meta, - }; - if (unit) { - timeSerie = { ...timeSerie, unit }; - } - result.push(timeSerie); - }); + result.push(timeSerie); }); - } - - resolve({ data: result }); - }); - return this.queryPromise; + }); + return { data: result }; + } else { + return { data: [] }; + } } async annotationQuery(options) { diff --git a/public/app/plugins/datasource/stackdriver/filter_segments.ts b/public/app/plugins/datasource/stackdriver/filter_segments.ts index 9eb27f31975b..5adb56e2fcf6 100644 --- a/public/app/plugins/datasource/stackdriver/filter_segments.ts +++ b/public/app/plugins/datasource/stackdriver/filter_segments.ts @@ -44,7 +44,7 @@ export class FilterSegments { this.removeSegment.value = DefaultRemoveFilterValue; return Promise.resolve([this.removeSegment]); } else { - return this.getFilterKeysFunc(); + return this.getFilterKeysFunc(segment, DefaultRemoveFilterValue); } } diff --git a/public/app/plugins/datasource/stackdriver/partials/query.filter.html b/public/app/plugins/datasource/stackdriver/partials/query.filter.html index 9ec59005a0b7..5043161c4920 100644 --- a/public/app/plugins/datasource/stackdriver/partials/query.filter.html +++ b/public/app/plugins/datasource/stackdriver/partials/query.filter.html @@ -28,7 +28,7 @@
Group By
- +
diff --git a/public/app/plugins/datasource/stackdriver/query_ctrl.ts b/public/app/plugins/datasource/stackdriver/query_ctrl.ts index 0996ce82919f..3a1961eb14e5 100644 --- a/public/app/plugins/datasource/stackdriver/query_ctrl.ts +++ b/public/app/plugins/datasource/stackdriver/query_ctrl.ts @@ -95,6 +95,5 @@ export class StackdriverQueryCtrl extends QueryCtrl { this.lastQueryError = jsonBody.error.message; } } - console.error(err); } } diff --git a/public/app/plugins/datasource/stackdriver/query_filter_ctrl.ts b/public/app/plugins/datasource/stackdriver/query_filter_ctrl.ts index df5fb0f29659..4c383e5d09ef 100644 --- a/public/app/plugins/datasource/stackdriver/query_filter_ctrl.ts +++ b/public/app/plugins/datasource/stackdriver/query_filter_ctrl.ts @@ -1,6 +1,6 @@ import coreModule from 'app/core/core_module'; import _ from 'lodash'; -import { FilterSegments, DefaultRemoveFilterValue } from './filter_segments'; +import { FilterSegments } from './filter_segments'; import appEvents from 'app/core/app_events'; export class StackdriverFilter { @@ -26,8 +26,10 @@ export class StackdriverFilter { export class StackdriverFilterCtrl { metricLabels: { [key: string]: string[] }; resourceLabels: { [key: string]: string[] }; + resourceTypes: string[]; defaultRemoveGroupByValue = '-- remove group by --'; + resourceTypeValue = 'resource.type'; loadLabelsPromise: Promise; service: string; @@ -72,7 +74,7 @@ export class StackdriverFilterCtrl { this.filterSegments = new FilterSegments( this.uiSegmentSrv, this.target, - this.getGroupBys.bind(this, null, null, DefaultRemoveFilterValue, false), + this.getFilterKeys.bind(this), this.getFilterValues.bind(this) ); this.filterSegments.buildSegmentModel(); @@ -151,6 +153,7 @@ export class StackdriverFilterCtrl { const data = await this.datasource.getLabels(this.target.metricType, this.target.refId); this.metricLabels = data.results[this.target.refId].meta.metricLabels; this.resourceLabels = data.results[this.target.refId].meta.resourceLabels; + this.resourceTypes = data.results[this.target.refId].meta.resourceTypes; resolve(); } catch (error) { if (error.data && error.data.message) { @@ -191,45 +194,66 @@ export class StackdriverFilterCtrl { this.$rootScope.$broadcast('metricTypeChanged'); } - async getGroupBys(segment, index, removeText?: string, removeUsed = true) { + async createLabelKeyElements() { await this.loadLabelsPromise; - const metricLabels = Object.keys(this.metricLabels || {}) - .filter(ml => { - if (!removeUsed) { - return true; - } - return this.target.aggregation.groupBys.indexOf('metric.label.' + ml) === -1; - }) - .map(l => { - return this.uiSegmentSrv.newSegment({ - value: `metric.label.${l}`, - expandable: false, - }); + let elements = Object.keys(this.metricLabels || {}).map(l => { + return this.uiSegmentSrv.newSegment({ + value: `metric.label.${l}`, + expandable: false, }); + }); - const resourceLabels = Object.keys(this.resourceLabels || {}) - .filter(ml => { - if (!removeUsed) { - return true; - } - - return this.target.aggregation.groupBys.indexOf('resource.label.' + ml) === -1; - }) - .map(l => { + elements = [ + ...elements, + ...Object.keys(this.resourceLabels || {}).map(l => { return this.uiSegmentSrv.newSegment({ value: `resource.label.${l}`, expandable: false, }); - }); + }), + ]; + + if (this.resourceTypes && this.resourceTypes.length > 0) { + elements = [ + ...elements, + this.uiSegmentSrv.newSegment({ + value: this.resourceTypeValue, + expandable: false, + }), + ]; + } + + return elements; + } + + async getFilterKeys(segment, removeText?: string) { + let elements = await this.createLabelKeyElements(); + if (this.target.filters.indexOf(this.resourceTypeValue) !== -1) { + elements = elements.filter(e => e.value !== this.resourceTypeValue); + } + + const noValueOrPlusButton = !segment || segment.type === 'plus-button'; + if (noValueOrPlusButton && elements.length === 0) { + return []; + } + + this.removeSegment.value = removeText; + return [...elements, this.removeSegment]; + } + + async getGroupBys(segment) { + let elements = await this.createLabelKeyElements(); + + elements = elements.filter(e => this.target.aggregation.groupBys.indexOf(e.value) === -1); const noValueOrPlusButton = !segment || segment.type === 'plus-button'; - if (noValueOrPlusButton && metricLabels.length === 0 && resourceLabels.length === 0) { - return Promise.resolve([]); + if (noValueOrPlusButton && elements.length === 0) { + return []; } - this.removeSegment.value = removeText || this.defaultRemoveGroupByValue; - return Promise.resolve([...metricLabels, ...resourceLabels, this.removeSegment]); + this.removeSegment.value = this.defaultRemoveGroupByValue; + return [...elements, this.removeSegment]; } groupByChanged(segment, index) { @@ -273,6 +297,10 @@ export class StackdriverFilterCtrl { return this.resourceLabels[shortKey]; } + if (filterKey === this.resourceTypeValue) { + return this.resourceTypes; + } + return []; } diff --git a/public/app/plugins/panel/graph/module.ts b/public/app/plugins/panel/graph/module.ts index c16cab52ccf6..9bb0213635a2 100644 --- a/public/app/plugins/panel/graph/module.ts +++ b/public/app/plugins/panel/graph/module.ts @@ -146,7 +146,7 @@ class GraphCtrl extends MetricsPanelCtrl { onInitPanelActions(actions) { actions.push({ text: 'Export CSV', click: 'ctrl.exportCsv()' }); - actions.push({ text: 'Toggle legend', click: 'ctrl.toggleLegend()' }); + actions.push({ text: 'Toggle legend', click: 'ctrl.toggleLegend()', shortcut: 'p l' }); } issueQueries(datasource) { diff --git a/public/app/plugins/panel/table/specs/transformers.test.ts b/public/app/plugins/panel/table/specs/transformers.test.ts index 8d581b68842c..49926aa00a8f 100644 --- a/public/app/plugins/panel/table/specs/transformers.test.ts +++ b/public/app/plugins/panel/table/specs/transformers.test.ts @@ -143,24 +143,6 @@ describe('when transforming time series table', () => { }, ]; - const multipleQueriesDataDifferentLabels = [ - { - type: 'table', - columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Value #A' }], - rows: [[time, 'Label Value 1', 42]], - }, - { - type: 'table', - columns: [{ text: 'Time' }, { text: 'Label Key 2' }, { text: 'Value #B' }], - rows: [[time, 'Label Value 2', 13]], - }, - { - type: 'table', - columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Value #C' }], - rows: [[time, 'Label Value 3', 7]], - }, - ]; - describe('getColumns', () => { it('should return data columns given a single query', () => { const columns = transformers[transform].getColumns(singleQueryData); @@ -177,16 +159,6 @@ describe('when transforming time series table', () => { expect(columns[3].text).toBe('Value #A'); expect(columns[4].text).toBe('Value #B'); }); - - it('should return the union of data columns given a multiple queries with different labels', () => { - const columns = transformers[transform].getColumns(multipleQueriesDataDifferentLabels); - expect(columns[0].text).toBe('Time'); - expect(columns[1].text).toBe('Label Key 1'); - expect(columns[2].text).toBe('Value #A'); - expect(columns[3].text).toBe('Label Key 2'); - expect(columns[4].text).toBe('Value #B'); - expect(columns[5].text).toBe('Value #C'); - }); }); describe('transform', () => { @@ -237,26 +209,6 @@ describe('when transforming time series table', () => { expect(table.rows[1][4]).toBeUndefined(); expect(table.rows[1][5]).toBe(7); }); - - it('should return 2 rows for multiple queries with different label values', () => { - table = transformDataToTable(multipleQueriesDataDifferentLabels, panel); - expect(table.rows.length).toBe(2); - expect(table.columns.length).toBe(6); - - expect(table.rows[0][0]).toBe(time); - expect(table.rows[0][1]).toBe('Label Value 1'); - expect(table.rows[0][2]).toBe(42); - expect(table.rows[0][3]).toBe('Label Value 2'); - expect(table.rows[0][4]).toBe(13); - expect(table.rows[0][5]).toBeUndefined(); - - expect(table.rows[1][0]).toBe(time); - expect(table.rows[1][1]).toBe('Label Value 3'); - expect(table.rows[1][2]).toBeUndefined(); - expect(table.rows[1][3]).toBeUndefined(); - expect(table.rows[1][4]).toBeUndefined(); - expect(table.rows[1][5]).toBe(7); - }); }); }); }); diff --git a/public/app/plugins/panel/table/transformers.ts b/public/app/plugins/panel/table/transformers.ts index 5a75fa7acf6c..c56d385505b6 100644 --- a/public/app/plugins/panel/table/transformers.ts +++ b/public/app/plugins/panel/table/transformers.ts @@ -1,7 +1,7 @@ import _ from 'lodash'; -import flatten from '../../../core/utils/flatten'; -import TimeSeries from '../../../core/time_series2'; -import TableModel from '../../../core/table_model'; +import flatten from 'app/core/utils/flatten'; +import TimeSeries from 'app/core/time_series2'; +import TableModel, { mergeTablesIntoModel } from 'app/core/table_model'; const transformers = {}; @@ -168,97 +168,7 @@ transformers['table'] = { }; } - // Single query returns data columns and rows as is - if (data.length === 1) { - model.columns = [...data[0].columns]; - model.rows = [...data[0].rows]; - return; - } - - // Track column indexes of union: name -> index - const columnNames = {}; - - // Union of all non-value columns - const columnsUnion = data.reduce((acc, series) => { - series.columns.forEach(col => { - const { text } = col; - if (columnNames[text] === undefined) { - columnNames[text] = acc.length; - acc.push(col); - } - }); - return acc; - }, []); - - // Map old column index to union index per series, e.g., - // given columnNames {A: 0, B: 1} and - // data [{columns: [{ text: 'A' }]}, {columns: [{ text: 'B' }]}] => [[0], [1]] - const columnIndexMapper = data.map(series => series.columns.map(col => columnNames[col.text])); - - // Flatten rows of all series and adjust new column indexes - const flattenedRows = data.reduce((acc, series, seriesIndex) => { - const mapper = columnIndexMapper[seriesIndex]; - series.rows.forEach(row => { - const alteredRow = []; - // Shifting entries according to index mapper - mapper.forEach((to, from) => { - alteredRow[to] = row[from]; - }); - acc.push(alteredRow); - }); - return acc; - }, []); - - // Returns true if both rows have matching non-empty fields as well as matching - // indexes where one field is empty and the other is not - function areRowsMatching(columns, row, otherRow) { - let foundFieldToMatch = false; - for (let columnIndex = 0; columnIndex < columns.length; columnIndex++) { - if (row[columnIndex] !== undefined && otherRow[columnIndex] !== undefined) { - if (row[columnIndex] !== otherRow[columnIndex]) { - return false; - } - } else if (row[columnIndex] === undefined || otherRow[columnIndex] === undefined) { - foundFieldToMatch = true; - } - } - return foundFieldToMatch; - } - - // Merge rows that have same values for columns - const mergedRows = {}; - const compactedRows = flattenedRows.reduce((acc, row, rowIndex) => { - if (!mergedRows[rowIndex]) { - // Look from current row onwards - let offset = rowIndex + 1; - // More than one row can be merged into current row - while (offset < flattenedRows.length) { - // Find next row that could be merged - const match = _.findIndex(flattenedRows, otherRow => areRowsMatching(columnsUnion, row, otherRow), offset); - if (match > -1) { - const matchedRow = flattenedRows[match]; - // Merge values from match into current row if there is a gap in the current row - for (let columnIndex = 0; columnIndex < columnsUnion.length; columnIndex++) { - if (row[columnIndex] === undefined && matchedRow[columnIndex] !== undefined) { - row[columnIndex] = matchedRow[columnIndex]; - } - } - // Don't visit this row again - mergedRows[match] = matchedRow; - // Keep looking for more rows to merge - offset = match + 1; - } else { - // No match found, stop looking - break; - } - } - acc.push(row); - } - return acc; - }, []); - - model.columns = columnsUnion; - model.rows = compactedRows; + mergeTablesIntoModel(model, ...data); }, }; diff --git a/public/app/routes/GrafanaCtrl.ts b/public/app/routes/GrafanaCtrl.ts index d6291c94a6fe..5dfa86226144 100644 --- a/public/app/routes/GrafanaCtrl.ts +++ b/public/app/routes/GrafanaCtrl.ts @@ -88,7 +88,7 @@ function setViewModeBodyClass(body, mode, sidemenuOpen: boolean) { break; } // 1 & true for legacy states - case 1: + case '1': case true: { body.removeClass('sidemenu-open'); body.addClass('view-mode--kiosk'); @@ -176,16 +176,16 @@ export function grafanaAppDirective(playlistSrv, contextSrv, $timeout, $rootScop const search = $location.search(); if (options && options.exit) { - search.kiosk = 1; + search.kiosk = '1'; } switch (search.kiosk) { case 'tv': { - search.kiosk = 1; + search.kiosk = true; appEvents.emit('alert-success', ['Press ESC to exit Kiosk mode']); break; } - case 1: + case '1': case true: { delete search.kiosk; break; diff --git a/public/app/types/explore.ts b/public/app/types/explore.ts index f6072becec46..918dd4e44837 100644 --- a/public/app/types/explore.ts +++ b/public/app/types/explore.ts @@ -3,6 +3,11 @@ interface ExploreDatasource { label: string; } +export interface HistoryItem { + ts: number; + query: string; +} + export interface Range { from: string; to: string; @@ -13,6 +18,19 @@ export interface Query { key?: string; } +export interface QueryTransaction { + id: string; + done: boolean; + error?: string; + hints?: any[]; + latency: number; + options: any; + query: string; + result?: any; // Table model / Timeseries[] / Logs + resultType: ResultType; + rowIndex: number; +} + export interface TextMatch { text: string; start: number; @@ -27,34 +45,25 @@ export interface ExploreState { datasourceMissing: boolean; datasourceName?: string; exploreDatasources: ExploreDatasource[]; - graphResult: any; - history: any[]; - latency: number; - loading: any; - logsResult: any; + graphRange: Range; + history: HistoryItem[]; /** * Initial rows of queries to push down the tree. * Modifications do not end up here, but in `this.queryExpressions`. * The only way to reset a query is to change its `key`. */ queries: Query[]; - /** - * Errors caused by the running the query row. - */ - queryErrors: any[]; /** * Hints gathered for the query row. */ - queryHints: any[]; + queryTransactions: QueryTransaction[]; range: Range; - requestOptions: any; showingGraph: boolean; showingLogs: boolean; showingTable: boolean; supportsGraph: boolean | null; supportsLogs: boolean | null; supportsTable: boolean | null; - tableResult: any; } export interface ExploreUrlState { @@ -62,3 +71,5 @@ export interface ExploreUrlState { queries: Query[]; range: Range; } + +export type ResultType = 'Graph' | 'Logs' | 'Table'; diff --git a/public/sass/pages/_dashboard.scss b/public/sass/pages/_dashboard.scss index d9ab29cc91c2..795766a22ded 100644 --- a/public/sass/pages/_dashboard.scss +++ b/public/sass/pages/_dashboard.scss @@ -21,6 +21,9 @@ div.flot-text { height: 100%; &--solo { + position: fixed; + bottom: 0; + right: 0; margin: 0; .panel-container { border: none; diff --git a/public/sass/pages/_explore.scss b/public/sass/pages/_explore.scss index c1198ce06be5..a3f60f2006bb 100644 --- a/public/sass/pages/_explore.scss +++ b/public/sass/pages/_explore.scss @@ -74,7 +74,7 @@ } } - .elapsed-time { + .navbar .elapsed-time { position: absolute; left: 0; right: 0; @@ -87,6 +87,37 @@ flex-wrap: wrap; } + .explore-graph__loader { + height: 2px; + position: relative; + overflow: hidden; + background: $text-color-faint; + margin: $panel-margin / 2; + } + + .explore-graph__loader:after { + content: ' '; + display: block; + width: 25%; + top: 0; + top: -50%; + height: 250%; + position: absolute; + animation: loader 2s cubic-bezier(0.17, 0.67, 0.83, 0.67); + animation-iteration-count: 100; + z-index: 2; + background: $blue; + } + + @keyframes loader { + from { + left: -25%; + } + to { + left: 100%; + } + } + .datasource-picker { min-width: 200px; } @@ -119,6 +150,7 @@ .query-row { display: flex; + position: relative; & + & { margin-top: 0.5rem; @@ -129,11 +161,53 @@ white-space: nowrap; } +.query-row-status { + position: absolute; + top: 0; + right: 90px; + z-index: 1024; + display: flex; + flex-direction: column; + justify-content: center; + height: 34px; +} + .query-row-field { margin-right: 3px; width: 100%; } +.query-transactions { + display: table; +} + +.query-transaction { + display: table-row; + color: $text-color-faint; + line-height: 1.44; +} + +.query-transaction--loading { + animation: query-loading-color-change 1s alternate 100; +} + +@keyframes query-loading-color-change { + from { + color: $text-color-faint; + } + to { + color: $blue; + } +} + +.query-transaction__type, +.query-transaction__duration { + display: table-cell; + font-size: $font-size-xs; + text-align: right; + padding-right: 0.25em; +} + .explore { .logs { .logs-entries {