Sort sub-aggregation axis by term not working

[pemontto]

Kibana version: 5.2.2

Elasticsearch version: 5.2.2

Browser version: Chrome, latest

Browser OS version: MacOS

Original install method (e.g. download page, yum, from source, etc.): download

Description of the problem including expected versus actual behavior: When ordering by terms the sub-bucket order isn't respected. I'm seeing the issue in both the heatmap and bar chart.

Steps to reproduce:

1. Create a heatmap visualisation
2. Add an X and Y axis aggregation and order by term
3. Notice the sub-aggregations order isn't respected

Provide logs and/or server output (if relevant):
dayOfWeek sorted correctly, sub-aggregation hourOfDay incorrectly ordered:

hourOfDay sorted correctly, sub-aggregation dayOfWeek incorrectly ordered:

[pemontto]

OK testing out on different data this issue only occurs when the top level aggregation buckets return partial results. So for instance in the first screenshot 01-Monday didn't return data for hours 01, 02, 03, 10 etc. The first time 02 appears is in the 03-Wednesday bucket.

On indices where all days have some value for each hour the ordering works as expected.

This affects the vertical bar chart as well as the heatmap.

[pemontto]

Just to clarify this isn't limited to the heatmap visualisation, partial buckets also have this effect on other charts.

[pemontto]

@spalger we discussed this at Elasticon

[pemontto]

Screenshot to follow

{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 15,
    "successful": 15,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 129,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "2": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "3": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Critical",
                "doc_count": 1
              },
              {
                "key": "High",
                "doc_count": 1
              },
              {
                "key": "Low",
                "doc_count": 1
              },
              {
                "key": "Medium",
                "doc_count": 4
              }
            ]
          },
          "key": "A1 - Injection",
          "doc_count": 7
        },
        {
          "3": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Medium",
                "doc_count": 1
              }
            ]
          },
          "key": "A10 - Unvalidated Redirects and Forwards",
          "doc_count": 1
        },
        {
          "3": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Low",
                "doc_count": 13
              },
              {
                "key": "Medium",
                "doc_count": 3
              }
            ]
          },
          "key": "A2 - Broken Authentication and Session Management",
          "doc_count": 16
        },
        {
          "3": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Medium",
                "doc_count": 4
              }
            ]
          },
          "key": "A3 - Cross-Site Scripting (XSS)",
          "doc_count": 4
        },
        {
          "3": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Medium",
                "doc_count": 2
              }
            ]
          },
          "key": "A4 - Insecure Direct Object References",
          "doc_count": 2
        },
        {
          "3": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "High",
                "doc_count": 3
              },
              {
                "key": "Info",
                "doc_count": 3
              },
              {
                "key": "Low",
                "doc_count": 39
              },
              {
                "key": "Medium",
                "doc_count": 11
              }
            ]
          },
          "key": "A5 - Security Misconfiguration",
          "doc_count": 56
        },
        {
          "3": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "High",
                "doc_count": 1
              },
              {
                "key": "Low",
                "doc_count": 3
              },
              {
                "key": "Medium",
                "doc_count": 6
              }
            ]
          },
          "key": "A6 - Sensitive Data Exposure",
          "doc_count": 10
        },
        {
          "3": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Critical",
                "doc_count": 1
              },
              {
                "key": "Info",
                "doc_count": 1
              },
              {
                "key": "Low",
                "doc_count": 3
              },
              {
                "key": "Medium",
                "doc_count": 8
              }
            ]
          },
          "key": "A7 - Missing Function Level Access Control",
          "doc_count": 13
        },
        {
          "3": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "Low",
                "doc_count": 1
              },
              {
                "key": "Medium",
                "doc_count": 1
              }
            ]
          },
          "key": "A8 - Cross-Site Request Forgery (CSRF)",
          "doc_count": 2
        },
        {
          "3": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "High",
                "doc_count": 2
              },
              {
                "key": "Low",
                "doc_count": 7
              },
              {
                "key": "Medium",
                "doc_count": 4
              }
            ]
          },
          "key": "A9 - Using Components with Known Vulnerabilities",
          "doc_count": 13
        }
      ]
    }
  },
  "status": 200
}

[pemontto]

Screenshot showing out of order X-axis

[dtungbmw]

any update on this?

[jpazsedano]

I'm using Elasticsearch and Kibana v6.4.2 and this still happens.

[elasticmachine]

Pinging @elastic/datavis (Feature:ElasticCharts)

[flash1293]

Blocked on #86184