MySQL TSDB Enablement

[agithomas]

Test Environment Setup

• Creation of MySQL setup for testing / Data preparation for testing.
• Creation of Galera Cluster for Galera Status

Datastream : Performance

• Add dimension fields
  • [MySQL] Add dimension mapping to support TSDB enablement #5613
• Adding metric_type mapping
  • [MySQL][TSDB] Add metric type mapping for performance datastream #5378

Verification and validation

• Verify if field mapping is correct in the data stream template.
• Verification of data in visualisation after enabling TSDB flag in kibana
• Verification of the count of documents (before & after TSDB enablement) in Discover Interface

Datastream : Status

• Add dimension fields
  • [MySQL] [TSDB] Add dimension fields to support TSDB Enablement #5337
• Adding metric_type mapping
  • [TSDB] [MySQL] Add metrictype mapping for MySQL status datastream #5387

Verification and validation

• Verify if field mapping is correct in the data stream template.
• Verification of data in visualisation after enabling TSDB flag in kibana. Details
• Verification of the count of documents (before & after TSDB enablement) in Discover Interface

Datastream : Galera Status

• Add dimension fields
  • [MySQL] [TSDB] Add dimension fields to support TSDB Enablement #5337
• Adding metric_type mapping
  • [TSDB] [MySQL]Add metrictype mapping for mysql galerastatus metrictype #5407

Verification and validation

• Verify if field mapping is correct in the data stream template.
• Verification of data in visualisation after enabling TSDB flag in kibana. Details
• Verification of the count of documents (before & after TSDB enablement) in Discover Interface

Dashboard fixed

• PR link to fix dashboard to support counter fields [MySql] Fix to correct the aggregate function used in dashboard #6828

TSDB Enablement PR

• [MySQL] TSDB Enablement #6968

Issues

• [TSDB] Enablement of TSDB fails for package with fields having flattened type elasticsearch#94113. [Blocker]

Backlog Items

Links to backlog issues that could not be addressed in this TSDB enablement activity

• [TSDB][MySQL] TSDB Enablement for Performance datastream in MySQL Integration Package #5385

[agithomas]

Verified the Dashboard Rendering

[lalit-satapathy]

Currently blocked on: elastic/elasticsearch#94113.

[agithomas]

MySQL Performance Field mapping details added below after using 8.8 and having flattened type field in the mappings.

"mysql": {
        "properties": {
          "performance": {
            "properties": {
              "events_statements": {
                "properties": {
                  "avg": {
                    "properties": {
                      "timer": {
                        "properties": {
                          "wait": {
                            "type": "long",
                            "time_series_metric": "gauge"
                          }
                        }
                      }
                    }
                  },
                  "count": {
                    "properties": {
                      "star": {
                        "type": "long",
                        "time_series_metric": "gauge"
                      }
                    }
                  },
                  "digest": {
                    "type": "flattened"
                  },
                  "last": {
                    "properties": {
                      "seen": {
                        "type": "date"
                      }
                    }
                  },
                  "max": {
                    "properties": {
                      "timer": {
                        "properties": {
                          "wait": {
                            "type": "long",
                            "time_series_metric": "gauge"
                          }
                        }
                      }
                    }
                  },
                  "quantile": {
                    "properties": {
                      "95": {
                        "type": "long",
                        "time_series_metric": "gauge"
                      }
                    }
                  },
                  "query_id": {
                    "type": "keyword",
                    "time_series_dimension": true
                  }
                }
              },
              "table_io_waits": {
                "properties": {
                  "count": {
                    "properties": {
                      "fetch": {
                        "type": "long",
                        "time_series_metric": "gauge"
                      }
                    }
                  },
                  "index": {
                    "properties": {
                      "name": {
                        "type": "keyword",
                        "ignore_above": 1024
                      }
                    }
                  },
                  "object": {
                    "properties": {
                      "name": {
                        "type": "keyword",
                        "time_series_dimension": true
                      },
                      "schema": {
                        "type": "keyword",
                        "time_series_dimension": true
                      }
                    }
                  }
                }
              }
            }
          }
        }
      },
      "service": {
        "properties": {
          "address": {
            "type": "keyword",
            "time_series_dimension": true
          },
          "type": {
            "type": "keyword",
            "ignore_above": 1024
          }
        }
      }

[agithomas]

PR review is currently blocked on - if it is Ok to keep 8.8.0 as the min dependency.

[lalit-satapathy]

@agithomas, Anything pending on this? Can we merge the dimension PRs? 8.8.0 can be added in future PR.

[lalit-satapathy]

@agithomas, Can we update the issue with plan for MySQL?

[agithomas]

@agithomas, Can we update the issue with plan for MySQL?

The MySQL pipeline is presently broken .
Once the issue is resolved, TSDB changes can be merged.

[agithomas]

[agithomas]

[agithomas]