Airflow TSDB Enablement

[agithomas]

Test Environment Setup

• Creation of Airflow Test Environment.

Datastream : StatsD

• Add dimension fields
  Added dimension fields for airflow to enable tsdb #5847
• Adding metric_type mapping for Airflow TSDB Enablement #6729
  [Airflow] Revert metrics field definition to the format used before i… #7469

Verification and validation

• Verification of data in visualisation after enabling TSDB flag in kibana
• Verification of the count of documents (before & after TSDB enablement) in Discover Interface
• Verify if field mapping is correct in the data stream template.

Issues

• [Fleet] TSDB: need metric_type mapping for dynamic templates kibana#155004. [Blocker for metric_type]
• Choosing the right metric_type mapping and aggregator function for the Airflow dashboard's scheduler.heartbeat field in the context of TSDB. #7504

Enable TSDB by default: #6776

Screenshots

[agithomas]

@ritalwar , please update the issue with the current status and please link the issue related to the metric_type mapping here. Also, add the details of the issue under Meta.

[ritalwar]

Closing this issue for now.

I'll create a metric_type PR once metric_type mapping for dynamic templates is available.

[lalit-satapathy]

@ritalwar, Can the metric_type mapping be added now since elastic/kibana#155004 is fixed. Also, keeping this meta open for now.