Add new cloud.* (or cloud related fields)

[legoguy1000]

Summary

Add a cloud.edge_location or something similar field.

Motivation:

Many cloud providers have not just regions but edge locations, many of which use Airport codes to identify them. This includes providers like cloudflare and AWS (used with route53, CloudFront...). It would nice to have a standard cloud.* field to put this information in.

Detailed Design:

Provide additional details around the design of the proposed changes.

• Field names
• Example values for the fields
• Suggested appropriate datatypes
• Any example events that map to the proposed use case(s)

Field: cloud.edge_location
Example: IAD (https://www.cloudflarestatus.com for a list of cloudflare edge locations and their codes)
Type: keyword
Samples:
AWS CloudFront
AWS Route 53
Cloudflare

[ebeahan]

The motivation for introducing a common field to store the edge/point of presence (PoP) makes sense. I'm not 100% sure about placing underneath cloud.*, though.

The big three public cloud providers all have CDN services, but not all CDNs will also be cloud providers. A CDN is a type of network, so perhaps network.edge_location?

I'd be curious about what others think.

There's an (inactive) meta issue to assess better how ECS should represent intermediaries like CDNs, proxies, load balancers, etc.: #938. Linking it here since capturing the PoP/edge location might also carry over to that work (and geo-based DNS load balancing use cases).

[legoguy1000]

I tend to consider CDNs to be a cloud resource which is why my initial thought was to put it there and its not only CDNs that have "edge locations" (Like the Route 53 resolveres), but under network could work too. I'm tracking #938 and the conversation for when I built the Cloudflare Integration for the agent. With that logic, you could also put it under observer.*

[legoguy1000]

@ebeahan Can we use this issue to discuss other fields to possible add to the cloud.* space? I'm working on a new integration for AWS GuardDuty and would be nice to have things like to name a few.

• cloud.instance.image.id
• cloud.instance.image.name
• cloud.vpc.name
• cloud.vpc.id

[ebeahan]

Can we use this issue to discuss other fields to possible add to the cloud.* space?

Feel free to expand the discussion, but please update the title to reflect the new scope.

Opening an RFC might be best if we're discussing adding several fields. The proposal template and different stages can help anchor the conversation and provide a place for example mappings, concerns, etc.

[ebeahan]

cloud.instance.image.id
cloud.instance.image.name
cloud.vpc.name
cloud.vpc.id

Do image.id and image.name need to be tied to a cloud.instance, or could those fields also work directly under cloud.*?

What about placing the VPC name/id values under network.*? This approach allows querying cloud and on-prem networks using one field: network.name. Note that network.name exists, but not network.id. I'm also unsure about using vpc in the naming. Different providers use different terms (many use VPC, but Azure uses VNet instead).

[legoguy1000]

Do image.id and image.name need to be tied to a cloud.instance, or could those fields also work directly under cloud.*?

They could be at the root. My thought was only that the image is a instance specific thing. kinda follows suite with container.image.name

What about placing the VPC name/id values under network.*?

I thought about network.name and while no, it doesn't have to use vpc in the name and I'll acknowledge Azure, I think its kinda the standard terminology. Also usually cloud VPCs are broken into subnets that have their own names/ids to could conflict with that too?

I was also thinking something similar for S3/Object store cloud.object.* or cloud.object_store.*..

[legoguy1000]

Bringing this back up. Any thoughts on the above. I can start drafting up an RFC. Any other cloud(ish) fields?

[ebeahan]

There's been past discussion about a cloud.instance.lifecycle field to capture if a cloud instance is ondemand, spot, preemptible, etc. It may have a place alongside these other proposed fields.

[amitkanfer]

One thing i'm missing in the cloud fields is the original id of a resource, any resource.
In AWS, for example, this could be the arn of a resource (Amazon Resource Name).
That will be a unique world-wide name for any resource in AWS, and can allow us to reuse it for our purposes. I think it's a useful ID to capture.

/cc @ebeahan

[legoguy1000]

stage 0 RFC merged so i'm going to close this and start working on the stage 1 PR