The Firecracker microVM Metadata Service (MMDS) is a mutable data store which can be used for sharing information between host and guests, in a secure and easy at hand way.
By default, MMDS is not reachable from the guest operating system. At microVM runtime, MMDS is tightly coupled with a network interface, which allows MMDS requests. When configuring the microVM, if MMDS needs to be activated, a network interface has to be configured to allow MMDS requests. This can be achieved in two steps:
- Attach one (or more) network interfaces through an HTTP
PUT
request to/network-interfaces/${MMDS_NET_IF}
. The full network configuration API can be found in the firecracker swagger file. - Configure MMDS through an HTTP
PUT
request to/mmds/config
resource and include the IDs of the network interfaces that should allow forwarding requests to MMDS in thenetwork_interfaces
list. The complete MMDS API is described in the firecracker swagger file.
Attaching a network device with ID MMDS_NET_IF
:
MMDS_NET_IF=eth0
curl --unix-socket /tmp/firecracker.socket -i \
-X PUT 'http://localhost/network-interfaces/${MMDS_NET_IF}' \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"iface_id": "${MMDS_NET_IF}",
"guest_mac": "AA:FC:00:00:00:01",
"host_dev_name": "tap0"
}'
Configuring MMDS to receive requests through the MMDS_NET_IF
network interface
ID:
MMDS_IPV4_ADDR=169.254.170.2
curl --unix-socket /tmp/firecracker.socket -i \
-X PUT "http://localhost/mmds/config" \
-H "Content-Type: application/json" \
-d '{
"network_interfaces": ["${MMDS_NET_IF}"]
}'
MMDS can be configured pre-boot only, using the Firecracker API server. Enabling MMDS without at least a network device attached will return an error.
The IPv4 address used by guest applications when issuing requests to MMDS can be
customized through the same HTTP PUT
request to /mmds/config
resource, by
specifying the IPv4 address to the ipv4_address
field. If the IP configuration
is not provided before booting up the guest, the MMDS IPv4 address defaults to
169.254.169.254
.
MMDS_IPV4_ADDR=169.254.170.2
curl --unix-socket /tmp/firecracker.socket -i \
-X PUT "http://localhost/mmds/config" \
-H "Content-Type: application/json" \
-d '{
"network_interfaces": ["${MMDS_NET_IF}"],
"ipv4_address": "${MMDS_IPV4_ADDR}"
}'
MMDS is tightly coupled with a network interface which is used to route MMDS packets. To send MMDS intended packets, guest applications must insert a new rule into the routing table of the guest OS. This new rule must forward MMDS intended packets to a network interface which allows MMDS requests. For example:
MMDS_IPV4_ADDR=169.254.170.2
MMDS_NET_IF=eth0
ip route add ${MMDS_IPV4_ADDR} dev ${MMDS_NET_IF}
MMDS supports two methods to access the contents of the metadata store from the
guest operating system: V1
and V2
. More about the particularities of the two
mechanisms can be found in the
Retrieving metadata in the guest operating system
section. The MMDS version used can be specified when configuring MMDS, through
the version
field of the HTTP PUT
request to /mmds/config
resource.
Accepted values are V1
(deprecated) and V2
and the default MMDS version used
in case the version
field is missing is Version 1.
MMDS_IPV4_ADDR=169.254.170.2
curl --unix-socket /tmp/firecracker.socket -i \
-X PUT "http://localhost/mmds/config" \
-H "Content-Type: application/json" \
-d '{
"network_interfaces": ["${MMDS_NET_IF}"],
"version": "V2",
"ipv4_address": "${MMDS_IPV4_ADDR}"
}'
Inserting and updating metadata is possible through the Firecracker API server.
The metadata inserted in MMDS must be any valid JSON. A user can create or
update the MMDS data store before the microVM is started or during its
operation. To insert metadata into MMDS, an HTTP PUT
request to the /mmds
resource has to be issued. This request must have a payload with metadata
structured in JSON format. To replace
existing metadata, a subsequent HTTP PUT
request to the /mmds
resource must
be issued, using as a payload the new metadata. A complete description of
metadata insertion firecracker API can be found in the
firecracker swagger file.
An example of an API request for inserting metadata is provided below:
curl --unix-socket /tmp/firecracker.socket -i \
-X PUT "http://localhost/mmds" \
-H "Content-Type: application/json" \
-d '{
"latest": {
"meta-data": {
"ami-id": "ami-12345678",
"reservation-id": "r-fea54097",
"local-hostname": "ip-10-251-50-12.ec2.internal",
"public-hostname": "ec2-203-0-113-25.compute-1.amazonaws.com",
"network": {
"interfaces": {
"macs": {
"02:29:96:8f:6a:2d": {
"device-number": "13345342",
"local-hostname": "localhost",
"subnet-id": "subnet-be9b61d"
}
}
}
}
}
}
}'
To partially update existing metadata, an HTTP PATCH
request to the /mmds
resource has to be issued, using as a payload the metadata patch, as
JSON Merge Patch functionality describes.
A complete description of updating metadata Firecracker API can be found in the
firecracker swagger file.
An example API for how to update existing metadata is offered below:
curl --unix-socket /tmp/firecracker.socket -i \
-X PATCH "http://localhost/mmds" \
-H "Content-Type: application/json" \
-d '{
"latest": {
"meta-data": {
"ami-id": "ami-87654321",
"reservation-id": "r-79054aef",
}
}
}'
MicroVM metadata can be retrieved both from host and guest operating systems. For the scope of this chapter, let's assume the data store content is the JSON below:
{
"latest": {
"meta-data": {
"ami-id": "ami-87654321",
"reservation-id": "r-79054aef"
}
}
}
To retrieve existing MMDS metadata from host operating system, an HTTP GET
request to the /mmds
resource must be issued. The HTTP response returns the
existing metadata, as a JSON formatted text. A complete description of
retrieving metadata Firecracker API can be found in the
firecracker swagger file.
Below you can see how to retrieve metadata from the host:
curl -s --unix-socket /tmp/firecracker.socket http://localhost/mmds
Output:
{
"latest": {
"meta-data": {
"ami-id": "ami-87654321",
"reservation-id": "r-79054aef"
}
}
}
Accessing the contents of the metadata store from the guest operating system can be done using one of the following methods:
V1
: simple request/response method (deprecated)V2
: session-oriented method
Version 1 is deprecated and will be removed in the next major version change. Version 2 should be used instead.
To retrieve existing MMDS metadata using MMDS version 1, an HTTP GET
request
must be issued. The requested resource can be referenced by its corresponding
JSON Pointer, which is also the path of
the MMDS request. The HTTP response content will contain the referenced metadata
resource.
The only HTTP method supported by MMDS version 1 is GET
. Requests containing
any other HTTP method will receive 405 Method Not Allowed error.
MMDS_IPV4_ADDR=169.254.170.2
RESOURCE_POINTER_OBJ=latest/meta-data
curl -s "http://${MMDS_IPV4_ADDR}/${RESOURCE_POINTER_OBJ}"
Similar to
IMDSv2,
MMDS version 2 (V2
) is a session oriented method, which makes use of a session
token in order to allow fetching metadata contents.
The session must start with an HTTP PUT
request that generates the session
token. In order to be successful, the request must respect the following
constraints:
- must be directed towards
/latest/api/token
path - must contain a
X-metadata-token-ttl-seconds
header specifying the token lifetime in seconds. The value cannot be lower than 1 or greater than 21600 (6 hours). - must not contain a
X-Forwarded-For
header.
MMDS_IPV4_ADDR=169.254.170.2
TOKEN=`curl -X PUT "http://${MMDS_IPV4_ADDR}/latest/api/token" \
-H "X-metadata-token-ttl-seconds: 21600"`
The HTTP response from MMDS is a plaintext containing the session token.
During the duration specified by the token's time to live value, all subsequent
GET
requests must specify the session token through the X-metadata-token
header in order to fetch data from MMDS.
MMDS_IPV4_ADDR=169.254.170.2
RESOURCE_POINTER_OBJ=latest/meta-data
curl -s "http://${MMDS_IPV4_ADDR}/${RESOURCE_POINTER_OBJ}" \
-H "X-metadata-token: ${TOKEN}"
After the token expires, it becomes unusable and a new session token must be issued.
The data store is not persisted across snapshots, in order to avoid leaking vm-specific information that may need to be reseeded into the data store for a new clone.
The MMDS version, network stack configuration and IP address used for accessing the service are persisted across snapshot-restore.
If the targeted snapshot version does not support Mmds Version 2, it will not be persisted in the snapshot (the clone will use the default, V1). Similarly, if a snapshotted Vm state contains the Mmds version but the Firecracker version used for restoring does not support persisting the version, the default will be used.
The response format can be JSON or IMDS. The IMDS documentation can be found
here.
The output format can be selected by specifying the optional Accept
header.
Using Accept: application/json
will format the output to JSON, while using
Accept: plain/text
or not specifying this optional header at all will format
the output to IMDS.
Retrieving MMDS resources in IMDS format, other than JSON string
and object
types, is not supported.
Below is an example on how to retrieve the latest/meta-data
resource in JSON
format:
MMDS_IPV4_ADDR=169.254.170.2
RESOURCE_POINTER_OBJ=latest/meta-data
curl -s -H "Accept: application/json" "http://${MMDS_IPV4_ADDR}/${RESOURCE_POINTER_OBJ}"
Output:
{
"ami-id": "ami-87654321",
"reservation-id": "r-79054aef"
}
Retrieving the latest/meta-data/ami-id
resource in JSON format:
MMDS_IPV4_ADDR=169.254.170.2
RESOURCE_POINTER_STR=latest/meta-data/ami-id
curl -s -H "Accept: application/json" "http://${MMDS_IPV4_ADDR}/${RESOURCE_POINTER_STR}"
Output:
"ami-87654321"
Retrieving the latest
resource in IMDS format:
MMDS_IPV4_ADDR=169.254.170.2
RESOURCE_POINTER=latest
curl -s "http://${MMDS_IPV4_ADDR}/${RESOURCE_POINTER}"
Output:
meta-data/
Retrieving the latest/meta-data/
resource in IMDS format:
MMDS_IPV4_ADDR=169.254.170.2
RESOURCE_POINTER=latest/meta-data
curl -s "http://${MMDS_IPV4_ADDR}/${RESOURCE_POINTER}"
Output:
ami-id
reservation-id
Retrieving the latest/meta-data/ami-id
resource in IMDS format:
MMDS_IPV4_ADDR=169.254.170.2
RESOURCE_POINTER=latest/meta-data/ami-id
curl -s "http://${MMDS_IPV4_ADDR}/${RESOURCE_POINTER}"
Output:
ami-87654321
200 - Ok
The request was successfully processed and a response was successfully formed.
400 - Bad Request
The request was malformed.
401 - Unauthorized
Only when using MMDS V2
. The HTTP request either lacks the session token, or
the token specified is invalid. A token is invalid if it was not generated using
an HTTP PUT
request or if it has expired.
404 - Not Found
The requested resource can not be found in the MMDS data store.
405 - Method Not Allowed
The HTTP request uses a not allowed HTTP method and a response with the Allow
header was formed. When using MMDS V1
, this is returned for any HTTP method
other than GET
. When MMDS V2
is configured, the only accepted HTTP methods
are PUT
and GET
.
501 - Not Implemented
The requested HTTP functionality is not supported by MMDS or the requested resource is not supported in IMDS format.
For this example, the guest expects to find some sort of credentials (say, a
secret access key) by issuing a GET
request to
http://169.254.169.254/latest/meta-data/credentials/secret-key
. Most similar
use cases will encompass the following sequence of steps:
- Some agent running on the host sends a
PUT
request with the initial contents of the MMDS, using the Firecracker API. This most likely takes place before the microVM starts running, but may also happen at a later time. Guest MMDS requests which arrive prior to contents being available receive a NotFound response. - The contents are saved to MMDS.
- The guest sends a
GET
request for the secret key, which is intercepted by MMDS. - MMDS processes the request and sends back an HTTP response with the ensembled secret key as a JSON string.
After a while, the host agent decides to rotate the secret key. It does so by
updating the data store with a new value. This can be done via a PUT
request
to the /mmds
API resource, which replaces everything, or with a PATCH
request that only touches the desired key. This effectively triggers the first
two steps again.
The guest reads the new secret key, going one more time through the last three steps. This can happen after a notification from the host agent, or discovered via periodic polling, or some other mechanism. Since access to the data store is thread safe, the guest can only receive either the old version, or the new version of the key, and not some intermediate state caused by the update.