Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add build metadata to boxes and build artifacts. #365

Merged
merged 1 commit into from
May 29, 2015
Merged

Conversation

fnichol
Copy link
Contributor

@fnichol fnichol commented May 29, 2015

For each Packer template that is run via bin/bento, a JSON file of
build metadata will be written to the builds/ directory, which looks
like the following (using the ubuntu-14.100i386 template as an example):

{
  "name": "ubuntu-14.10-i386",
  "version": "2.0.20150528211301",
  "build_timestamp": "20150528211301",
  "git_revision": "6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b",
  "box_basename": "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b",
  "atlas_org": "chef",
  "arch": "32",
  "template": "ubuntu-14.10-i386",
  "md5": {
    "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.parallels.box": "e3a18b096cddc73384f0912c3a65ebad",
    "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.virtualbox.box": "106f2ca4e6da18663e7216a72dd62e56",
    "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.vmware.box": "8990550bc2a0e2e7515ed3433ec54b46"
  },
  "sha256": {
    "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.parallels.box": "0a0e3c9369de005a456f0cd7d94ba4d4b562d7231c11d9c5af8e40ef77131d3d",
    "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.virtualbox.box": "0c23480a99294aea8f42daea2576a41820ec3bebb99a9d0a8ab72a3de1b24137",
    "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.vmware.box": "9128b66ef4bae323a123fcdd0be5a598bb538f822295ab6bf043e7630a49b608"
  }
}

In addition to the "sidecar" metadata file, a trimmed down version will
is added to each Vagrant box in /etc/bento-metadata.json. Using the
example above, here is what the file would look like:

{
  "name": "ubuntu-14.10-i386",
  "version": "2.0.20150528211301",
  "build_timestamp": "20150528211301",
  "git_revision": "6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b_dirty",
  "box_basename": "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b_dirty",
  "atlas_org": "chef",
  "arch": "32",
  "template": "ubuntu-14.10-i386"
}

Also note that this changes the file naming scheme of the resulting box
artifacts in an effort to host multiple builds of the same templates in
one directory while maintaining enough information about the box within
the filename itself.

Using the same example as above, the VirtualBox provider box name is:

chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.virtualbox.box

Which uses the following recipe to construct the filename:

  • atlas_org value (default: "chef")
  • double underscore, which could be later interpreted as a slash (/)
    for an Atalas box name
  • name value which may or may not equal the name of the template
    (captured as the template value)
  • a dash
  • version value, which removes the last digit in a version string and
    replaces it with the build_timestamp (a
    Year/Month/Day/Hour/Minute/Second format in UTC timezone)
  • a period
  • the string "git"
  • a period
  • git_revision value, which will append "_dirty" if the current
    state of the git repository is not completely clean (i.e., there are
    uncommitted changes which happens in active development)
  • a period
  • the value of the {{.Provider}} Packer variable, being one of
    "virtualbox", "vmware", or "parallels"
  • finished with ".box"

Closes #364

For each Packer template that is run via `bin/bento`, a JSON file of
build metadata will be written to the `builds/` directory, which looks
like the following (using the `ubuntu-14.100i386` template as an example):

    {
      "name": "ubuntu-14.10-i386",
      "version": "2.0.20150528211301",
      "build_timestamp": "20150528211301",
      "git_revision": "6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b",
      "box_basename": "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b",
      "atlas_org": "chef",
      "arch": "32",
      "template": "ubuntu-14.10-i386",
      "md5": {
        "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.parallels.box": "e3a18b096cddc73384f0912c3a65ebad",
        "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.virtualbox.box": "106f2ca4e6da18663e7216a72dd62e56",
        "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.vmware.box": "8990550bc2a0e2e7515ed3433ec54b46"
      },
      "sha256": {
        "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.parallels.box": "0a0e3c9369de005a456f0cd7d94ba4d4b562d7231c11d9c5af8e40ef77131d3d",
        "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.virtualbox.box": "0c23480a99294aea8f42daea2576a41820ec3bebb99a9d0a8ab72a3de1b24137",
        "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.vmware.box": "9128b66ef4bae323a123fcdd0be5a598bb538f822295ab6bf043e7630a49b608"
      }
    }

In addition to the "sidecar" metadata file, a trimmed down version will
is added to each Vagrant box in `/etc/bento-metadata.json`. Using the
example above, here is what the file would look like:

    {
      "name": "ubuntu-14.10-i386",
      "version": "2.0.20150528211301",
      "build_timestamp": "20150528211301",
      "git_revision": "6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b_dirty",
      "box_basename": "chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b_dirty",
      "atlas_org": "chef",
      "arch": "32",
      "template": "ubuntu-14.10-i386"
    }

Also note that this changes the file naming scheme of the resulting box
artifacts in an effort to host multiple builds of the same templates in
one directory while maintaining enough information about the box within
the filename itself.

Using the same example as above, the VirtualBox provider box name is:

    chef__ubuntu-14.10-i386-2.0.20150528211301.git.6b23dd8d8ff0fb9cc4473f510bc3c54f0b415d1b.virtualbox.box

Which uses the following recipe to construct the filename:

* `atlas_org` value (default: `"chef"`)
* double underscore, which could be later interpreted as a slash (`/`)
  for an Atalas box name
* `name` value which may or may not equal the name of the template
  (captured as the `template` value)
* a dash
* `version` value, which removes the last digit in a version string and
  replaces it with the `build_timestamp` (a
  Year/Month/Day/Hour/Minute/Second format in UTC timezone)
* a period
* the string `"git"`
* a period
* `git_revision` value, which will append `"_dirty"` if the current
  state of the git repository is not completely clean (i.e., there are
  uncommitted changes which happens in active development)
* a period
* the value of the `{{.Provider}}` Packer variable, being one of
  `"virtualbox"`, `"vmware"`, or `"parallels"`
* finished with `".box"`

Closes #364
@fnichol
Copy link
Contributor Author

fnichol commented May 29, 2015

@yzl Ready for a review (it's a bit big, but self-contained)!

@scotthain Any thoughts as well?

@scotthain
Copy link

This looks amazing. My only question (and this may be because I don't have all the context) is what is the goal of having these files? (If that's a long conversation that you and yzl have already figured out, don't worry about it!)

Either way, having the metadata inside the box itself is awesome.

@yzl
Copy link

yzl commented May 29, 2015

@scotthain It's to support a build pipeline for bento boxes, similar to the omnibus pipelines where we have a stage that builds all the packages associated to an artifact, a stage that tests all the packages, and a stage that releases them.

@yzl
Copy link

yzl commented May 29, 2015

👍 This is awesome!

@scotthain
Copy link

@yzl freaking awesome. That was my suspicion!

fnichol added a commit that referenced this pull request May 29, 2015
Add build metadata to boxes and build artifacts.
@fnichol fnichol merged commit 526662d into master May 29, 2015
@fnichol fnichol removed the Waiting label May 29, 2015
@fnichol fnichol deleted the metadata branch May 29, 2015 19:45
@fnichol fnichol mentioned this pull request May 29, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add build metadata to boxes and build artifacts
3 participants