Merge pull request #2859 from lucywyman/GH-2815

(GH-2815) Support plan-hierarchy lookups on the CLI
puppetlabs · Jun 10, 2021 · c3299e3 · c3299e3
2 parents eb9e014 + 9dabe63
commit c3299e3
Show file tree

Hide file tree

Showing 16 changed files with 377 additions and 194 deletions.
diff --git a/documentation/hiera.md b/documentation/hiera.md
@@ -5,8 +5,9 @@ built-in key-value configuration data lookup system, which lets you separate
 data from your code. There are some key differences in how Bolt and Puppet use
 Hiera, which this page covers in more detail.
 
-Before you start using Hiera in Bolt, get familiar with [how to use
-Hiera](https://puppet.com/docs/puppet/latest/hiera_intro.html).
+Before you start using Hiera in Bolt, get familiar with:
+- [How to use Hiera](https://puppet.com/docs/puppet/latest/hiera_intro.html).
+- [Applying Puppet code](applying_manifest_blocks.html) with Bolt.
 
 ## Hiera configuration layers
 
@@ -28,12 +29,6 @@ You can specify a different configuration file for the project layer by setting
 the `hiera-config` option in your project configuration, or by using the
 `--hiera-config` command-line option.
 
-📖  **Related information**
-
-- [bolt-project.yaml options](bolt_project_reference.md#hiera-config)
-- [*nix shell commands](bolt_command_reference.md)
-- [PowerShell cmdlets](bolt_cmdlet_reference.md)
-
 ### Module layer
 
 The configuration for a module layer is located, by default, in the module's
@@ -43,89 +38,17 @@ The module layer sets default values and merge behavior for a module’s class
 parameters. The module layer comes last in Hiera’s lookup order, so environment
 data set by a user overrides the default data set by the module’s author.
 
-## Look up data from the command line
-
-You can use the `bolt lookup` command and `Invoke-BoltLookup` PowerShell cmdlet
-to look up data from the command line. The `lookup` command looks up data in the
-context of a target, allowing you to interpolate target facts and variables in
-your hierarchy.
-
-> **Note:** The `bolt lookup` and `Invoke-BoltLookup` commands only look up data
-> using the `hierarchy` key in the Hiera configuration file. `plan_hierarchy`
-> is not supported from the command line.
-
-When you run the `bolt lookup` and `Invoke-BoltLookup` commands, Bolt first
-runs an `apply_prep` on each of the targets specified. This installs the
-`puppet-agent` package on the target, collects facts, and then stores the facts
-on the target to be used in interpolations.
-
-Looking up data from the command line is particularly useful if you need to
-debug a plan that includes calls to the `lookup()` function, or if you need to
-look up target-specific data such as a password for authenticating connections
-to the target.
-
-Given the following Hiera configuration at `<PROJECT DIRECTORY>/hiera.yaml`:
-
-```yaml
-# hiera.yaml
-version: 5
-
-hierarchy:
-  - name: "Per-OS defaults"
-    path: "os/%{facts.os.name}.yaml"
-  - name: "Common data"
-    path: "common.yaml"
-```
-
-And the following data source at `<PROJECT DIRECTORY>/data/os/Windows.yaml`:
-
-```yaml
-# data/os/Windows.yaml
-password: Bolt!
-```
-
-And the following data source at `<PROJECT DIRECTORY>/data/os/Ubuntu.yaml`:
-
-```yaml
-# data/os/Ubuntu.yaml
-password: Puppet!
-```
-
-You can look up the value for the `password` key from the command line using
-facts collected from your targets:
-
-_\*nix shell command_
-
-```shell
-bolt lookup password --targets windows_target,ubuntu_target
-```
-
-_PowerShell cmdlet_
-
-```powershell
-Invoke-BoltLooup -Key 'password' -Targets 'windows_target,ubuntu_target'
-```
-
-Bolt prints the value for the key to the console:
-
-```shell
-Starting: install puppet and gather facts on windows_target, ubuntu_target
-Finished: install puppet and gather facts with 0 failures in 6.7 sec
-Finished on windows_target:
-  Bolt!
-Finished on ubuntu_target:
-  Puppet!
-Successful on 2 targets: windows_target, ubuntu_target
-Ran on 2 targets
-```
+📖  **Related information**
 
+- [bolt-project.yaml options](bolt_project_reference.md#hiera-config)
+- [*nix shell commands](bolt_command_reference.md)
+- [PowerShell cmdlets](bolt_cmdlet_reference.md)
 ## Look up data in plans
 
 You can use the [Puppet `lookup()`
 function](https://puppet.com/docs/puppet/latest/hiera_automatic.html#puppet_lookup)
 in plans to look up data. It's useful to think of looking up Hiera data in Bolt
-plans in two different contexts: [inside apply
-blocks](applying_manifest_blocks.md) and outside apply blocks.
+plans in two different contexts: inside apply blocks and outside apply blocks.
 
 ### Inside apply blocks
 
@@ -282,8 +205,8 @@ And the following data source at `<PROJECT DIRECTORY>/data/static.yaml`:
 api_key: 12345
 ```
 
-The previous plan will now run successfully, as Hiera will look up the `api_key`
-key using the `plan_hierarchy`, which does not include interpolations.
+The plan run now succeeds, because Hiera uses the `plan_hierarchy` key to look
+up the static value for `api_key`.
 
 By specifying both keys in the same Hiera configuration, you can look up data
 inside and outside apply blocks in the same plan. This allows you to use your
@@ -293,8 +216,8 @@ apply block.
 
 ### Interpolations outside apply blocks
 
-Interpolations are not well supported in the `plan_hierarchy` hierarchy. Target
-level data such as facts are not available to lookups outside of apply blocks,
+Interpolations are not well supported in the `plan_hierarchy` hierarchy.
+Target-level data such as facts are not available to lookups outside of apply blocks,
 so the normal hierarchy interpolation does not work. The `lookup()` function
 only interpolates based on the variables currently in scope.
 
@@ -312,14 +235,14 @@ plan_hierarchy:
     path: "%{application}.yaml"
 ```
 
-And the following data source at `<PROJECT DIRECTORY>/data/kittycats.yaml`:
+A data source at `<PROJECT DIRECTORY>/data/kittycats.yaml`:
 
 ```yaml
 # data/kittycats.yaml
 site_path: /var/www/kittycats.tld/public_html
 ```
 
-And the following data source at `<PROJECT DIRECTORY>/data/doggos.yaml`:
+And a data source at `<PROJECT DIRECTORY>/data/doggos.yaml`:
 
 ```yaml
 # data/doggos.yaml
@@ -339,8 +262,8 @@ plan plan_lookup(
 }
 ```
 
-Note that if you tried to call `lookup('site_path')` from a subplan of
-`plan_lookup`, like so:
+Note that if you tried to call `lookup('site_path')` from a sub-plan of
+`plan_lookup`:
 
 ```puppet
 # plans/plan_lookup.pp
@@ -365,3 +288,105 @@ plan other_plan(
 
 Bolt would error, because `$application` is not in scope in `other_plan` and the
 interpolation in the Hiera configuration would fail.
+
+## Look up data from the command line
+
+You can use the `bolt lookup` command and `Invoke-BoltLookup` PowerShell cmdlet
+to look up Hiera data from the command line. By default, Bolt uses the `hierarchy`
+key in your `hiera.yaml` file to look up data and performs the lookup inside an apply
+block. Similar to looking up data outside of an apply block in a plan, you can use
+the `--plan-hierarchy` or `-PlanHierarchy` command options to look up data from 
+the `plan_hierarchy` key. 
+
+### Inside apply blocks
+
+Without additional options, the `lookup` command looks up data in the context of a target,
+allowing you to interpolate target facts and variables in your hierarchy.
+
+When you run the `bolt lookup` and `Invoke-BoltLookup` commands, Bolt first
+runs an `apply_prep` on each of the targets specified. This installs the
+`puppet-agent` package on the target, collects facts, and then stores the facts
+on the target to be used in interpolations.
+
+Looking up data from the command line is particularly useful if you need to
+debug a plan that includes calls to the `lookup()` function, or if you need to
+look up target-specific data such as a password for authenticating connections
+to the target.
+
+Given the following Hiera configuration at `<PROJECT DIRECTORY>/hiera.yaml`:
+
+```yaml
+# hiera.yaml
+version: 5
+
+hierarchy:
+  - name: "Per-OS defaults"
+    path: "os/%{facts.os.name}.yaml"
+  - name: "Common data"
+    path: "common.yaml"
+```
+
+A data source at `<PROJECT DIRECTORY>/data/os/Windows.yaml`:
+
+```yaml
+# data/os/Windows.yaml
+password: Bolt!
+```
+
+And a data source at `<PROJECT DIRECTORY>/data/os/Ubuntu.yaml`:
+
+```yaml
+# data/os/Ubuntu.yaml
+password: Puppet!
+```
+
+You can look up the value for the `password` key from the command line using
+facts collected from your targets:
+
+_\*nix shell command_
+
+```shell
+bolt lookup password --targets windows_target,ubuntu_target
+```
+
+_PowerShell cmdlet_
+
+```powershell
+Invoke-BoltLooup -Key 'password' -Targets 'windows_target,ubuntu_target'
+```
+
+Bolt prints the value for the key to the console:
+
+```shell
+Starting: install puppet and gather facts on windows_target, ubuntu_target
+Finished: install puppet and gather facts with 0 failures in 6.7 sec
+Finished on windows_target:
+  Bolt!
+Finished on ubuntu_target:
+  Puppet!
+Successful on 2 targets: windows_target, ubuntu_target
+Ran on 2 targets
+```
+
+### Outside apply blocks
+
+To look up data from Hiera's `plan_hierarchy` key, use the `lookup` command with
+the `--plan-hierarchy` option. This mimics performing a lookup outside an apply block
+in a Bolt plan.
+
+Because `plan_hierarchy` values are not specific to individual targets, this command performs
+just one lookup and returns the bare value. You can't use the `--plan-hierarchy` option
+with the `--targets`, `--rerun`, or `--query` options.
+
+If your `plan_hierarchy` contains [interpolations from plan
+variables](#interpolations-outside-apply-blocks), you can pass values to interpolate to `lookup`:
+
+_\*nix shell command_
+```
+bolt lookup key --plan-hierarchy plan_var=interpolate_me
+```
+
+_PowerShell cmdlet_
+```
+Invoke-BoltLookup -Key key -PlanHierarchy plan_var=interpolate_me
+```
diff --git a/lib/bolt/bolt_option_parser.rb b/lib/bolt/bolt_option_parser.rb
@@ -67,7 +67,7 @@ def get_help_text(subcommand, action = nil)
         { flags: OPTIONS[:global] + %w[format],
           banner: GUIDE_HELP }
       when 'lookup'
-        { flags: ACTION_OPTS + %w[hiera-config],
+        { flags: ACTION_OPTS + %w[hiera-config plan-hierarchy],
           banner: LOOKUP_HELP }
       when 'module'
         case action
@@ -417,7 +417,7 @@ module            Manage Bolt project modules
           lookup
 
       #{colorize(:cyan, 'Usage')}
-          bolt lookup <key> {--targets TARGETS | --query QUERY | --rerun FILTER}
+          bolt lookup <key> {--targets TARGETS | --query QUERY | --rerun FILTER | --plan-hierarchy}
             [options]
 
       #{colorize(:cyan, 'Description')}
@@ -428,6 +428,7 @@ module            Manage Bolt project modules
 
       #{colorize(:cyan, 'Examples')}
           bolt lookup password --targets servers
+          bolt lookup password --plan-hierarchy variable=value
     HELP
 
     MODULE_HELP = <<~HELP
@@ -1029,6 +1030,11 @@ def initialize(options)
         @options[:resolve] = resolve
       end
 
+      separator "\n#{self.class.colorize(:cyan, 'Lookup options')}"
+      define('--plan-hierarchy', 'Look up a value with Hiera in the context of a specific plan.') do |_|
+        @options[:plan_hierarchy] = true
+      end
+
       separator "\n#{self.class.colorize(:cyan, 'Plan options')}"
       define('--pp', 'Create a new Puppet language plan.') do |_|
         @options[:puppet] = true