How to manage properly copier.yml using composition ?

[achaussier]

Hello,

I've some questions about copier.yml composition, using !include. You can find above my different tries, to have the context.

Targets

1. Include common default values from a shared default values repository (for YAML type values)
2. Include common sets of questions from a shared questions repository (one set per file)

The first point can be done quickly using !include as default value, and works well:

# copier.yml
...
environments:
  default: !include "shared-resources/config/default_values/environments.yml"
  type: yaml
...

But "problems" come with the second point, how to include top level keys (as questions are) into copier.yml file.

Some tries to include top level content

Method 1: mixed content

Firstly, I've think about using composition mixing top level !include statements, but after some tries, it's not possible (see #236 ).

Result: KO

Method 2: multiple top level !include

# copier.yml
!include "copier.yml.d/base.yml"
!include "shared-resources/config/questions/project_identity.yml"

But, not the expected output :)

Traceback (most recent call last):
  File "/home/achaussier/.local/lib/python3.7/site-packages/copier/config/user_data.py", line 54, in load_yaml_data
    return yaml.load(f, Loader=yaml.FullLoader)
  File "/home/achaussier/.local/lib/python3.7/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/home/achaussier/.local/lib/python3.7/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/home/achaussier/.local/lib/python3.7/site-packages/yaml/composer.py", line 39, in get_single_node
    if not self.check_event(StreamEndEvent):
  File "/home/achaussier/.local/lib/python3.7/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/home/achaussier/.local/lib/python3.7/site-packages/yaml/parser.py", line 174, in parse_document_start
    self.peek_token().start_mark)
yaml.parser.ParserError: expected '<document start>', but found '<tag>'
  in "tpl-terraform/copier.yml", line 2, column 1

YAML parser exception seems say : One top level !include per file, right ?

Result: KO

Method 3: Multiple files

Another try ...

# copier.yml
!include "copier.yml.d/*.yml"

# copier.yml.d/base.yml
...
_exclude:
  - .git
  - "*.swo"
  - "*.swp"
_min_copier_version: "4.0.2"
...

# copier.yml.d/project_identity.yml
!include "shared-resources/config/questions/project_identity.yml"

... but same result, because as the pyyaml-include documentation says:

File name can contain shell-style wildcards. Data loaded from the file(s) found by wildcards will be set in a sequence.

Result: KO

Questions time

The documentation says in the Include other yaml files section: *Multiple files can be included per copier.yml.

So,

• can we use multiple !include statements as values inside the copier.yml only ?
• can have only one conflguration file, without top level !include ?
• it is possible to import questions from different sources ?

My tries seem answer yes, yes, no, but is it right ?

If this is not possible today, I think the ability to import question can be really useful to keep config file smaller and avoid duplication between templates for common questions (smtp, repository, environments, licenses, ...).

Have a nice day !