aws codeartifact login does not work with PyYAML v6

[agoblet]

Describe the bug

Recently, support for PyYAML v6 has been added. Some parts of the awscli do not work with that PyYAML version though. When trying to login to CodeArtifact, awscli raises an exception. This does not happen when downgrading to PyYAML v5.4.

Expected Behavior

I expected awscli to just login to codeartifact without any errors, like it does when PyYAML v5.4 is installed.

Current Behavior

Traceback (most recent call last):
  File "/opt/conda/bin/aws", line 27, in <module>
    sys.exit(main())
  File "/opt/conda/bin/aws", line 23, in main
    return awscli.clidriver.main()
  File "/opt/conda/lib/python3.7/site-packages/awscli/clidriver.py", line 69, in main
    driver = create_clidriver()
  File "/opt/conda/lib/python3.7/site-packages/awscli/clidriver.py", line 79, in create_clidriver
    event_hooks=session.get_component('event_emitter'))
  File "/opt/conda/lib/python3.7/site-packages/awscli/plugin.py", line 44, in load_plugins
    modules = _import_plugins(plugin_mapping)
  File "/opt/conda/lib/python3.7/site-packages/awscli/plugin.py", line 61, in _import_plugins
    module = __import__(path, fromlist=[module])
  File "/opt/conda/lib/python3.7/site-packages/awscli/handlers.py", line 27, in <module>
    from awscli.customizations.cloudformation import initialize as cloudformation_init
  File "/opt/conda/lib/python3.7/site-packages/awscli/customizations/cloudformation/__init__.py", line 13, in <module>
    from awscli.customizations.cloudformation.package import PackageCommand
  File "/opt/conda/lib/python3.7/site-packages/awscli/customizations/cloudformation/package.py", line 22, in <module>
    from awscli.customizations.cloudformation.artifact_exporter import Template
  File "/opt/conda/lib/python3.7/site-packages/awscli/customizations/cloudformation/artifact_exporter.py", line 27, in <module>
    from awscli.customizations.cloudformation.yamlhelper import yaml_dump, \
  File "/opt/conda/lib/python3.7/site-packages/awscli/customizations/cloudformation/yamlhelper.py", line 17, in <module>
    from yaml.resolver import ScalarNode, SequenceNode
ModuleNotFoundError: No module named 'yaml.resolver'

Reproduction Steps

aws codeartifact login --tool pip --domain abc --domain-owner 123456789012 --repository abcd

Possible Solution

make the yamlhelper compatible with PyYAML v6

Additional Information/Context

It would be great if awscli would support PyYAML v6, because other AWS packages such as sagemaker depend on PyYAML v6.

CLI version used

1.29.5

Environment details (OS name and version, etc.)

AWS Sagemaker SparkMagic kernel, awscli==1.29.5, PyYAML==6.0.1

[agoblet]

The issues with yaml.resolver seem to happen in other parts of awscli as well, not just when authenticating to CodeArtfiact.

[jonathan343]

Hey @agoblet!

This issue wasn't reproducible with the given command. I was able to make the call successfully:

$ aws codeartifact login --tool pip --domain my-domain --domain-owner 1234567890 --repository my-repo
Successfully configured pip to use AWS CodeArtifact repository https://my-domain-1234567890.d.codeartifact.us-west-2.amazonaws.com/pypi/my-repo/ 
Login expires in 12 hours at 2023-07-19 20:48:17-07:00
$ aws --version
aws-cli/1.29.5 Python/3.11.1 Darwin/21.6.0 botocore/1.31.5

Can you provide debug logs (with any sensitive info redacted) by adding --debug to the command thats failing for you?
Can you also provide some specifics about how the AWS CLI is being installed? If this is being done using pip can you provide the output for pip freeze?

[agoblet]

hey @jonathan343 after your message I tried the same in some other Sagemaker notebook kernels, and it appears that this problem only occurs in SparkMagic kernel. I need to do some additional research why this is the case. I'll get back to you once I know more. Will close the issue if I find out that it is not awscli related.

[agoblet]

reinstalling PyYAML v6 in the environment seems to solve the issue. The problem seems to be inside the kernel image rather than in awscli. Will close the issue, thanks for your help!

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[eugenefm]

@agoblet seeing the same thing when running the cli commands in Github Actions. Tried installing PyYAML along with the CLI in the workflow and still seeing the same issue.

[agoblet]

@eugenefm how did you install the packages? If using pip, did you use the —force-reinstall flag?

[eugenefm]

@eugenefm how did you install the packages? If using pip, did you use the —force-reinstall flag?

Using pip but did not use the -force-reinstall flag. Currently the script in my workflow looks like this:

  apt install python3 python3-pip -y
  pip3 --no-cache-dir install --upgrade awscli PyYAML

Should I try something like this instead, to force a reinstall of PyYAML after the awscli install?

  pip3 --no-cache-dir install --upgrade awscli
  pip3 --no-cache-dir install --force-reinstall PyYAML

[agoblet]

For me that solved the problem, can you try it?