Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ngclient: simplify storing a downloaded file #1799

Merged
merged 1 commit into from
Jan 27, 2022

Conversation

MVrachev
Copy link
Collaborator

@MVrachev MVrachev commented Jan 26, 2022

Fixes #1761

Description of the changes being introduced by the pull request:

Replace the usage of securesystemslib.util.persist_temp_file() with
shutil.copyfileobj() as file system abstraction is not used in the
client.
This way we prevent securesystemslib.exception.StorageError from
leaking through client API calls.

Note: with those changes, we are no longer do fsync.

Signed-off-by: Martin Vrachev mvrachev@vmware.com

Please verify and check that the pull request fulfills the following
requirements
:

  • The code follows the Code Style Guidelines
  • Tests have been added for the bug fix or new feature
  • Docs have been added for the bug fix or new feature

@coveralls
Copy link

coveralls commented Jan 26, 2022

Pull Request Test Coverage Report for Build 1756649124

  • 4 of 4 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.002%) to 97.691%

Totals Coverage Status
Change from base Build 1756435191: 0.002%
Covered Lines: 4085
Relevant Lines: 4166

💛 - Coveralls

Copy link
Member

@jku jku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks correct to me.

To make it more obvious you could mention in the commit message that this means we no longer do fsync. It believe that's fine in our use case but could be worth mentioning.

@MVrachev
Copy link
Collaborator Author

@jku I updated pr, so it changes the exceptions docstring inside updater.download_target().

Additionally, I made it dependent on #1787 to make sure we end up with the correct exception docstring for updater.download_target().

Copy link
Member

@jku jku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is correct, left a comment about the docstring

@@ -218,8 +217,7 @@ def download_target(
ValueError: Invalid arguments
DownloadError: Download of the target file failed in some way
RepositoryError: Downloaded target failed to be verified in some way
exceptions.StorageError: Downloaded target could not be written
to disk
OSError: Cannot open "filepath" for writing.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strictly speaking there are quite a few reasons for OSError here, not just failing to open file (like not enough space as an example).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the message.
Do you think that now is better?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure there was anything wrong with the original message? Alternatively "Failed to write target to file" could work.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understood you had some reservations about it.
I updated the pr with your suggestion.

Replace the usage of securesystemslib.util.persist_temp_file() with
shutil.copyfileobj() as file system abstraction is not used in the
client.
This way we prevent securesystemslib.exception.StorageError from
leaking through client API calls.

Note: with those changes we are no longer do fsync.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Copy link
Member

@jku jku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks.

If another maintainer wants to chime in on the issue of not using persist_temp_file() (and as a result, not fsyncing) that would be welcome: I've tried to think the failure scenarios through and this seems like the simplest code that does what we want.

@lukpueh lukpueh merged commit 4de5617 into theupdateframework:develop Jan 27, 2022
@MVrachev MVrachev deleted the rm-persist_temp_file branch January 27, 2022 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ngclient: throws securesystemslib errors
4 participants