Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use MD5Sum of the tarfile as the task version #931

Merged
merged 5 commits into from
Apr 12, 2022
Merged

Conversation

EngHabu
Copy link
Collaborator

@EngHabu EngHabu commented Apr 12, 2022

Signed-off-by: Haytham Abuelfutuh haytham@afutuh.com

TL;DR

In script mode, we rely on the hash of the generated tar file to idempotent-ly register tasks/workflows. Because tar file metadata changes every time, we end up with a different version S3 path for the task. Because we use the script md5 hash as the task version, admin rejects the registration because the content of the task proto changed.

This PR unifies versioning logic to rely on the tar file content md5 for the task version

Type

  • Bug Fix
  • Feature
  • Plugin

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>
@codecov
Copy link

codecov bot commented Apr 12, 2022

Codecov Report

Merging #931 (3db8121) into master (a6363ca) will decrease coverage by 0.07%.
The diff coverage is 27.27%.

❗ Current head 3db8121 differs from pull request most recent head 933c86c. Consider uploading reports for the commit 933c86c to get more accurate results

@@            Coverage Diff             @@
##           master     #931      +/-   ##
==========================================
- Coverage   85.82%   85.74%   -0.08%     
==========================================
  Files         246      246              
  Lines       23494    23486       -8     
  Branches     2655     2321     -334     
==========================================
- Hits        20164    20139      -25     
- Misses       2888     2898      +10     
- Partials      442      449       +7     
Impacted Files Coverage Δ
flytekit/remote/remote.py 35.01% <0.00%> (-0.07%) ⬇️
flytekit/tools/script_mode.py 25.75% <29.41%> (+0.30%) ⬆️
flytekit/clis/sdk_in_container/run.py 37.06% <50.00%> (+1.23%) ⬆️
tests/flytekit/unit/extras/tasks/test_shell.py 90.09% <0.00%> (-9.91%) ⬇️
flytekit/extras/persistence/http.py 63.26% <0.00%> (-6.13%) ⬇️
flytekit/extras/tasks/shell.py 85.15% <0.00%> (-0.79%) ⬇️
flytekit/configuration/internal.py 95.16% <0.00%> (-0.50%) ⬇️
flytekit/models/security.py 86.84% <0.00%> (-0.34%) ⬇️
flytekit/types/schema/types.py 76.63% <0.00%> (-0.10%) ⬇️
flytekit/types/file/file.py 90.90% <0.00%> (-0.07%) ⬇️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a6363ca...933c86c. Read the comment docs.

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>
wild-endeavor
wild-endeavor previously approved these changes Apr 12, 2022
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
wild-endeavor
wild-endeavor previously approved these changes Apr 12, 2022
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
@wild-endeavor wild-endeavor merged commit 7718f67 into master Apr 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants