fix(hash): recreate container on project config content change #11931

idsulik · 2024-06-23T16:11:50Z

What I did
Fixed hash.ServiceHash() to support config content change

Related issue
#11900

ndeloof · 2024-07-01T14:21:07Z

While I understand the intent, I don't like we get the config content added into the service hash. This also only makes sense as the config content is inlined.
I wonder we could rely on a label to track the config state by the time it was created : com.docker.compose.config.name=<config hash>.

ndeloof · 2024-07-02T07:02:07Z

pkg/compose/hash.go

+
+	for _, serviceConfig := range o.Configs {
+		if projectConfig, ok := project.Configs[serviceConfig.Source]; ok {
+			bytes = append(bytes, []byte(projectConfig.Content)...)


This won't detect changes for config set by environment of file

added support for env and config file

idsulik · 2024-07-06T18:32:39Z

I don't like we get the config content added into the service hash

why?

I wonder we could rely on a label to track the config state by the time it was created : com.docker.compose.config.name=<config hash>.

I don't get the idea, time it was created - do you mean config file? or docker-compose.yaml? but docker-compose can refer to external config file

ndeloof · 2024-07-08T15:01:12Z

time container was created, so we can check it needs to be recreated if current config doesn't match

jhrotko · 2024-07-09T15:34:32Z

pkg/compose/hash.go

@@ -37,5 +38,22 @@ func ServiceHash(o types.ServiceConfig) (string, error) {
 	if err != nil {
 		return "", err
 	}
+
+	for _, serviceConfig := range o.Configs {
+		if projectConfig, ok := project.Configs[serviceConfig.Source]; ok {


nit: I would invert the if statements so that we do not get the code so nested. Other than that looks good to me!

@jhrotko done

ndeloof · 2024-07-10T06:12:59Z

Some poins:

At some point we will have to do the same for secrets (same constraints) and networks/volumes (which can get labeled with a has). Solution we adopt here should be extensible to allow this in the future
Configs set by file are implemented by a bind mount but this may change in near future (see Enable configs.file's on remote docker hosts #11871) so we also need to consider those
There's no technical reason docker engine can't offer secrets/configs natively, just this is guarded by Swarm mode, but AFAIK some discussion happened to get them also available in standalone mode. I can't tell if/when this would take place, but preferably the logic here should consider this may happen
Last but not least, I'd prefer we don't mix service config hash with resources it depends on, so my suggestion to introduce an additional label com.docker.compose.config.xx=hash that we can use to track this relation, and need to recreate container, without the need to change the service hash computation (which as impact on existing installations)

idsulik · 2024-07-11T16:37:10Z

@ndeloof thanks for the details. pushed changes:

reverted old changes
added new func ServiceDependenciesHash and label

// ConfigHashDependenciesLabel stores configuration hash for a compose service dependencies
ConfigHashDependenciesLabel = "com.docker.compose.config-hash-dependencies"

Let me know if you have better idea for the label name, because I'm not fully satisfied with my name)

ndeloof · 2024-07-22T08:50:16Z

pkg/compose/hash.go

+		if projectConfig.Content != "" {
+			bytes = append(bytes, []byte(projectConfig.Content)...)
+		} else if projectConfig.File != "" {
+			content, err := os.ReadFile(projectConfig.File)


File could also be a folder. Maybe an option would be to read content into a tar (code exists to inject secrets) and compute hash from it

While the general approach looks good, I'd prefer we have a more explicit hash and label per resource, and a dedicated hash function for configs (and secrets), so that it make it easier to diagnose use of those labels

@ndeloof thank you the comment. Pushed fixes: added secrets hash, added separate hashes for configs and secrets, added support for file.File directory

Signed-off-by: Suleiman Dibirov <idsulik@gmail.com>

This reverts commit 64c37bf. Signed-off-by: Suleiman Dibirov <idsulik@gmail.com>

Signed-off-by: Suleiman Dibirov <idsulik@gmail.com>

…older support Signed-off-by: Suleiman Dibirov <idsulik@gmail.com>

idsulik · 2024-09-16T08:32:54Z

Hi @ndeloof , what do you think about thinking about the PR?

ndeloof · 2024-09-18T07:31:22Z

pkg/compose/hash.go

+				return err
+			}
+			if !info.IsDir() {
+				fileContent, err := os.ReadFile(path)


this will not detect file being rename. Maybe better to create a tar stream from file/dir content

@ndeloof it creates a tar from the content here pkg/compose/hash.go:122.

b, err := utils.CreateTar(content, serviceConfig, modTime)

ndeloof reviewed Jul 2, 2024

View reviewed changes

idsulik force-pushed the issue-11900 branch from 19daddd to f75d940 Compare July 6, 2024 18:30

jhrotko reviewed Jul 9, 2024

View reviewed changes

idsulik force-pushed the issue-11900 branch from b67627e to b506004 Compare July 11, 2024 16:35

idsulik force-pushed the issue-11900 branch from b506004 to 17237d4 Compare July 12, 2024 06:54

schaubl mentioned this pull request Jul 17, 2024

Support remote DOCKER_HOST by copying configs and secrets files instead of bind mounting them #11867

Open

idsulik requested review from ndeloof and jhrotko July 22, 2024 08:03

ndeloof reviewed Jul 22, 2024

View reviewed changes

idsulik force-pushed the issue-11900 branch from f7667bb to 878fea5 Compare July 22, 2024 16:30

idsulik requested a review from ndeloof July 22, 2024 16:32

idsulik added 4 commits September 16, 2024 11:32

fix(hash): recreate container on project config content change

ce9b022

Signed-off-by: Suleiman Dibirov <idsulik@gmail.com>

Revert "fix(hash): recreate container on project config content change"

fbb04d5

This reverts commit 64c37bf. Signed-off-by: Suleiman Dibirov <idsulik@gmail.com>

feat: Add service dependencies hash

381d24b

Signed-off-by: Suleiman Dibirov <idsulik@gmail.com>

feat: add separate hashes for service configs and secrets, add file f…

2aa70c0

…older support Signed-off-by: Suleiman Dibirov <idsulik@gmail.com>

idsulik force-pushed the issue-11900 branch from 284468c to 2aa70c0 Compare September 16, 2024 08:34

ndeloof reviewed Sep 18, 2024

View reviewed changes

aequitas mentioned this pull request Sep 18, 2024

Changes in Docker compose configs not propagated to containers during compose up internetstandards/Internet.nl#1490

Open

idsulik requested a review from ndeloof September 19, 2024 04:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(hash): recreate container on project config content change #11931

fix(hash): recreate container on project config content change #11931

idsulik commented Jun 23, 2024 •

edited

Loading

ndeloof commented Jul 1, 2024

ndeloof Jul 2, 2024

idsulik Jul 6, 2024

idsulik commented Jul 6, 2024

ndeloof commented Jul 8, 2024

jhrotko Jul 9, 2024 •

edited

Loading

idsulik Jul 10, 2024

ndeloof commented Jul 10, 2024

idsulik commented Jul 11, 2024 •

edited

Loading

ndeloof Jul 22, 2024

idsulik Jul 22, 2024

idsulik commented Sep 16, 2024 •

edited

Loading

ndeloof Sep 18, 2024

idsulik Sep 18, 2024

fix(hash): recreate container on project config content change #11931

Are you sure you want to change the base?

fix(hash): recreate container on project config content change #11931

Conversation

idsulik commented Jun 23, 2024 • edited Loading

ndeloof commented Jul 1, 2024

ndeloof Jul 2, 2024

Choose a reason for hiding this comment

idsulik Jul 6, 2024

Choose a reason for hiding this comment

idsulik commented Jul 6, 2024

ndeloof commented Jul 8, 2024

jhrotko Jul 9, 2024 • edited Loading

Choose a reason for hiding this comment

idsulik Jul 10, 2024

Choose a reason for hiding this comment

ndeloof commented Jul 10, 2024

idsulik commented Jul 11, 2024 • edited Loading

ndeloof Jul 22, 2024

Choose a reason for hiding this comment

idsulik Jul 22, 2024

Choose a reason for hiding this comment

idsulik commented Sep 16, 2024 • edited Loading

ndeloof Sep 18, 2024

Choose a reason for hiding this comment

idsulik Sep 18, 2024

Choose a reason for hiding this comment

idsulik commented Jun 23, 2024 •

edited

Loading

jhrotko Jul 9, 2024 •

edited

Loading

idsulik commented Jul 11, 2024 •

edited

Loading

idsulik commented Sep 16, 2024 •

edited

Loading