packaging plugin #254

tomerd · 2022-03-18T07:37:33Z

motivation: an easy way to archive lambda for uploading to AWS

changes:

add SwiftPM plugin with the verb "archive"
on Amazon Linux, build and package directly using the plugin bridge into SwiftPM
on other platforms use docker to build and package the lambda(s)

tomerd · 2022-03-18T07:45:46Z

cc @abertelrud - this requires escaping the sandbox because of docker. would appreciate your feedback on the direction here

fabianfett

Great start. Love to see this arriving! A couple of quick thoughts.

Examples/Echo/Package.swift

Package@swift-5.2.swift

Plugins/AWSLambdaPackager/Plugin.swift

Package@swift-5.4.swift

Package@swift-5.5.swift

Plugins/AWSLambdaPackager/Plugin.swift

tomerd · 2022-04-15T19:13:26Z

@fabianfett I think this is ready to merge as iteration 1. next iteration we can explore how to integrate ziplib

fabianfett

As part of my review, I tried to run it in the EchoHandler... However I got the following output:

➜  Echo git:(feature/packaging-plugin) ✗ swift package archive     
-------------------------------------------------------------------------
building "echo" in docker
-------------------------------------------------------------------------
error: processFailed(126)

I think we should at least print the processes output, in the error case. This error is undebugable for me.

Examples/Echo/Package.swift

tomerd · 2022-04-18T17:04:57Z

I think we should at least print the processes output

fixed

fabianfett · 2022-04-19T06:50:09Z

Interesting, just pulled the newest version and tried to rerun:

➜  Echo git:(feature/packaging-plugin) ✗ swift package archive
-------------------------------------------------------------------------
building "echo" in docker
-------------------------------------------------------------------------
docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/create": dial unix /var/run/docker.sock: connect: operation not permitted.
See 'docker run --help'.
error: /usr/local/bin/docker run --rm -v /Users/fabian/Developer/swift-server/swift-aws-lambda-runtime/Examples/Echo:/workspace -w /workspace swift:amazonlinux2 bash -cl swift build -c release --show-bin-path failed with code 126
➜  Echo git:(feature/packaging-plugin) ✗ docker container create swift:5.6-focal
98decac8da409010202c45428d18e06f9761f1b130c649afc14778b6d74aa72b
➜  Echo git:(feature/packaging-plugin) ✗

Apparently from the plugin I don't have enough rights to create a new container, but just from the shell I have. Do you have any idea what this might be related to @tomerd.

stevapple · 2022-04-19T07:24:02Z

I think the reason’s here:
https://github.com/apple/swift-evolution/blob/55f9bb7eab8e89bb8cd2794083aba101ac07777a/proposals/0303-swiftpm-extensible-build-tools.md?plain=1#L1018-L1020

Wonder if we can surpass the restriction.

tomerd · 2022-04-19T20:19:52Z

right, this is because talking to docker is networking which is denied by the sandbox. for local testing I have been using the plugin without the sandbox, e.g: swift package --disable-sandbox --package-path Examples/Echo archive

@abertelrud would be interesting to see if we can poke a hole for docker in the sandbox, or be able to express it as a plugin requirement that the user can approve

tomerd · 2022-05-10T18:42:24Z

@fabianfett ?

motivation: add an easy wasy for lambda users to package their lambda and upload it to AWS changes: * add SwiftPM plugin with the verb "archive" * use docker to build and package the lambda(s)

Co-authored-by: Yim Lee <yim_lee@apple.com>

fabianfett

Looks good and works for me!

Package@swift-5.6.swift

fabianfett · 2022-05-16T21:05:37Z

Plugins/AWSLambdaPackager/Plugin.swift

+            // rename artifact to "bootstrap"
+            let relocatedArtifactPath = workingDirectory.appending(artifactPath.lastComponent)
+            let symbolicLinkPath = workingDirectory.appending("bootstrap")
+            try FileManager.default.copyItem(atPath: artifactPath.string, toPath: relocatedArtifactPath.string)
+            try FileManager.default.createSymbolicLink(atPath: symbolicLinkPath.string, withDestinationPath: relocatedArtifactPath.lastComponent)


Code comment does not match behavior. I wonder if just renaming the executable (like the code comment says), is actually the better alternative here. What do we gain through the symlink indirection?

I cant remember tbh, I think I tired it and it caused some issue, will need to try again. in either case its the same end result

Co-authored-by: Fabian Fett <fabianfett@apple.com>

tomerd force-pushed the feature/packaging-plugin branch 2 times, most recently from 9866b0b to 4d09892 Compare March 18, 2022 07:40

tomerd added this to the 1.0.0-alpha.1 milestone Mar 18, 2022

tomerd added the 🔼 needs-minor-version-bump For PRs that when merged cause a bump of the minor version, ie. 1.x.0 -> 1.(x+1).0 label Mar 18, 2022

tomerd force-pushed the feature/packaging-plugin branch 2 times, most recently from 27dece5 to c75825c Compare March 18, 2022 07:44

tomerd force-pushed the feature/packaging-plugin branch 2 times, most recently from c5963ad to 907da4e Compare March 18, 2022 08:19

fabianfett reviewed Mar 18, 2022

View reviewed changes

Examples/Echo/Package.swift Outdated Show resolved Hide resolved

Package@swift-5.2.swift Outdated Show resolved Hide resolved

Plugins/AWSLambdaPackager/Plugin.swift Outdated Show resolved Hide resolved

Plugins/AWSLambdaPackager/Plugin.swift Show resolved Hide resolved

abertelrud reviewed Mar 18, 2022

View reviewed changes

Plugins/AWSLambdaPackager/Plugin.swift Show resolved Hide resolved

abertelrud reviewed Mar 18, 2022

View reviewed changes

Plugins/AWSLambdaPackager/Plugin.swift Outdated Show resolved Hide resolved

tomerd force-pushed the feature/packaging-plugin branch 4 times, most recently from cd04fc7 to a882987 Compare March 18, 2022 23:52

tomerd requested a review from yim-lee March 19, 2022 20:48

yim-lee reviewed Mar 22, 2022

View reviewed changes

tomerd force-pushed the feature/packaging-plugin branch 4 times, most recently from eb9923a to ab8c681 Compare April 15, 2022 19:10

fabianfett reviewed Apr 16, 2022

View reviewed changes

Examples/Echo/Package.swift Outdated Show resolved Hide resolved

tomerd changed the title ~~[wip] packaging plugin~~ packaging plugin Apr 18, 2022

fabianfett mentioned this pull request Apr 19, 2022

[docc] "Deploy your first Lambda" article #261

Closed

tomerd mentioned this pull request May 11, 2022

Allow plugins to communicate with docker without disabling the sandbox swiftlang/swift-package-manager#5489

Closed

tomerd and others added 12 commits May 16, 2022 09:50

packaging plugin

ebd585b

motivation: add an easy wasy for lambda users to package their lambda and upload it to AWS changes: * add SwiftPM plugin with the verb "archive" * use docker to build and package the lambda(s)

check for amazonlinux, zip correctly, drop 5.2 and 5.3

95134c0

parse arguments

5c035d2

Update Plugins/AWSLambdaPackager/Plugin.swift

4b23bb6

Co-authored-by: Yim Lee <yim_lee@apple.com>

fixup

df4ab5d

swift version

ff17559

cleanup

6a758e7

cleanup

1a249d8

error handling

ff7d30a

Fix Package.swift symlink issues (#1)

6b63d0e

cleanup

f43f914

info about products, allways pull latest docker iamge

0ddf8fb

tomerd force-pushed the feature/packaging-plugin branch from da7f95f to 0ddf8fb Compare May 16, 2022 16:51

debug

0eed404

tomerd force-pushed the feature/packaging-plugin branch 3 times, most recently from 89a4ba9 to ebd3f27 Compare May 16, 2022 18:03

error handling

b2842ba

tomerd force-pushed the feature/packaging-plugin branch from ebd3f27 to b2842ba Compare May 16, 2022 18:07

fixup

0457a37

fabianfett approved these changes May 16, 2022

View reviewed changes

tomerd and others added 2 commits May 16, 2022 18:53

Update Package@swift-5.6.swift

820eb62

Co-authored-by: Fabian Fett <fabianfett@apple.com>

better way to pull

936eb8f

tomerd force-pushed the feature/packaging-plugin branch from e08db84 to 936eb8f Compare May 26, 2022 18:01

nicer output

03ff066

tomerd force-pushed the feature/packaging-plugin branch from 6664486 to 03ff066 Compare May 26, 2022 23:44

tomerd merged commit 81f8d27 into swift-server:main Jun 1, 2022

tomerd mentioned this pull request Aug 2, 2022

DocC setup apple/swift-statsd-client#36

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

packaging plugin #254

packaging plugin #254

tomerd commented Mar 18, 2022 •

edited

Loading

tomerd commented Mar 18, 2022

fabianfett left a comment

tomerd commented Apr 15, 2022

fabianfett left a comment

tomerd commented Apr 18, 2022

fabianfett commented Apr 19, 2022

stevapple commented Apr 19, 2022 •

edited

Loading

tomerd commented Apr 19, 2022 •

edited

Loading

tomerd commented May 10, 2022

fabianfett left a comment

fabianfett May 16, 2022

tomerd May 17, 2022

packaging plugin #254

packaging plugin #254

Conversation

tomerd commented Mar 18, 2022 • edited Loading

tomerd commented Mar 18, 2022

fabianfett left a comment

Choose a reason for hiding this comment

tomerd commented Apr 15, 2022

fabianfett left a comment

Choose a reason for hiding this comment

tomerd commented Apr 18, 2022

fabianfett commented Apr 19, 2022

stevapple commented Apr 19, 2022 • edited Loading

tomerd commented Apr 19, 2022 • edited Loading

tomerd commented May 10, 2022

fabianfett left a comment

Choose a reason for hiding this comment

fabianfett May 16, 2022

Choose a reason for hiding this comment

tomerd May 17, 2022

Choose a reason for hiding this comment

tomerd commented Mar 18, 2022 •

edited

Loading

stevapple commented Apr 19, 2022 •

edited

Loading

tomerd commented Apr 19, 2022 •

edited

Loading