cmd/snap: implement userd command as replacement for snapd-xdg-open #3260

morphis · 2017-05-02T13:36:49Z

In https://forum.snapcraft.io/t/integrate-snapd-xdg-open-into-snapd-repository/100 we decided to implement a snap userd command which replaces snapd-xdg-open.

Based on discussion from https://forum.snapcraft.io/t/integrate-snapd-xdg-open-into-snapd-repository/100

zyga · 2017-05-02T13:50:09Z

daemon/user/daemon.go

+type Daemon struct {
+	tomb   tomb.Tomb
+	conn   *dbus.Conn
+	ifaces []registeredInterface


The overlord of the word interface is unfortunate.

True, but we deal with dbus interfaces here ..

ogra1 · 2017-05-02T14:12:45Z

daemon/user/safelauncher.go

+)
+
+var (
+	allowedUrlSchemes = []string{"http", "https", "mailto"}


we audit this already inside the core snap, before sending the call to the dbus ... i think it would be nice to agree to only do this auditing in one place (either at the start or at the end of the chain) so you do not need to change two places (core snap as well as snapd code) when adding or removing a protocol handler (i dont care which side we drop, but having a single place will prevent errors)

Where are we doing this? The xdg-open script in the core snap has only:

$ cat /snap/core/current/usr/local/bin/xdg-open #!/bin/sh dbus-send --print-reply --session --dest=com.canonical.SafeLauncher / com.canonical.SafeLauncher.OpenURL string:"$1"

in the mimetype handler of the .desktop file we install inside core: https://github.com/snapcore/core/blob/master/live-build/hooks/500-create-xdg-wrapper.binary#L23
and in the mimeapps.list we implement:
https://github.com/snapcore/core/blob/master/live-build/hooks/500-create-xdg-wrapper.binary#L28
protocols not defined in there will simply be ignored. as i said above i'm not opposed to drop this if it doesnt break things, i'd just like if we have only one place defining it.

Anything inside the core snap is untrusted as it can be altered by the attacker. This filtering really does belond on this side.

ogra1 · 2017-05-02T14:17:37Z

packaging/ubuntu-16.04/control

@@ -112,3 +112,12 @@ Section: oldlibs
 Pre-Depends: dpkg (>= 1.15.7.2)
 Description: Transitional package for snapd
 This is a transitional dummy package. It can safely be removed.
+
+Package: snapd-xdg-open


keeping this as separate package will break for users using only "apt-get upgrade" (vs dist-upgrade) if snapd gets a dependency on it.
...it was one of the main reasons to actually start discussing to move snapd-xdg-open into snapd in the beginning to actually get it into distros that have not seeded it (like elementary) without introducing a package dependency.

similar to snap-confine btw ...

It will stay a separate package but produced by the snapd source package so we can migrate people away from the other snapd-xdg-open deb package. That was the idea. Didn't tested yet this works well.

well, but that means you have to either add a recommends (in which case the package will never be installed for users only doing "apt-get upgrade") or you add a hard dependency in which case users of "apt-get upgrade" will get an upgrade error. we have tons of bugs for that case from snap-confine which was one of the reasons to include it without a separate package, keeping a separate binary package looks to me like we are exactly re-introducing the same problem again just with another package now.

We need one or another way to get the snapd-xdg-open package removed from installations out there. If we have a binary package produced by two different source packages but the second one has a higher version as the first one isn't then the one from the first source package overwritten?

The Recommends is for sure missing. Any other ways how we can achieve this?

yes, include the binaries in snapd and have the proper breaks/replaces lines (and perhaps a provides) in the snapd package.

Ok, including the binaries in the snapd package was the plan. Will add the Breaks/Replaces line.

CC @mvo5 - I think we just want to have this in the main snapd package and have snapd-xdg-open as an empty transitional package.

@zyga that is what we already do here. But leaving this to @mvo5 to approve.

zyga · 2017-05-08T11:12:30Z

Personally, given the scope of this branch, I'd like to re-target a smaller subset. Let's start with the basic mechanics (user session daemon) even if it does nothing. Then follow up with more things.

zyga

Some more comments.

zyga · 2017-05-08T11:16:03Z

data/upstart/Makefile

+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+LIBEXECDIR := /usr/lib


I think this should come from the outside. Ideally we'd have one makefile for everything in data/

My point is that := is stronger than ?=

Got it, will change to ?=

zyga · 2017-05-08T11:17:43Z

packaging/ubuntu-14.04/rules

@@ -103,6 +103,8 @@ override_dh_auto_build:
 	cd cmd && ( autoreconf -i -f )
 	cd cmd && ( ./configure --prefix=/usr --libexecdir=/usr/lib/snapd $(VENDOR_ARGS))
 	$(MAKE) -C cmd all
+	# Build any available upstart configurations
+	$(MAKE) -C data/upstart all


We could convey things like LIBEXECDIR and anything else through make variables.

Aren't we doing that already? We just take the defaults here as we do similar for the systemd build step. This works similar like with default values for configure scripts.

Well, not with LIBEXECDIR := /usr/lib that you can see in data/upstart/Makefile

Isn't that already a variable set in the Makefile which can be optionally overridden by the caller?

zyga · 2017-05-08T11:18:19Z

packaging/ubuntu-14.04/rules

@@ -142,6 +144,10 @@ override_dh_install:
 	# and now the normal install rules
 	install --mode=0644 debian/snapd.system-shutdown.service debian/snapd/$(SYSTEMD_UNITS_DESTDIR)
 	$(MAKE) -C cmd install DESTDIR=$(CURDIR)/debian/tmp
+
+	# install any available upstart jobs
+	$(MAKE) -C data/upstart install DESTDIR=$(CURDIR)/debian/snapd/


If you go with one makefile you could just roll that into the existing make install on data.

We can do a single makefile but this would make it a bit harder for people to skip the installation of the upstart files.

zyga

Some comments, good luck! :-)

zyga · 2017-05-15T11:25:15Z

cmd/snapd/main.go

-	if err := run(); err != nil {
+
+	parser := flags.NewParser(&opts, flags.HelpFlag|flags.PassDoubleDash|flags.PassAfterNonOption)
+	_, err := parser.ParseArgs(os.Args[1:])


You can combine this into one line:

if _, err := parser.ParseArgs(os.Args[1:]); err != nil { ... }

Can combine that. The code was exactly this way before.

zyga · 2017-05-15T11:25:59Z

cmd/snapd/main.go

+	if opts.User {
+		d, err = user.NewDaemon()
+	} else {
+		httputil.SetUserAgentFromVersion(cmd.Version)


Can we call SetUserAgentFromVersion unconditionally? I suspect we can and it would not hurt in any way.

It was done this way before so I guess it's ok.

zyga · 2017-05-15T11:26:54Z

daemon/user/daemon.go

+import (
+	"bytes"
+
+	"github.com/godbus/dbus"


Good luck a packaging that everywhere ^_^

Quick question: how many dependencies does godbus itself pull in?

I only had to add godbus itself to our vendoring setup, no more. And the good thing is, it is already on Fedora, the only distro where this would hurt.

What about Debian?

Somebody would have to submit a package but our story for debian is basically that we rely on re-execution to work. For unstable and next testing this would be a thing somebody has to look into but I guess that will be a natural thing once we task somebody to package a more recent snapd version for debian.

I think this somebody is you and me We should pick this up in anticipating of this thing landing.

I've talked with Michael and Steve in the past and they weren't much interested as re-execution is supposed to handle this and we can't really update snapd in stable or testing. So unstable is our only possible landing bed. You're a debian developer? If you're I would prefer you do this as I am not one.

I'm a DM, in either case we must package this for Sid.

zyga · 2017-05-15T11:27:20Z

daemon/user/daemon.go

+	"fmt"
+
+	"github.com/godbus/dbus/introspect"
+	tomb "gopkg.in/tomb.v2"


I think you can skip tomb as it will be imported like that anyway.

zyga · 2017-05-15T11:28:11Z

daemon/user/daemon.go

+)
+
+const (
+	busName  = "com.canonical.SafeLauncher"


Nitpick: I know this is what we currently implement but I'd like us to listen on both io.snapcraft.SomethingNice and com.canonical.SafeLauncher (the second one can be deprecated later)

I would not like to open another gate here with the upcoming xdg portal working which will introduce a very similar API. Lets keep it to what it is today and then refine / rework in a follow up PR.

zyga · 2017-05-15T11:38:35Z

data/upstart/Makefile

+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+LIBEXECDIR := /usr/lib


My point is that := is stronger than ?=

zyga · 2017-05-15T11:38:52Z

data/upstart/Makefile

+UPSTART_SERVICES := $(patsubst %.conf.in,%.conf,$(wildcard *.conf.in))
+
+%.conf: %.conf.in
+	cat $< | \


Nitpick, this fits on one line nicely:

zyga · 2017-05-15T11:39:09Z

data/upstart/Makefile

+
+%.conf: %.conf.in
+	cat $< | \
+		sed s:@libexecdir@:${LIBEXECDIR}:g | \


Please use single quote around this

zyga · 2017-05-15T11:40:25Z

data/systemd/snapd-user.service.in

@@ -0,0 +1,10 @@
+[Unit]


I bet I'm missing something but isn't it the case that user services have this funky syntax including @ in the service name?

They don't have to. @ basically allows to pass arguments to the systemd unit which we don't need in this case as systemd will ensure we're started as the right user and in the right scope.

Aha, I see, thank you!

zyga · 2017-05-15T11:41:22Z

packaging/ubuntu-16.04/control

@@ -112,3 +112,12 @@ Section: oldlibs
 Pre-Depends: dpkg (>= 1.15.7.2)
 Description: Transitional package for snapd
 This is a transitional dummy package. It can safely be removed.
+
+Package: snapd-xdg-open


CC @mvo5 - I think we just want to have this in the main snapd package and have snapd-xdg-open as an empty transitional package.

mvo5 · 2017-08-24T09:09:40Z

I also created the companion PR for the core side: canonical/core#54

zyga · 2017-08-25T08:11:44Z

data/dbus/io.snapcraft.Launcher.service.in

@@ -0,0 +1,3 @@
+[D-BUS Service]
+Name=io.snapcraft.Launcher
+Exec=@bindir@/snap userd


Feels like libexecdir thing to me.

This is the regular snap binary that provides the userd subcommand. So we need bindir here unless I miss something.

This is correct, though I expected this to be implemented in a libexecdir command, as users shouldn't usually do things here. Not a biggie, though.

zyga

Hey

Thank you for picking this up mvo. Two nitpicks, one about libexecdir vs bindir (no manual page for us to write) and one more about ping. I really really want to use the ping method because it's the right thing to do in this context.

zyga · 2017-08-25T08:12:53Z

tests/main/snap-userd/task.yaml

+            --type=method_call                                     \
+            --print-reply                                          \
+            /io/snapcraft/Launcher                                 \
+            io.snapcraft.Launcher.OpenURL string:"ping-ping-ping:" \


Please use the regular peer ping method here.

Unfortuantely ping is not implemented by us, I can dig a bit more into this but it looks like a limiation/missing feature of the godbus library which is not very high-level.

I changed the code to use "ping" now.

Conan-Kudo

Looks great, but a few comments.

Conan-Kudo · 2017-08-25T17:51:49Z

data/dbus/io.snapcraft.Launcher.service.in

@@ -0,0 +1,3 @@
+[D-BUS Service]
+Name=io.snapcraft.Launcher
+Exec=@bindir@/snap userd


This is correct, though I expected this to be implemented in a libexecdir command, as users shouldn't usually do things here. Not a biggie, though.

Conan-Kudo · 2017-08-25T17:52:23Z

interfaces/builtin/unity7.go

+    path=/io/snapcraft/Launcher
+    interface=io.snapcraft.Launcher
+    member=OpenURL
+    peer=(label=unconfined),


Conan-Kudo · 2017-08-25T17:54:19Z

packaging/fedora/snapd.spec

@@ -380,12 +380,11 @@ autoreconf --force --install --verbose
 popd

 # Build systemd units
-pushd ./data/systemd
-make BINDIR="%{_bindir}" LIBEXECDIR="%{_libexecdir}" \
+make -C data \


Please refrain from making unrelated changes to the spec. Changing from pushd / popd to using make -C is a different type of change.

Sorry, I just inherited this change, I can revert this back if you prefer.

Conan-Kudo · 2017-08-25T17:55:06Z

packaging/fedora/snapd.spec

@@ -441,13 +440,11 @@ rm -fv %{buildroot}%{_bindir}/ubuntu-core-launcher
 popd

 # Install all systemd units
-pushd ./data/systemd
-%make_install SYSTEMDSYSTEMUNITDIR="%{_unitdir}" BINDIR="%{_bindir}" LIBEXECDIR="%{_libexecdir}"
+%make_install -C data SYSTEMDSYSTEMUNITDIR="%{_unitdir}" BINDIR="%{_bindir}" LIBEXECDIR="%{_libexecdir}"


Ditto here.

Same as above.

Conan-Kudo · 2017-08-25T17:56:02Z

packaging/ubuntu-16.04/control

@@ -64,8 +64,8 @@ Depends: adduser,
         systemd,
         ${misc:Depends},
         ${shlibs:Depends}
-Replaces: ubuntu-snappy (<< 1.9), ubuntu-snappy-cli (<< 1.9), snap-confine (<< 2.23), ubuntu-core-launcher (<< 2.22)
-Breaks: ubuntu-snappy (<< 1.9), ubuntu-snappy-cli (<< 1.9), snap-confine (<< 2.23), ubuntu-core-launcher (<< 2.22)
+Replaces: ubuntu-snappy (<< 1.9), ubuntu-snappy-cli (<< 1.9), snap-confine (<< 2.23), ubuntu-core-launcher (<< 2.22), snapd-xdg-open (<< 0.0.0)


That is an insane number of Breaks/Replaces directives!

…6.04

niemeyer · 2017-08-28T23:36:14Z

Thanks for the long work here @morphis, thanks for taking this over @mvo, and thanks for the reviews everybody else!

feikname · 2017-08-28T23:57:06Z

Thank you guys for this!

So, snaps that call xdg-open will work out of the box without any dependences when 2.28 gets released?

mvo5 · 2017-08-29T05:35:30Z

@feikname correct :)

morphis · 2017-08-29T05:52:38Z

@mvo5 I owe you a beer or something else on the next sprint for taking this over and finishing it!

This should have been caught when canonical#3260 was being worked on. Signed-off-by: Neal Gompa <ngompa13@gmail.com>

ratijas · 2019-09-26T13:55:59Z

Hello.

I have no idea how this is supposed to work internally, but apparently it doesn't for me. As far as i understood, it should work out-of-the-box after 2.28 or something.

snap    2.41-1
snapd   2.41-1
series  16
arch    -
kernel  5.3.0-arch1-1-ARCH

Particular snap which causes xdg-open fork-bomb:

Name         Version   Rev   Tracking     Publisher   Notes
webstorm     2019.2.3  112   stable       jetbrains✓  classic

I have snapd-xdg-open-git^AUR package installed on host which provides 2 binaries /usr/lib/snapd-xdg-open/{snapd-xdg-open,xdg-open} and a service /usr/share/dbus-1/services/com.canonical.SafeLauncher.service.

Why is that? Should I file an issue?

Thanks.

anonymouse64 · 2019-09-26T15:23:53Z

@ratijas you should file an issue on https://forum.snapcraft.io, there you will have more visibility for your issue. It's not clear to me if this PR is the cause of your issue or not, more debugging is necessary

zyga reviewed May 2, 2017

View reviewed changes

ogra1 reviewed May 2, 2017

View reviewed changes

morphis force-pushed the f/snapd-user-instance branch 2 times, most recently from 8645bb7 to 1df07bc Compare May 4, 2017 08:49

morphis changed the title ~~WIP: Implement --user instance for snapd~~ daemon: implement --user instance for snapd May 4, 2017

morphis force-pushed the f/snapd-user-instance branch from 1e7a9c9 to ac5783e Compare May 8, 2017 06:01

zyga reviewed May 8, 2017

View reviewed changes

morphis mentioned this pull request May 12, 2017

many: add support for session dbus services via snaps #2592

Closed

zyga reviewed May 15, 2017

View reviewed changes

morphis added 19 commits May 18, 2017 15:20

cmd/snapd: add --user argument to start user instance

46a9432

daemon: implement basic dbus service part

169b142

data/systemd: add systemd service unit for the snapd user instance

f4e173c

daemon/user: add test cases for the SafeLauncher implementation

b8c31e3

daemon: refactor daemon into a interface structure

3f8f542

debian: add transitional package for snapd-xdg-open

c41ebcf

daemon/user: correct formatting according to gofmt

5bbb9bb

Add godbus to our vendor file

b0babff

daemon/user: use correct error handling

8109a38

tests: add test case for snapd --user

d7f1aea

debian: add proper replaces/breaks for snapd-xdg-open

41a5709

daemon/user: extend set of unit tests for the daemon implementation

59ccf07

tests/main: add description for snapd --user test case

a7de2c2

tests/upgrade: add test case to verify snapd-xdg-open migration

6216329

Also drop snapd-xdg-open on test cleanup

5381db5

data/upstart: add job definition for the snapd --user instance

9684c11

debian: fix installation of our systemd/upstart service files

adf835a

tests/upgrade: purge snapd-xdg-open on restore

0bf51cb

data: add global makefile which takes care about upstart and systemd

bd5e6ad

mvo5 mentioned this pull request Aug 24, 2017

Use io.snapcraft.Launcher instead of com.canonical.SafeLauncher canonical/core#54

Merged

address review feedback

fcb0890

zyga reviewed Aug 25, 2017

View reviewed changes

zyga approved these changes Aug 25, 2017

View reviewed changes

mvo5 added 7 commits August 25, 2017 13:57

add org.freedesktop.DBus.Peer to launcherIntrospectionXML

d740c70

add tests for cmdUserd

d5a4b1d

use org.freedesktop.DBus.Peer.Ping in snap-userd spread test

bfcb309

extract details from cmd_userd into userd/userd.go

804dcbb

Refactor userd code and extract testutil.DBusTest helper

5c2884b

add dbus to the build-dependencies

cdbe78e

DBusTest: do not crash if no dbus-daemon is found

1782192

niemeyer approved these changes Aug 25, 2017

View reviewed changes

Conan-Kudo approved these changes Aug 25, 2017

View reviewed changes

mvo5 added 3 commits August 25, 2017 22:23

also require dbus-launch in DBusTest

0ac7b57

change fedora/snapd.spec as requested by Neal Gompa

3e04c6d

fix circular import

cf2ac03

mvo5 force-pushed the f/snapd-user-instance branch from bdbf7ee to cf2ac03 Compare August 28, 2017 11:04

fix upgrade test, the version of snapd-xdg-open is not always 0.0.0~1…

278ac67

…6.04

niemeyer merged commit 047538e into canonical:master Aug 28, 2017

feikname mentioned this pull request Aug 28, 2017

xdg-open does not work with Snap version of OpenSpades yvt/openspades#639

Closed

Conan-Kudo added a commit to Conan-Kudo/snapd that referenced this pull request Sep 6, 2017

packaging/fedora: Add missing godbus dependencies

4a66417

This should have been caught when canonical#3260 was being worked on. Signed-off-by: Neal Gompa <ngompa13@gmail.com>

mvo5 pushed a commit to mvo5/snappy that referenced this pull request Sep 11, 2017

packaging/fedora: Add missing godbus dependencies

010ccf5

This should have been caught when canonical#3260 was being worked on. Signed-off-by: Neal Gompa <ngompa13@gmail.com>

cmd/snap: implement userd command as replacement for snapd-xdg-open #3260

cmd/snap: implement userd command as replacement for snapd-xdg-open #3260

Conversation

morphis commented May 2, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ogra1 May 3, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ogra1 May 3, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zyga commented May 8, 2017

zyga left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zyga left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

morphis May 15, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mvo5 commented Aug 24, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zyga left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Conan-Kudo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

morphis commented May 2, 2017 •

edited

Loading

ogra1 May 3, 2017 •

edited

Loading

ogra1 May 3, 2017 •

edited

Loading

morphis May 15, 2017 •

edited

Loading