@kubiko Thanks for taking a look!

I have two systems affected by this, both running Ubuntu 16.04[.3] LTS desktop and both tracking the core snap, with the following specifics:

Server (though with Ubuntu desktop installed)

This system is running stock 16.04 (no -proposed or HW) and Apache doing proxying. I had previously installed the snap on this system, set my root-url and changed the port to 3333 (IIRC), and it was running fine when the issue occurred sometime around the January 15th refresh. I’ve since removed the snap along with the settings so the attempts on this system should in theory be clean installs.

17.33 jani@battra:~$ uname -a
Linux battra 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
17.33 jani@battra:~$ snap version
snap    2.31
snapd   2.31
series  16
ubuntu  16.04
kernel  4.4.0-112-generic
17.33 jani@battra:~$ snap info core
name:      core
summary:   snapd runtime environment
publisher: canonical
contact:   snappy-canonical-storeaccount@canonical.com
license:   unknown
description: |
  The core runtime environment for snapd
type:        core
snap-id:     99T7MUlRhtI3U0QFgl5mXXESAiSwt776
tracking:    stable
installed:   16-2.31 (4017) 85MB core
refreshed:   2018-02-06 17:18:04 +0200 EET
channels:                                  
  stable:    16-2.31                (4017) 85MB -
  candidate: 16-2.31.1              (4110) 85MB -
  beta:      16-2.31.1              (4110) 85MB -
  edge:      16-2.31+git586.d89ba3c (4127) 85MB -
17.34 jani@battra:~$ snap list
Name  Version  Rev   Developer  Notes
core  16-2.31  4017  canonical  core

Main desktop

This system has the -proposed repository enabled and the HWE stack installed, no proxy. I’ve only ever attempted to install Wekan on this system after the issue occurred on the other system, and it has never succeeded (so in that sense it’s a clean install… attempt anyway).

17.08 jani@saegusa:~$ uname -a
Linux saegusa 4.13.0-36-generic #40~16.04.1-Ubuntu SMP Fri Feb 16 23:25:58 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
17.11 jani@saegusa:~$ snap version
snap    2.31.1
snapd   2.31.1
series  16
ubuntu  16.04
kernel  4.13.0-36-generic
17.11 jani@saegusa:~$ snap info core
name:      core
summary:   snapd runtime environment
publisher: canonical
contact:   snappy-canonical-storeaccount@canonical.com
license:   unknown
description: |
  The core runtime environment for snapd
type:        core
snap-id:     99T7MUlRhtI3U0QFgl5mXXESAiSwt776
tracking:    stable
installed:   16-2.31 (4017) 85MB core
refreshed:   2018-02-06 17:18:04 +0200 EET
channels:                                  
  stable:    16-2.31                (4017) 85MB -
  candidate: 16-2.31.1              (4110) 85MB -
  beta:      16-2.31.1              (4110) 85MB -
  edge:      16-2.31+git586.d89ba3c (4127) 85MB -
17.12 jani@saegusa:~$ snap list
Name  Version  Rev   Developer  Notes
core  16-2.31  4017  canonical  core

Since the issue with Wekan occurred, I’ve tried installing other snaps (on both systems) and they all work just fine.

If there’s anything else you need or something I missed, please let me know and I’ll be happy to provide.

I’m fairly sure it’s a matter of some configuration these two systems happen to have, and the Wekan snap issue is mainly just that of it not failing gracefully under the circumstances and leaving me in the dark about the exact cause.

Still no fix I’m afraid, but watching /var/snap/wekan/common/hook.log did provide a clue. The hook appears to get stuck at snapctl get caddy-enabled, after that there are no more entries.

I tried tweaking the hook by replacing value=$(snapctl get caddy-enabled) with value="", but installing the resulting snap then just gets stuck at the next snapctl call:

+ '[' '' = true ']'
+ snapctl stop --disable wekan.caddy

There’s also an ’error running snapctl: unknown service: ”caddy-service”’ line in syslog, but this seems to occur just alike on systems where the installation does not get stuck, so I’m not sure if it’s relevant.

Over at Launchpad, Jamie Strandboge had this to say:

”As for install/refresh getting stuck, it seems that the wekan hooks aren’t exiting so snapd has no choice but to wait a while then timeout.

At this point, this seems like the issues are in the wekan snap: it needs to plug hardware-observe and needs to handle error conditions better in the hooks so the exit.”

Alright, thanks Jamie!

Output of `snap interfaces` during `snap install wekan` Edit (2.4 KiB, text/plain)

The Apparmor denial seems to no longer occur with 2.31.1.

`snap interfaces` lists ’wekan’ connected to network and network-bind, but not hardware-observe. I’m struggling with the correct syntax for manually connecting it to anything:

root@saegusa:~# snap connect wekan :hardware-observe
error: cannot resolve connection, plug snap name is empty
root@saegusa:~# snap connect wekan:wekan :hardware-observe
error: snap ”wekan” has no plug named ”wekan”
root@saegusa:~# snap connect wekan: :hardware-observe
error: invalid value: ”wekan:” (want snap:name or snap)
root@saegusa:~# snap connect :wekan :hardware-observe
error: cannot resolve connection, plug snap name is empty

`snap interfaces` does list a ’wekan:mongodb-plug’, which (if I’m reading the output right) is unattached. Attempting to connect that to hardware observe:

root@saegusa:~# snap connect wekan:mongodb-plug :hardware-observe
error: cannot connect wekan:mongodb-plug (”content” interface) to core:hardware-observe
(”hardware-observe” interface)

There’s also a ’wekan:mongodb-slot’. Attempting to connect mongodb-plug to that:

root@saegusa:~# snap connect wekan:mongodb-plug wekan:mongodb-slot
error: snap ”wekan” has ”install-snap” change in progress

That’s true, since I’m only able to see those two during the time that the installation is stuck.

I’ll attach the full output of `snap interfaces` below. It’s the same output that `snap interfaces` produces on the VM without the issue (after installation, when the service is running).

As for the other question, there are wekan-related commands running when it’s stuck (listing below). The service seems to be up and running already (I can open it in a browser) for the time it remains in the configuration phase, but when the hook times out it of course gets cancelled and the installation is undone.

root@saegusa:~# ps aux | grep wekan
root 10517 1.2 0.1 935608 23360 pts/22 Sl+ 17:45 0:00 snap install wekan
root 10782 0.0 0.0 18056 2760 ? Ss 17:45 0:00 /bin/bash /snap/wekan/124/bin/mongodb-control
root 10809 4.4 0.3 283812 54748 ? Sl 17:45 0:01 mongod –dbpath /var/snap/wekan/common –logpath /var/snap/wekan/common/mongodb.log –logappend –journal –unixSocketPrefix /var/snap/wekan/124/share –port 27019
root 10811 0.0 0.0 18052 2892 ? S 17:45 0:00 /bin/bash /snap/wekan/124/meta/hooks/configure
root 10871 0.0 0.0 18056 2676 ? Ss 17:45 0:00 /bin/bash /snap/wekan/124/bin/wekan-control
root 10898 7.6 0.6 1172036 103648 ? Sl 17:45 0:02 /snap/wekan/124/bin/node main.js
root 10975 0.0 0.0 15464 1012 pts/23 S+ 17:46 0:00 grep –color=auto wekan

Just a minor ”no update” update: SRU 2.31 of snapd arrived via -proposed, but did not resolve this issue unfortunately.

Update: got snapd 2.31.1 from -proposed this morning, and replaced the customized /etc/apparmor.d/usr.lib.snapd.snap-confine.real with the package-provided version. This did not fix the issue unfortunately.

I went ahead and reported this on Launchpad, we’ll see if anyone there has any ideas. A new SRU (2.30) of snapd also seems to be in the works, but no release as of yet.

Since some time over the holidays I’ve had problems refreshing/installing the Wekan snap [1] on my home server and also my desktop. The installation stalls at the configuration phase, which on the surface looks a bit like bug #1674193 [2], but here core gets installed just fine, and the hang occurs just alike if I first install just core, then the `wekan` snap separately.

14.52 jani@saegusa:~$ sudo snap install wekan
[sudo] salasana henkilölle jani:
error: cannot perform the following tasks:
– Run configure hook of ”wekan” snap if present (run hook ”configure”: <exceeded maximum runtime of 5m0s>)

Installing other snaps works (the couple that I tried just to be able to say this did anyway).

I’ve reported this on the Wekan snap Github page [3], but there’s been no confirmation from anyone else affected so far. Also, I’m unable to reproduce this myself in a VM and on at least one other (physical) desktop I have access to.

So naturally I’ve looked for differences between these systems, but so far the only correlating one I’m pretty sure of is an Apparmor denial:

apparmor=”DENIED” operation=”open” profile=”snap.wekan.mongodb” name=”/sys/block/” pid=9478 comm=”mongod” requested_mask=”r” denied_mask=”r” fsuid=0 ouid=0

The two systems where Apparmor denies mongodb’s access to /sys/block get stuck at the configure hook, whereas systems that don’t deny access finish the configuration (and installation) successfully.

I have not tweaked any Apparmor configuration on any of these systems prior to this issue cropping up (not that I can remember anyway). I’ve also not touched anything snap-related, as Wekan was one of the first snaps I’ve ever tried and is (or would be) the only one (besides core) currently installed on these systems.

All systems are running Ubuntu 16.04, with my (affected) desktop having both HWE and -proposed enabled, my (affected) server running a 4.4-series kernel (no HWE or -proposed) and the other (unaffected) desktop having HWE but no -proposed. The (unaffected) VM starts with kernel 4.4 and remains unaffected if I upgrade it with HWE.

I’m submitting this from the (HWE+proposed-enabled) desktop, so any logs attached here are from one of the two affected systems. I’ll of course provide other logs too if requested.

* [1] https://snapcraft.io/wekan/
* [2] https://bugs.launchpad.net/snappy/+bug/1674193
* [3] https://github.com/wekan/wekan-snap/issues/25

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: snapd 2.29.4.2
ProcVersionSignature: Ubuntu 4.13.0-30.33~16.04.1-generic 4.13.13
Uname: Linux 4.13.0-30-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.15
Architecture: amd64
CurrentDesktop: Unity
Date: Mon Jan 22 15:44:20 2018
InstallationDate: Installed on 2016-10-13 (466 days ago)
InstallationMedia: Ubuntu-Server 16.04.1 LTS ”Xenial Xerus” – Release amd64 (20160719)
SourcePackage: snapd
UpgradeStatus: No upgrade log present (probably fresh install)
mtime.conffile..etc.X11.Xsession.d.65snappy: 2018-01-19T18:18:12.001969
mtime.conffile..etc.apparmor.d.usr.lib.snapd.snap-confine.real: 2018-01-22T15:46:34.793893

(This came up for me in Wekan issue #1389)

When I try to install the snap (in Ubuntu 16.04), it gets stuck in Run configure hook of "wekan" snap if present. At this time Wekan service is already up and running, but snapd gives up on the configuration after a (built-in) 5 minute timeout and undoes the installation. (I initially thought the Node process was misbehaving, but I no longer think that’s the case here.)

Wekan snap issue #10 seems like the same problem. I enabled snap debugging as mentioned there, and will attach the result of grep snapd from during the installation attempt here.

While the service is up during the configuration phase, it produces journal log entries as usual and I’m attaching the relevant lines here.

I’ve tried purging and reinstalling snapd, disabling IPv6, turning off Apache and any other services that might be in the way (though none of them have caused issues previously), rebuilding the snap with my previous settings built in but none of it has made any difference.

In addition to my main server, I’ve since reproduced this on my desktop machine, and failed to reproduce it on another desktop and a VM with a fairly clean Ubuntu 16.04 install. The only difference between the reproducing and non-reproducing systems I’ve so far found is an Apparmor denial in /var/log/kern.log:

apparmor="DENIED" operation="open" profile="snap.wekan.mongodb" name="/sys/block/" pid=9478 comm="mongod" requested_mask="r" denied_mask="r" fsuid=0 ouid=0

The systems where Apparmor denies mongodb’s access to /sys/block get stuck at the configure hook, whereas systems that don’t report a denial finish the configuration (and installation) successfully.

I haven’t done any customization of Apparmor rules on any of these that I can remember; it’s pretty much dark arts to me which is why I’ve avoided touching it.