A systemd mystery with a very slow session start

I'll start with what I said on the Fediverse a while back :

Today's mystery: on exactly one system (with shared NFS home directories), logins take 3+ seconds to start because '/usr/lib/systemd/systemd --user' sits on its hands between running systemd-xdg-autostart-generator and running the next step of a remarkably opaque login process.

I don't even know where to start and I only got this far with a hacked version of the great extrace that also reports a process start time number.

(Although I didn't mention it in the post, these are SSH logins.)

As mentioned, this only happens on one system (running Ubuntu 26.04). Other systems, with the same software configuration and the same NFS mounted home directories , don't have anything like this slow login. The slowness has persisted over multiple reboots of the system (and a hostname change as it moved into production). Once a systemd user session exists because of an ongoing SSH login, further SSH logins run much faster. As far as I can tell, this isn't specific to my own login, it happens to other ones as well.

(This leads to my hack workaround for my own account, ' ssh -o "ControlPersist 30d" -N -M <host> -f ', which not only keeps my session active basically forever but also sets it up so I don't even need to start up a new SSH connection as such.)

You can extract a bunch of information about your session startup with ' systemd-analyze --user dump ', and when I did this there was a clear problem :

Timestamp units-load-start: Thu 2026-07-09 18:40:30 EDT
Timestamp units-load-finish: Thu 2026-07-09 18:40:33 EDT

Three seconds to do this isn't normal, and this is before systemd starts really doing anything (but after unit generators have run).

This was traced to a specific section of the systemd code, cf , which makes only two calls, manager_enumerate_perpetual() and manager_enumerate() (which are also in manager.c ). These appear to set up all of the known systemd units of various types ( cf ). There seem to only be a few perpetual units (especially for user sessions), things like '-.slice' and 'init.scope'. Non-perpetual enumeration appears to happen for devices, mounts, and swap, and then triggers processing the 'load queue' (of units).

Unfortunately, using opensnoop-bpfcc to watch what things get opened shows nothing obvious (although it does show significant gaps in file open activity). More detailed system call traces might provide some more insight, but I don't know of any good way to do system call tracing outside of strace , ideally narrowing the trace down to a process running under a specific UID with a specific command name (the top level systemd user session process is started on the fly, so its PID is unpredictable in advance).

At the root, I suspect that this is something other than systemd, and systemd is just accessing whatever is slow, or doing some operation a lot that is unusually slow on this system. There are some other odd indicators on this system, where other things are sometimes slower than I expect them to be (such as 'dpkg -S'), but at the same time I can't find any obvious signs in our metrics system .

(I've even considered thoughts like a throttled CPU or slow RAM, but I can't find any particular signs of that and it feels like I may be distracting myself with complex explanations for what may turn out to have a simple cause.)

( One comment .)