Notes on using GNU Emacs' Tramp system in an unusual shell environment

Tramp is a famous and often praised GNU Emacs system for editing remote files; lots of people will call it one of Emacs' compelling features. I've always had a decidedly different view of Tramp because Tramp has mostly not worked for me in opaque ways. I recently took another run at getting Tramp working (so I could have an informed opinion on why I'm not a fan), and in the process I've learned a bunch of things that I don't want to forget.

Although Tramp has a bunch of ways to get access to files remotely ('methods' in Tramp jargon), the dominant way is for Tramp to SSH in to the remote system and do stuff. In order to work with your remote shell, Tramp really wants your login on the remote system to have a conventional shell environment, ideally one that uses the Bourne shell (especially Bash).

(But see Remote shell setup hints and the Tramp FAQ .)

In specific, Tramp has requirements for its ssh method in a stock setup:

  • Your shell must have a relatively conventional shell prompt. Defining this is beyond the scope of this entry; see the definition of tramp-shell-prompt-pattern in tramp.el .
  • Your shell must accept and use backslash quoting of more or less arbitrary characters in command lines.
  • Your shell login can't pause to ask questions; it can produce some additional output but it needs to drop you to a shell prompt (that Tramp can recognize).

All of these are required because with the ' ssh ' method, Tramp ssh's in and starts a full login session, then switches to /bin/sh (or the Tramp remote shell you've set) with some special things that will let it reliably recognize its own Tramp (shell) prompts. Using the ' sshx ' method can bypass a lot of this because with it, Tramp directly runs /bin/sh without going through your remote login session. I believe sshx is also often going to be faster, at the cost of not establishing all of the environment variables and so on that your login session would (including your remote shell's normal $PATH).

If your login shell environment doesn't match all of these you're going to have a varying amount of problems, especially with the ' ssh ' method. If you have an unconventional prompt , you can sort of fix it, but a shell with different quoting rules will be painful. Tramp has some mechanisms to deal with additional questions but my impression is that they're at least a slog (see parts of Remote shell setup ).

(Since I went through this, to deal with quoting issues you need to redefine tramp-end-of-output to something that doesn't require quoting that your shell doesn't support, and then make sure that your tramp-shell-prompt-pattern matches it in addition to everything else. The only characters that won't be quoted with backslashes by GNU Emacs are -, ., /, 0-9. and a-zA-Z (this is deep in shell-quote-argument ). There are some things that may break inside GNU Emacs and Tramp if you do this but I haven't had any problems yet.)

If you ask Tramp to use the (remote) $PATH your remote environment sets up, it must be able to run '/bin/sh -l -c ...' in a way that successfully runs the command string without having your .profile blow things up, despite your .profile probably not being able to detect this . This is typically triggered by you putting 'tramp-own-remote-path' somewhere in tramp-remote-path (either the global version or a connection profile ). Because Tramp is that way, the remote path is not part of the predefined connection information that you can set directly.

Despite Tramp carefully initializing your remote login session (if you use ' ssh '), Tramp then normally ignores your remote $PATH and instead generates its own, based on tramp-remote-path . Various bits of Tramp documentation will imply that you can use '~' in things you add to tramp-remote-path ( cf some of the examples ), but as far as I can tell this is what you would call inoperable. As part of connection setup, Tramp reduces tramp-remote-path down to the directories that exist on the remote machine, and the mechanism Tramp uses for this appears to be incompatible with the use of either '~' or environment variables like ' $HOME '.

(Tramp does this path check using the tramp-bundle-read-file-names defconst and you can read what that expands to in order to see the details, along with the tramp-get-remote-path function and the stuff it calls. Since the shell snippet Tramp sends to the remote end quotes all of the directory names it checks, whether or not the remote shell supports '~' is irrelevant and it won't expand $HOME for you. It's possible that this is a bug and Tramp will get fixed some day, but don't hold your breath.)

There's no particularly good fix to this that I know of; instead, I think you have two options. The first is to make tramp-own-remote-path work (it probably will if you use a conventional shell and .profile), add it to tramp-remote-path , and set up and handle your $PATH properly in each machine's .profile. This is probably the better option if you can arrange it, in part because you probably want a correctly set remote $PATH for when you're logged in to the machine directly. The second option, suitable only if you have a common home directory name pattern or two across all your machines, is to add all likely directories to your tramp-remote-path in whatever variations of your home directory you might have:

(dolist (pe '("/home/cks/go/bin" "/u/cks/go/bin" ....))
  (add-to-list 'tramp-remote-path pe))

(Or you could write an ELisp function that generated the list from multiple sublists, one for things relative to your home directory and one a list of possible home directories.)

Many modern Unix systems in standard configurations will make your home directory be /home/<login> , so you can cover all of them by a few paths in tramp-remote-path . Well, assuming you have the same login on all of them. Otherwise, you'll probably have to venture into the world of connection local variables and profiles .

When changing tramp-remote-path there is something very important that can cause you (me) a great deal of frustration if you don't know the full story. At the very end of Tramp's documentation on remote programs , there is this critically important bit:

When remote search paths are changed, local Tramp caches must be recomputed . To force Tramp to recompute afresh, call M-x tramp-cleanup-this-connection RET or friends (see Cleanup remote connections ).

If you're me, you might innocently think that it's safe to, for example, set or modify tramp-remote-path before you make any connections. This is false, and calling tramp-cleanup-this-connection is not sufficient to force 'local Tramp caches' to be recomputed. In fact, not even quitting and restarting Emacs will do so. Tramp maintains a persistent file based cache of information about each host you've ever connected to , including the remote $PATH it determined at the time of the first connection (with the first connection's tramp-remote-path ), and it will use that cached remote $PATH value until and unless you clear the entire cache by, for example, deleting ~/.emacs.d/tramp (with Emacs not running), or you use tramp-cleanup-all-connections , which I think is probably sufficient.

Given its persistent and dangerous effects, you might want to disable this Tramp cache file. The fine documentation asserts that you can do this by setting tramp-persistency-file-name to nil . This appears to be technically correct but practically inoperative, because you cannot customize the variable to nil (only to a filename) or usefully setq it before Tramp is loaded. You can only setq it to nil (and have it stick) after Tramp is loaded (and you probably also want to invoke tramp-cleanup-all-connections to get rid of anything Tramp may have loaded).

Tramp isn't a mode and so doesn't have any hook that fires when it loads and starts to activate, which would be the right time to augment tramp-remote-path , clear any cached data Tramp loaded, and so on. This is unfortunate but use-package provides a way to work around it:

(use-package tramp
  :defer t
  ;; :config will be run right after Tramp loads.
  :config
  (cks/tramp-setup)
  )

This appears to reliably fire as I start to enter '/sshx:' or '/ssh:' or what have you.

(The manual version of this would be to directly use eval-after-load , but I might as well stick with use-package even if that's what use-package is using under the (macro) hood.)

When it works, Tramp can be pretty magical . However, my voyage of getting to this point was anything but smooth , and parts of it were extremely frustrating . That part was the part with the Tramp file cache, which made various changes to tramp-remote-path have no effect and then sometimes have effect and then go back to having no effect because I wasn't religiously clearing and removing the cache.

(Tramp badly needs a command that reports all of the relevant parameters for the current connection, such as the current remote path that Tramp is using. I could probably put my own version together with enough determination, but I shouldn't have to.)

PS: This entry was written in my working Tramp configuration from my home desktop , but I'm not sure I'm going to bother doing this again (I normally write entries in vim on the host that Wandering Thoughts is on). The red squiggles under (potentially) misspelled words are sort of nice, but on the other hand I turn out to have lots of vim reflexes for writing Wandering Thoughts entries.

(The reflexes aren't triggered by writing in general, because these days I write a lot of email in GNU Emacs and that goes fine.)