Recently I've become aware that some things about specifying keys in .ssh/config don't work quite the way I absently thought they did. Let's start with a simple situation:

Host *
  # encrypted & usually loaded into
  # ssh-agent
  IdentityFile /u/cks/.ssh/ids/key-ed2

Host github.com
  IdentitiesOnly
  IdentityFile /u/cks/.ssh/ids/github

When I set up my .ssh/config , this configuration looked like it would offer Github only my Github specific key. It doesn't; it offers key-ed2 as well (in fact it offers key-ed2 first). What's happening is that IdentityFile is cumulative , and so every IdentityFile in every matching Host stanza is another key that will be offered. When you connect to github.com , both the ' Host * ' and the ' Host github.com ' stanzas match, so the IdentityFile directives from both are used.

(Where this matters is that servers have a limit on how many keys you can offer them before they drop your connection. This can make it important to send them the right key first or at least very early.)

This can make IdentitiesOnly into an essentially meaningless directive. If the only key we ever load into our ssh-agent is the key-ed2 generic key, that's what happens here; it's always a candidate IdentityFile key so it will never be excluded. IdentitiesOnly only matters if you load extra keys from outside your .ssh/config (or at least from outside a ' Host * ').

If ssh-agent is not running or does not have keys loaded, you can get around this by reordering your .ssh/config . Without ssh-agent , keys are offered to servers in the order of IdentityFile directives . If the ' Host github.com ' stanza is before ' Host * ', the Github specific key will be offered first (and then accepted). Unfortunately this doesn't work if you have keys loaded into ssh-agent , as keys from ssh-agent are always tried first before keys from .ssh/config ( in the order they were loaded into ssh-agent , not the order in .ssh/config ). This is the case even with IdentitiesOnly specified.

The rather awkward and not very scalable hack fix for this is to specifically exclude some destinations from the generic ' Host * ' stanza:

Host * !github.com !elsewhere.com ...
  ...

As long as you specify IdentitiesOnly in the host-specific Host stanza, things work right (and the order in .ssh/config doesn't matter). This is clearly not scalable if you have very many of these hosts; every time you add a new one, you need to remember to add it to the ' Host * ... ' exclusion as well or things will go subtly wrong. And if you have truly generic parameters other than IdentityFile , you'll need a separate ' Host * ' stanza for them.

The other workaround is to only use your general keys through ssh-agent and strip them out of .ssh/config (here we'd delete the IdentityFile line for key-ed2 ). This costs you the ability to use them with ssh when ssh-agent is not running (or doesn't have them loaded), but it makes IdentitiesOnly do what we want it to.

As far as I know there's no other way around this, although I'd love to be wrong. The whole situation is kind of irritating and I wish there was a version of IdentitiesOnly that meant 'only identities from this specific Host/Match stanza'. It's also irritating that such an important issue as key choice and key ordering is not discussed explicitly in the OpenSSH client documentation.

(Of course this also means that OpenSSH is free to change any or all of this on you in future versions, because they never made any official promises about how it worked. It's almost all 'undocumented behavior' and we're on our own.)