One of my standard little scripts is something I call sshup , which waits for a machine to be 'up' by periodically checking to see if its SSH port is responding. As mentioned in my original entry on sshup , I actually have two versions of this script and recently I discovered that the difference is quietly important.

One version of the script uses Netcat, on our Ubuntu machines. The specific Netcat command line it uses is:

nc -w3 -q3 -z "$1" ssh >/dev/null 2>&1

(I didn't used to need the redirection, but Debian reverted a patch and Ubuntu 24.04 picked up the change .)

The technical effect of this is to try to connect to the SSH TCP port and exit once it's done that (or been rejected or timed out), with an appropriate exit status.

The other version doesn't use Netcat, but in Netcat terms what it does is the equivalent of:

nc -N "$1" ssh </dev/null >/dev/null >2&1

The technical effect of this is to try to connect to the server (if possible), immediately shut down the sending side of the TCP conversation, and wait until the server closes the connection completely.

If the server is working properly, these two versions have the same result. But if the server isn't working, the answer these give is different, with the second version being more useful. The problem with the first version is that it only checks if the kernel is willing to let you make a TCP connection to the SSH port; it doesn't check if the SSH daemon is actually responding (normally, the SSH daemon will print a banner, then see that there's nothing more to read from the network and close the connection). If the server is broken sufficiently so that the kernel will accept the connection but the SSH daemon won't run, the first version will tell me that the machine is up and the second one will correctly tell me I can't SSH in to the machine.

(Recently a server hung during shutdown in exactly this way, which is a story for another entry.)

This difference is also relevant for health and monitoring checks for services (and famously so). Connecting to a TCP port is the very basic step in a health check; after that is checking that you received a banner or a canned response from a service (for HTTP services, perhaps a '200 Okay' from a health check endpoint), and after that is checking that you can do something meaningful that's part of the service's regular activity. Of course the higher level you go in checking the service, the more specific your health check is to the service (whether you implement the check in code or in a configuration file that says what to look for ). But at the same time, it's more meaningful because it comes closer and closer to what actually matters.

(But there's a tradeoff in how close you come versus how much work and so on it is. You make that tradeoff for individual services based partly on your knowledge of how that service works and what's likely to fail. For my 'is a machine up' checking script, seeing the SSH server banner is good enough basically all of the time. And if my sshup script says the machine is up but then I can't actually log in, that's telling me something useful too.)