Secure and persistent NAT busting using SSH and autossh

SSH can be more than a secure shell. It has a cornucopia of features and use cases, it’s mature, and extremely widely used. I’ll cover NAT busting today with SSH. Say your parents are behind a shitty ISP and an unstable connection, and your mother has Linux installed, while your father has Windows installed. His computer sometimes misbehaves, and you want to connect via VNC, but the shitty router of the shitty ISP has managed to mangle all the port forward configs and the dyndns script failed to update the IP because $REASONS. As with all NAT busting schemes, you’ll need a third computer somewhere connected to the internet with a fixed IP address.

What’s the problem?

Simply put, NAT (Network address translation) is a popular way to group many IP addresses (computers and various devices, phones on a local area network) into a public facing one. When you connect to a remote host, the router keeps track of individual requests. If your computer’s address is 192.168.1.4, the remote host can’t send data back to a private network across the internet. The response is sent back to the public IP address and a random port chosen by your home router, which is then forwarded to 192.168.1.4 from the router. This all works great for connecting to websites, game servers, and whatnot, but what if you want to connect from the outside to a specific computer inside a NAT-ed environment in a way that is robust and secure?

SSH to the rescue

I’ll first cover the steps how to perform NAT busting to get access to the hypothetical mother computer:

  • Create a user for that purpose on the server with the fixed IP. Use /bin/false as the user shell as an extra security measure.
  • Place the mother’s public SSH key in the user’s authorized_keys file for passwordless login.
  • You’ll need to put GatewayPorts yes and ClientAliveInterval 10 in the sshd config on the server and restart the SSH daemon. The ClientAliveInterval setting is very important, I’ll come back to it.

After this is done, we can proceed with setting up the actual tunnel:

$ ssh -fNg -R 52004:localhost:22 user@fixedipserver.net # On the mother's computer
$ ssh -p 52004 mother@fixedipserver.net # Connect to the mother's computer 

"-f" switches ssh right to the background, "-N" is a switch to not execute any remote command, "-g" allows remote hosts to connect to local forwarded ports, and "-R" is here to specify the remote port, the IP on the local server, in our case localhost and local port, port:host:hostport. The "GatewayPorts yes" option is needed because SSH by default won’t allow to bind on anything other than 127.0.0.1, I guess this is a security feature. Basically, we have now exposed the port 22 of my mother’s computer and bound it to the 52004 port on fixedipserver.net so make sure that your dear mother’s SSH is properly secure, it would be best to disable password logins altogether if possible. The fixed IP server now acts as a bridge between you and the computer behind the NAT, and when you ssh to the port 52004 you’ll be expected to present the credentials of the mother’s computer, and not the fixed IP server. Nice, huh?

Wait a second Mr. Kitty, how exactly is this persistent?

It’s not. Two problems come to mind; the mother reboots the computer and poof, the tunnel is gone, or the shitty ISP router reconnects. Enter Autossh, it comes packaged with most newer distros. The author says: “autossh is a program to start a copy of ssh and monitor it, restarting it as necessary should it die or stop passing traffic.”

Wow, just what we need! We’ll just fire it up like so:

$ autossh -fNg -R 52004:localhost:22 user@fixedipserver.net

As you can see the options for ssh are identical. To make it truly persistent add to the mother’s crontab the following entry:

@reboot autossh -fNg -R 52004:localhost:22 user@fixedipserver.net

Finally, to connect to the father’s private address 192.168.1.2 on their LAN and VNC port we can use something like:

$ ssh -fNg -R 5900:192.168.1.2:5900 user@fixedipserver.net

Please note that you need to enable GatewayPorts yes on the mother’s computer as well if you want to route traffic like this. No need for autossh for these temporary interventions, and it’s probably a bad idea to have port 5900 open to the public all the time.

Autossh tuning for an even more robust tunnel

Autossh with its default settings is probably good enough for most cases. Certain ISP routers have a very nasty habit of killing idle connections. No communications on an open TCP connection for 2 minutes? Drop it like a hot potato without letting anyone know. The computer behind the router is convinced the connection is still alive, but if it tries to communicate via that open connection all the packets seemingly go into a black hole and the network stack takes a while to realize what’s going on. Now, let’s get back to ClientAliveInterval 10 and why it’s so important:

export AUTOSSH_POLL=45
autossh -M 0 -o "ExitOnForwardFailure yes" -o "ServerAliveInterval 15" -o "ServerAliveCountMax 2" -fNg -R 52004:localhost:22 user@fixedipserver.net

This is all fine and dandy, but a true sysadmin will always test his assumptions and procedures. I setup a simple autossh tunnel, browsed to my DD-WRT web GUI and reset the modem to simulate an internet crash. The modem quickly recovered, the internet connection was restored and after a few minutes, whoops:

$ telnet fixedipserver.net 52004
Trying 100.100.100.100...
Connected to fixedipserver.net.
Escape character is '^]'.

Not good, where the hell is “SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.1” that I usually get? I only managed to connect to the dangling port on the server that leads nowhere. The connection server-side is disconnected after 30 seconds at these settings, however under slightly different conditions while I was testing the port 52004 is still bound by the old connection the ssh daemon hasn’t cleaned up yet causing ssh to fail to create a tunnel because the server says that port 52004 is busy. Fine, fair enough, however the big problem is that ssh will not exit because of this when trying to establish a tunnel. It’s a warning only and the process is still up & running. As far as autossh is concerned all is dandy. Well, it’s not! With the -o "ExitOnForwardFailure yes" we force that this kind of error makes ssh exit with a non-zero exit status and makes autossh do its intended purpose; try to reconnect again and again. The environment variable AUTOSSH_POLL makes it do internal checks of the ssh process a little bit more frequently.

The -M 0 disables autossh’s builtin monitoring using echo and relying on SSH’s very own ServerAliveInterval 15 which probes the connection every 15 seconds, and ServerAliveCountMax 2 is the number of tries until SSH will kill off the connection if the connection is down. In conjunction with ServerAliveInterval 15 assuming the connection was reset, ssh will again exit with a nasty non-zero exit status which will force autossh to react. It is an absolutely crucial setting to achieve true high availability with a disconnecty modem. The idea is to get an up & running tunnel as soon as it’s theoretically possible, IE, the end client has a fully working internet connection once again. Without the echo method we achieve a pretty simple and robust setup in the end.

If you connect to fixedipserver.net port 5900 it will route you to the father’s computer over the mother’s computer. If you connect to port 52004 you will get the mother’s ssh service, port 22.

TL;DR – get it up & running, the robust version

  • Create a user on the fixed IP server with /bin/false as the shell and enable passwordless login with SSH
  • Set the two following options in the sshd config and reload the daemon:
GatewayPorts yes
ClientAliveInterval 10
  • We’ll need to set an environment variable before starting Autossh. You can use this as a reference for a simple script:
#!/bin/bash

REMOTE_PORT=40000 # The exposed port on the fixed IP server that you'll be connecting to
DESTINATION_HOST=localhost # Any host that is connectable from the computer initiating ssh/autossh can be set, be it a server inside the LAN, or a server on the internet, or localhost, it doesn't matter to SSH
LOCAL_PORT=22 # The destination port we want to have access to (port 22 - SSH in our example, can be anything, doesn't even need to be open)

export AUTOSSH_POLL=45
autossh -M 0 -o "ExitOnForwardFailure yes" -o "ServerAliveInterval 15" -o "ServerAliveCountMax 2" -fNg -R $REMOTE_PORT:$DESTINATION_HOST:$LOCAL_PORT user@fixedipserver.net
  • And finally, you can set a cronjob:
@reboot connect_script

Final thoughts

The robust tunnel part is really necessary if you have a mission critical server or client somewhere behind a flaky connection, whether you need it for pure ssh access or you need it for something like VNC or another service somewhere on the remote private network. Please keep in mind that any forwards you setup like this will be available from anywhere on our hypothetical fixedipserver.net server. Who knows what kind of vulnerabilities a service has, so firewall everything off on the server with the fixed IP. There is some good news though, if someone does hack into the remote machine via your open VNC port or whatever, you can rest assured in the knowledge that their traffic between the fixed IP server and your pwned service are safe with a state of the art encryption protocol provided by the wonderful Secure Shell.

Fast movie reviews

Mistress America

An intriguing movie covering a coming-of-age sort of thing in New York City. Tracy (Lola Kirke) is an aspiring would-be writer having trouble fitting in at Barnard, so on her mother’s advice she contacts her soon to be step-sister Brooke (Greta Girwig). The movie doesn’t have the sort of predictability you would come to expect, the characters are likable, their relationships and dialogues interesting. Brooke wants to open up a restaurant which is also a hipster paradise, a place where you can hang around and talk, and of course order food, be able to trade etc. She goes a long way but needs some capital to get things started. She’s a smart and independent woman, but still realistically clueless, as we all are when we’re in our thirties. She goes to her ex to get him to invest in the new business, Tracy accompanies her because she really believes in Brooke and is sort of in love with her. There is no one to drive her but Tracy’s ex-crush and his new girlfriend. A mishmash of characters that somehow work together nicely.

The whole movie in the end is sort of depressing, but hey, that’s life. Greta Girwig is a really beautiful and talented actress, I’ll have to keep an eye out for this one. Definitely recommended.

Passengers

Yes, there’s a female character in the movie too.
An aesthetically pleasing sci-fi flick with a minimal cast and beautiful scenery. The starship Avalon is travelling at 0.5c towards a distant colony. The entire crew and its 5000 passengers are in deep stasis of some kind since the voyage to Homestead II takes about 120 years. Our protagonist Jim’s (Chris Pratt) stasis pod starts a reanimation sequence due to a very specific and unlikely event. The passengers are suppose to be reanimated some 4 months before arriving to the distant colony, but Jim is awaken some 88 years before the voyage ends.

There’s a lot of details in the movie, it’s a part of a new trend where the sci-fi film is realistic as possible. No FTL, no wormholing, no fancy SF shit, a ship like this is maybe possible within 10 generations, maybe. Jim sends a message back to home, but since they’re 30 years out already, it’d take approximately 17 years for the message to reach Earth and then it would have to catch up to the Avalon when they reply, so the earliest reply is in 55 years. Great, a lifetime spent in the bowels of a ship, but it’s not all so grim, he has an android bartender named Arthur (Michael Sheen) to keep him company, and has plenty of food. The movie has a weird melancholy attached to it, how everyone has a different reason for choosing such a strange life as by the time you reach the other world all your family and friends will have died. It’s the ultimate fresh start. Not going to spoil the movie further, surprisingly recommendable for a mainstream movie.

Kiss of the Damned

Now this movie is a particularly interesting take on the vampire genre by a director called Xen Cassavetes. An exquisitely fresh combination of erotic scenes, common vampire themes along with a narrative to explain and explore how an underground vampire society could co-exist with humans. Humans are still preyed upon for food, but is generally not done as the vampires are afraid their cover would be blown. The vampires drink animal blood and substitutes, live in lavish and remote houses where maids reside during the day to keep an eye out for their employers while they sleep and of course do housework. The maid in the movie has a rare genetic blood disorder which makes her completely unattractive to the vampires and that makes her very sought after.

The story begins where Paolo (the always lovable Milo Ventimigla) meets Djuna (Joséphine de La Baume), a hot vampire. They of course, fall in love and very soon proceed to have hot human-vampire sex. She’s shy at first because she has to reveal her true form and insists Paolo chains her to her bed so she can’t hurt him. She turns and the shackles are barely able to hold her sexy demonic form on the bed. This is a major turn on for Paolo who promptly unchains her and they have sex. Oh, she technically kills him in the process but it’s all right since he’s now a vampire. His senses are heightened, can’t stand the daylight and is always battling the urge to suck a human dry.

They’re a match made in heaven, all they have to do is fuck all night and be philosophical since Djuna has a very wealthy and influential vampire upper society friend who lets them use her fabulous house somewhere in the country. They got no jobs, no bills, very convenient for a love story. Their seemingly peaceful (un)life is disrupted when Djuna’s disturbed and of course hot vampire sister arrives, Mimi (Roxane Mesquida). The sisters are pretty ancient, hundreds of years or so and are always fighting. Mimi seems to resent the fact that Djuna turned a human as they swore they’d never do something like that. Mimi is a disruptive element to anyone that has anything to do with her and causes chaos for everyone. Her story line has a great ending, not gonna spoil that one for you.

In essence, a totally unheard of movie which I personally kind of liked. Definitely recommended if you don’t only favor movies with an ensemble cast or a strong and popular lead. Decent exploration of the vampire theme, actually pretty decent acting in a way, and a surprising amount of eroticism in it.