Centralized SSH Authentication With Redis

5 min readOct 25, 2020

SSH is an undeniably popular protocol that we use for remote access to computer instances that we own (or maybe not?). It’s secure, simple to configure, and it comes built in with the majority of modern OS that we have right now for both client and server. It supports a couple methods of authentication, most notably password and public key based authentication. Granting access to a user is as simple as creating a user account on the instance and whitelist the user’s public key by adding it as a new line on ~/.ssh/authorized_keys

However like everything else, it’s getting complicated with scale. How do we grant access for hundreds of users to hundreds or thousands of computing instances? And yet that amount is completely normal now with most organizations that have systems serving thousands of users per day.

The simplest way would be creating a script to update authorized_keys file to the machines. We can use automation platforms such as Ansible, Puppeteer, Chef, etc. Heck, we can even update that file periodically by setting a CRON job running rsync. There are countless ways to get around this limitation.

But well, while these solutions work, they lack elegance and efficiency. For most cases, what we truly need is a way to somehow make SSH query our database system to determine if a user can pass or not. Then we can of course build a complex system on top of that database to define the flow that we need. Either it’s to add some kind of approval pipeline, implementing RBAC (Role Based Access) for users based on their groups/departments/etc, we can do that easily on the application. The good news is, we can do this without hassle, thanks to the flexibility of OpenSSH.

Authorized Keys Command to The Rescue

By default, OpenSSH will lookup for authorized keys in a file defined by AuthorizedKeysFile in the server config sshd_config. However, OpenSSH got an alternative called AuthorizedKeysCommand which provide the same function in different way. Instead of reading the file, OpenSSH will execute the executable provided in AuthorizedKeysCommand with arguments of your choice. Then the executable is expected to return a list of public keys, one per line, to the stdout. That is your regular print or cout etc.

That means we can do anything in the executable, whether it’s a query to database, HTTP call, FTP, anything you can implement in your favorite programming language. But of course, it would be better if the authentication method doesn’t slow down the process by a huge margin and doesn’t break for your scale.

Redis: Fast enough for most requirements

You don’t need to use Redis. But for my case, I think Redis suit my need best. It’s a fast key value storage, which means you can store a key and associate a value to it. The value can be anything serializable and the key is a string. OpenSSH will just provide the username during the authentication process anyway, so we don’t really need a robust and complex query system that other database solution provides.
(Also read: How fast is redis?)

The Implementation

First of all, let’s implement the authentication script. I will be using Python 3 for this.

Simple enough right? We can breakdown what the script does into these steps:

Get the username from the execution argument
Get the computer hostname
Connect to redis and get members of the key <hostname>_<username>
Print out the keys

Note: this implementation is laughably not secure. Anyone would be able to change the value in redis and grant access to anyone to any server. Make sure to wrap the redis access into another service. For example, you can wrap it into an HTTP service and simply call the endpoint in the script.

So to test if our script is working properly, we will add a dummy public key on the redis instance.

ubuntu@Gabriel-PC:~$ redis-cli
127.0.0.1:6379> sadd Gabriel-PC_ubuntu public_key
(integer) 1

Give the executable permission to the script with chmod and make root user own it with chown

ubuntu@Gabriel-PC:~$ sudo chmod 755 /etc/ssh/auth.py
ubuntu@Gabriel-PC:~$ sudo chown root:root /etc/ssh/auth.py

Install the dependencies (redis library)

python3 -m pip install redis

And fire it!

ubuntu@Gabriel-PC:~$ /etc/ssh/auth.py ubuntu
public_key

Looking good! Now let’s move on to the SSH server config.

SSH Server Configuration

The configuration is also simple, what you need to do is delete the AuthorizedKeysFile directive and replace it with AuthorizedKeysCommand. By default the configuration file should be located in /etc/ssh/sshd_config.

...
#LoginGraceTime 2m
#PermitRootLogin prohibit-password
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10PubkeyAuthentication yes# Expect .ssh/authorized_keys2 to be disregarded by default in future.
#AuthorizedKeysFile     .ssh/authorized_keys .ssh/authorized_keys2#AuthorizedPrincipalsFile noneAuthorizedKeysCommand /etc/ssh/auth.py %u                                                                                        AuthorizedKeysCommandUser root
...

As you can see, from my default configuration, I did two things:

Uncomment the PubkeyAuthentication yes
Uncomment the AuthorizedKeysCommand and add the executable path + %u as the argument (token for user argument)
Uncomment the AuthorizedKeysCommandUser and add a username. This would be the executor for the command.
Another optional step is to add LogLevel INFO to make debugging easier

After saving the configuration above, you can restart/reload the ssh service. Restart will well.. restart the SSH service while reload will just load the latest config. I’m using Ubuntu 18.04 and the command to do so is

sudo service ssh restart

Or alternatively if you want to debug, you can stop the service and start it manually. Don’t forget to start the service again after you’re done debugging.

ubuntu@Gabriel-PC:~$ sudo service ssh stop
 * Stopping OpenBSD Secure Shell server sshd                                                                     [ OK ]
ubuntu@Gabriel-PC:~$ sudo /usr/sbin/sshd -d
debug1: sshd version OpenSSH_7.6, OpenSSL 1.0.2n  7 Dec 2017
debug1: private host key #0: ssh-rsa SHA256:xygmXSTGgwKX46CG42khhP7dO8vRk4oGtdIRFBXrcFI
debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:UsEHbhhZUr79/9rjY+T6DpsVEx8kdUlaeb9yXN1pq9w
debug1: private host key #2: ssh-ed25519 SHA256:odASZpd3w/x/KsZdPh4/wBAzTJbRO6vrNnZ9ZVLBp5A
debug1: rexec_argv[0]='/usr/sbin/sshd'
debug1: rexec_argv[1]='-d'
debug1: Set /proc/self/oom_score_adj from 0 to -1000
debug1: Bind to port 22 on 0.0.0.0.
Server listening on 0.0.0.0 port 22.
debug1: Bind to port 22 on ::.
Server listening on :: port 22.

And done! Test out the authentication with an SSH client of your choice. Play around with the value in redis to ensure that the script works properly.

Conclusion

Customizing your SSH server authentication flow is easy with the AuthorizedKeysCommand directive. The next step would be for example, building an interface for access request and approval, making a role system to enable only certain roles for certain servers, etc.