blog/source/_posts/ssh-certificate-cloudflare-tunnel.md at 772df1da56ead3c51dee9af8c6e1a8c94c6483bf

10 KiB

Raw Blame History

title

excerpt

date

updated

Introduction

One unpleasant task I had previously in an enterprise with Linux servers was SSH key management, specifically checking the SSH public keys of departed staff have been removed from the Ansible config. Then I learned from this article that it is possible to SSH using a short-lived (<1 day) certificate that is only issued to the user after successfully authenticate with the enterprise identity provider's (e.g. Azure AD) single sign-on (SSO). This means once a user is revoked from the identity provider, that user would not be issued with a new certificate to SSH again the next day. At that time, I didn't feel like configuring and integrating an identity provider, so I held off trying the feature.

Recently, I wanted to try out the Cloudflare Zero Trust free tier. While reading through the SSH configuration guide, I found out that Cloudflare support issuing SSH user certificate. While Cloudflare supports several SSO integration, it also supports authenticating using one-time PIN sent to an email address that does not have to be a Cloudflare account. Cloudflare also supports browser-based shell, just like the AWS Session Manager.

Prerequisites

A domain hosted on Cloudflare DNS
Cloudflare Zero Trust (free for 50 users)
A VM or cloud instance (optional, easier to clean up)

Cloudflare Zero Trust

Navigate to Zero Trust page shown on the sidebar after you login to dash.cloudflare.com. If this is your first time, Cloudflare will ask for billing info in which you can use an existing one or add a new credit card. You won't get charged as long as you stay within the free tier (50 users), I will show you how to check later in this article.

The setup will then ask you to name your team domain team-name.cloudflareaccess.com. Just create a random name for now, you can always change it later.

Add an application

Once you're in Zero Trust console, navigate to Access -> Applications. Add an application and choose Self-hosted.

Configure app tab,

Application name: any name
Session duration: 15 minutes.
- In a corporate environment, "6 hours" is probably more user-friendly.
- For sensitive server, consider "No duration".
Application domain: test.yourdomain.com
- The subdomain should not have an existing website.
- It may be possible to use an existing website, by specifying test.yourdomain.com/custom-path for SSH, though I haven't try it.
App Launcher visbility: No
Accept all available identity providers: No, unless you have integrated an identity provider.
Select One-time PIN
Instant Auth: Yes

Add policies tab,

Policy name: any name
Action: Allow
Session duration: same
Configure rules: (Include) Emails = an email address
- Any of your email is fine, regardless whether it's a Cloudflare account.
- Cloudflare will not create an account using that email, it will only be used to receive one-time PIN.

Setup tab:

CORS settings: leave it as is
Cookies settings:
- SameSite Attribute: blank or Lax
  - Either setting is practically the same, browsers default to Lax when SameSite is not set.
  - "Strict" value cannot be used because Cloudflare will authenticate the user on team-name.cloudflareaccess.com and issue a cookie on test.yourdomain.com.
- HTTP Only: Yes
Additional settings:
- Enable automatic cloudflared authentication: Yes
- Browser rendering: SSH

Generate a CA certificate

Navigate to Access -> Service Auth -> SSH tab. Select the application you just created and Generate certificate.

Copy the generated public key and save it to /etc/ssh/ca.pub in your host (the host you're going to SSH into).

sudo -e /etc/ssh/ca.pub

Create a tunnel

Navigate to Access -> Tunnels

Name: any name

Install connector tab, choose the relevant OS and run the installation command. Once installed, you should see "connected" status.

Route tunnel tab,

Public hostname: test.yourdomain.com
- This is the application domain in the Add an application step.
Service
- SSH type: URL = localhost:22
  - Replace 22 with the custom SSH port you are going to use.

After finishing creating a tunnel, you should have a new CNAME DNS record that points to tunnel-id.cfargotunnel.com. If there is no CNAME entry, grab the tunnel ID and create a new DNS record.

Start SSH server

Install openssh-server.

sudo -e /etc/ssh/sshd_config.d/cf.conf

TrustedUserCAKeys /etc/ssh/ca.pub
ListenAddress 127.0.0.1
ListenAddress ::1
PasswordAuthentication no
# Uncomment below line for custom port
# Port 1234

systemctl restart ssh or systemctl restart sshd

Create a test user

The easiest setup is one where a Unix username matches the email that you configured to receive one-time PIN in previous steps. For example, if you set loremipsum@youremail.com, then create a new user loremipsum.

sudo adduser loremipsum

Set a random password and leave everything else blank.

Matching email to different username

To match loremipsum@youremail.com to lipsum user:

Match user lipsum
  AuthorizedPrincipalsCommand /bin/echo 'loremipsum'
  AuthorizedPrincipalsCommandUser nobody

loremipsum+somealias@youremail.com also works.

Match user lipsum
  AuthorizedPrincipalsCommand /bin/echo 'loremipsum+somealias'
  AuthorizedPrincipalsCommandUser nobody

AuthorizedPrincipalsFile

For NixOS user, AuthorizedPrincipalsCommand will not work because the command will run within "/nix/store" but it is read-only. Instead, you should use AuthorizedPrincipalsFile. This config also enables you to match multiple emails to a username, just separate each email user by newline. This applies to all OpenSSH instances, not just NixOS.

echo 'loremipsum' | sudo tee /etc/ssh/authorized_principals

  services.openssh = {
    enable = true;
    permitRootLogin = "no";
    passwordAuthentication = false;
    # ports = [ 1234 ];
    extraConfig =
      ''
        TrustedUserCAKeys /etc/ssh/ca.pub
        Match User lipsum
          AuthorizedPrincipalsFile /etc/ssh/authorized_principals
          # if there is no existing AuthenticationMethods
          AuthenticationMethods publickey
      '';
  };
```

### Other use cases

https://developers.cloudflare.com/cloudflare-one/identity/users/short-lived-certificates/#2-ensure-unix-usernames-match-user-sso-identities

## Initiate SSH connection

Install `cloudflared` on the host that you're going to SSH from.

`cloudflared access ssh-config --hostname test.yourdomain.com --short-lived-cert`

Example output:

```plain ~/.ssh/config
Match host test.yourdomain.com exec "/usr/local/bin/cloudflared access ssh-gen --hostname %h"
    ProxyCommand /usr/local/bin/cloudflared access ssh --hostname %h
    IdentityFile ~/.cloudflared/%h-cf_key
    CertificateFile ~/.cloudflared/%h-cf_key-cert.pub

Host test.yourdomain.com
    ProxyCommand bash -c '/usr/local/bin/cloudflared access ssh-gen --hostname %h; ssh -tt %r@cfpipe-test.yourdomain.com >&2 <&1'

Host cfpipe-test.yourdomain.com
    HostName test.yourdomain.com
    ProxyCommand /usr/local/bin/cloudflared access ssh --hostname %h
    IdentityFile ~/.cloudflared/test.yourdomain.com-cf_key
    CertificateFile ~/.cloudflared/test.yourdomain.com-cf_key-cert.pub

Save the output to $HOME/.ssh/config.

Now, the moment of truth.

ssh loremipsum@test.yourdomain.com (replace the username with the one you created in Create a test user step.)

The terminal should launch a website to team-name.cloudflareaccess.com. Enter the email you configured in Add an application step and then enter the received 6-digit PIN.

Back to the terminal, wait for at least 5 seconds and you should see the usual SSH authentication.

You may wondering why you still see fingerprint warning, I find this article SSH Best Practices using Certificates, 2FA and Bastions explains it well.

Browser-based shell

As a bonus, head to test.yourdomain.com (see Add an application step) which will redirect you to a login page just the previous step. After login with a 6-digit PIN, you shall see a browser-based shell.

Usage monitoring

Head to Settings -> Account to monitor how many users you have, each email address you configured to receive one-time PIN is counted as one user.

To delete user(s), head to Users, tick the relevant users, Update status and then Remove. The seat usage column should show Inactive.

Inspect user certificate

ssh-keygen -L -f ~/.cloudflared/test.yourdomain.com-cf_key-cert.pub

10 KiB Raw Blame History