Konstruktor

Deploying asynchronous Django

March 01, 2020

Asyncronous Django

Django has slowly but steadily been adding support for async Python. To get access to this functionality Django needs to be deployed with an ASGI server.

At this stage async support only applies to the outer ASGI application. Internally everything remains synchronous. Asynchronous middleware, views, etc. are not yet supported. You can, however, use ASGI middleware around Django’s application, allowing you to combine Django with other ASGI frameworks.

Support for async views, middleware and ORM are not yet there but a Django 3.1 LTS release will be coming out around august and hopefully some of these will be supported by then.

Asyncronous benefits

Great benefits are to be had once full async support arrives in Django.

  • Parallel ORM queries
  • Views could query external API’s without blocking threads
  • Running slow-response / long-poll endpoints efficiently

Until then there is no need to switch your applications over unless you want to start experimenting with asynchronous code.

ASGI

We are all used to deploying Django with WSGI by gunicorn on uWSGI. ASGI is a successor to WSGI and translates as Asynchronous Server Gateway Interface - meant to support asyncronous Python.

A very nice talk “An introduction to ASGI” was given by Philip Jones at PyLondinium 2019

Firs 5 minutes on a server

After you have provisioned your fresh server for deployment you should follow a guide to lock down your server before we begin the actual deployment process.

Update the repositories and upgrade your system.

apt update
apt upgrade

Install Fail2ban

Fail2ban scans log files and bans IPs that show the malicious signs.

apt-get install fail2ban

Deployment user

useradd -m deploy -s /bin/bash
usermod -aG sudo deploy

As we are not using passwords allow to use sudo withour a password.

visudo
```

Add NOPASSWD:ALL at the end of the sudo group configuration.

````ini
%sudo ALL=(ALL:ALL) NOPASSWD:ALL

SSH keys

To enhance your server’s security, it’s strongly recommend setting up SSH key authentication instead of using passwords.

Add your SSH key (contents of the idrsa.pub on your local machine) to the .ssh/authorizedkeys in the created users home folder.

mkdir /home/deploy/.ssh
nano /home/deploy/.ssh/authorized_keys
chmod 400 /home/deploy/.ssh/authorized_keys
chown deploy:deploy /home/deploy -R

Lock down SSH

nano /etc/ssh/sshd_config

Disable root login & dont allow password authentication.

PermitRootLogin no
PasswordAuthentication no

Restart the ssh service for the changes to take effect.

service ssh restart

Firewall setup

ufw app list
ufw allow OpenSSH
ufw enable
ufw status
Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)

Test the deploy user

Do not shut down the root SSH window before you have fully tested that the setup we just did works. You can be locked out from your server without any access.

To test just access the server with the created user via ssh and run a random command with sudo access.

ssh [email protected]<yourip>
sudo ls /

If you’d like to read more extensively on securing your server a “Guide to User Data Security” by Brian Pontarelli is an extensive writeup.

NB! Everything from here on should be done with the deploy user.

Python

Lets deploy our application with the latest stable Python version 3.8. Django 3 supports Python 3.6+.

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install python3.8 python3.8-dev python3.8-distutils python3-pip
python3.8 -m pip install --upgrade pip setuptools wheel
python3.8 -m pip install virtualenv

Application

Make a folder to deploy your application into.

sudo mkdir /srv/upacount/ /srv/upacount/log/
sudo chown deploy:deploy /srv/upacount/
cd mkdir /srv/upacount/
python3.8 -m virtualenv env
git clone [email protected]:madisvain/upcount-api.git app
cd app/
pip install -r requirements.txt --upgrade

NGINX

Install NGINX, make sure it’s running and remove the default site configuration.

sudo apt install nginx
sudo service nginx status
sudo rm /etc/nginx/sites-enabled/default
sudo service nginx reload
sudo ufw allow 'Nginx Full'
sudo ufw reload

LetEncryt

sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository universe
sudo add-apt-repository ppa:certbot/certbot
sudo apt update
sudo apt install certbot python-certbot-nginx

sudo certbot certonly --nginx

Follow the instructions on screen to configure your certificate & domain.

Generate DH parameters.

sudo openssl dhparam -out /etc/nginx/dhparam.pem 4096

This is going to take time. 2048 could be used but SSL Labs requires a 4096 bit key to get a 100% score for Key Exchange.

Test automatic renewal.

sudo certbot renew --dry-run

Proxy configuration

Create our domain specific proxy configuration for NGINX.

nano /etc/nginx/sites-available/upcount
server {
  listen 443;
  server_name api.upcount.app;

  ssl_certificate /etc/letsencrypt/live/api.upcount.app/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/api.upcount.app/privkey.pem;

  ssl on;
  ssl_session_cache shared:SSL:10m;
  ssl_session_timeout 10m;
  ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
  ssl_ciphers 'kEECDH+ECDSA+AES128 kEECDH+ECDSA+AES256 kEECDH+AES128 kEECDH+AES256 kEDH+AES128 kEDH+AES256 DES-CBC3-SHA +SHA !aNULL !eNULL !LOW !kECDH !DSS !MD5 !RC4 !EXP !PSK !SRP !CAMELLIA !SEED';
  ssl_prefer_server_ciphers on;
  ssl_dhparam /etc/nginx/dhparam.pem;

  location / {
    proxy_pass http://localhost:8000;
    proxy_redirect off;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Ssl on;
  }

  location /static/ {
    root /srv/upcount/app;
  }
}

Gunicorn + Uvicorn worker

Uvicorn currently in production should be run with Gunicorn based on the Uvicorn deployment guide.

Uvicorn includes a Gunicorn worker class allowing you to run ASGI applications, with all of Uvicorn’s performance benefits, while also giving you Gunicorn’s fully-featured process management.

This allows you to increase or decrease the number of worker processes on the fly, restart worker processes gracefully, or perform server upgrades without downtime.

So make sure your requirements.txt contains both.

Running your app

For a typical Django project, invoking Gunicorn with a Uvicorn worker for production

gunicorn myproject.asgi:application  -w 4 -k uvicorn.workers.UvicornWorke

This would launch the application with 4 workers. If you want to know more about “How many workers?” please read the Gunicorn ducumentation for more details.

Keeping the app running

To keep your application running you can use systemd - a service manager already installed on Ubuntu 18.04.

There are many options but the main reason to use it is because it’s already there and installed. Very well tested and widely used.

Lets create a systemd service configiration at /etc/systemd/system/upcount.service

[Unit]
Description=Upcount API
After=network.target

[Service]
User=deploy
Group=deploy
WorkingDirectory=/srv/upcount/app
ExecStart=/srv/upcount/env/bin/gunicorn myproject.asgi:application  -w 4 -k uvicorn.workers.UvicornWorke
Restart=always

[Install]
WantedBy=multi-user.target

After adding this configuration, you can start the service

sudo systemctl daemon-reload
sudo systemctl start upcount

PostgreSQL

Install postgres

sudo apt install postgresql

Create a user ‘deploy’ with the interactive createuser command.

sudo -u postgres createuser --interactive

And create a database for the application

sudo -u postgres createdb upcount

Visit your site & continue development!

These steps complete the Django setup. Visit your site and enjoy - start coding your application features!


A blog by Madis Väin
Thoughts on product & software engineering.