Asyncronous Django

Django has slowly but steadily been adding support for async Python. To get access to this functionality Django needs to be deployed with an ASGI server.

At this stage async support only applies to the outer ASGI application. Internally everything remains synchronous. Asynchronous middleware, views, etc. are not yet supported. You can, however, use ASGI middleware around Django’s application, allowing you to combine Django with other ASGI frameworks.

Support for async views, middleware and ORM are not yet there but a Django 3.1 LTS release will be coming out around august and hopefully some of these will be supported by then.

Asyncronous benefits

Great benefits are to be had once full async support arrives in Django.

• Parallel ORM queries
• Views could query external API’s without blocking threads
• Running slow-response / long-poll endpoints efficiently

Until then there is no need to switch your applications over unless you want to start experimenting with asynchronous code.

ASGI

We are all used to deploying Django with WSGI by gunicorn on uWSGI. ASGI is a successor to WSGI and translates as Asynchronous Server Gateway Interface - meant to support asyncronous Python.

A very nice talk “An introduction to ASGI” was given by Philip Jones at PyLondinium 2019

Firs 5 minutes on a server

After you have provisioned your fresh server for deployment you should follow a guide to lock down your server before we begin the actual deployment process.

Update the repositories and upgrade your system.

apt update
apt upgrade
apt install autoconf build-essential python3-dev

Install Fail2ban

Fail2ban scans log files and bans IPs that show the malicious signs.

apt-get install fail2ban

Deployment user

useradd -m deploy -s /bin/bash
usermod -aG sudo deploy

As we are not using passwords allow to use sudo withour a password.

visudo

Add NOPASSWD:ALL at the end of the sudo group configuration.

%sudo ALL=(ALL:ALL) NOPASSWD:ALL

SSH keys

To enhance your server’s security, it’s strongly recommend setting up SSH key authentication instead of using passwords.

Add your SSH key to the .ssh/authorized_keys in the created deploy users home folder.

mkdir /home/deploy/.ssh
nano /home/deploy/.ssh/authorized_keys
chmod 400 /home/deploy/.ssh/authorized_keys
chown deploy:deploy /home/deploy -R

Lock down SSH

nano /etc/ssh/sshd_config

Disable root login & dont allow password authentication.

PermitRootLogin no
PasswordAuthentication no

Restart the ssh service for the changes to take effect.

service ssh restart

Firewall setup

ufw app list
ufw allow OpenSSH
ufw enable
ufw status

Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)

Test the deploy user

Do not shut down the root SSH window before you have fully tested that the setup we just did works. You can be locked out from your server without any access.

To test just access the server with the created user via ssh and run a random command with sudo access.

ssh deploy@<yourip>
sudo ls /

If you’d like to read more extensively on securing your server a “Guide to User Data Security” by Brian Pontarelli is an extensive writeup.

Python

NB! Everything from here on should be done with the deploy user.

Lets deploy our application with the latest stable Python version 3.8. Django 3 supports Python 3.6+.

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install git
sudo apt-get install python3.8 python3.8-dev python3.8-distutils python3-pip
python3.8 -m pip install --upgrade pip setuptools wheel
python3.8 -m pip install virtualenv

Application

Make a folder to deploy your application into.

sudo mkdir /srv/<application name>/ /srv/<application name>/log/
sudo chown deploy:deploy /srv/<application name>/
cd /srv/<application name>/
python3.8 -m virtualenv env
source env/bin/activate
git clone [email protected]:<username>/<repository>.git app
cd app/
pip install -r requirements.txt --upgrade

NGINX

Install NGINX, make sure it’s running and remove the default site configuration.

sudo apt install nginx
sudo service nginx status
sudo rm /etc/nginx/sites-enabled/default
sudo service nginx reload
sudo ufw allow 'Nginx Full'
sudo ufw reload

LetEncryt

sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository universe
sudo add-apt-repository ppa:certbot/certbot
sudo apt update
sudo apt install certbot python-certbot-nginx

sudo certbot certonly --nginx

Follow the instructions on screen to configure your certificate & domain. Also remember to add the domain you are provisioning the certificate for be pointing at your new server.

Generate DH parameters.

sudo openssl dhparam -out /etc/nginx/dhparam.pem 4096

This is going to take time. 2048 could be used but SSL Labs requires a 4096 bit key to get a 100% score for Key Exchange.

Test SSL certificate automatic renewal.

sudo certbot renew --dry-run

Renewal setup?

Proxy configuration

Create our domain specific proxy configuration for NGINX.

sudo nano /etc/nginx/sites-available/upcount

server {
  listen 443;
  server_name api.upcount.app;

  ssl_certificate /etc/letsencrypt/live/api.upcount.app/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/api.upcount.app/privkey.pem;

  ssl on;
  ssl_session_cache shared:SSL:10m;
  ssl_session_timeout 10m;
  ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
  ssl_ciphers 'kEECDH+ECDSA+AES128 kEECDH+ECDSA+AES256 kEECDH+AES128 kEECDH+AES256 kEDH+AES128 kEDH+AES256 DES-CBC3-SHA +SHA !aNULL !eNULL !LOW !kECDH !DSS !MD5 !RC4 !EXP !PSK !SRP !CAMELLIA !SEED';
  ssl_prefer_server_ciphers on;
  ssl_dhparam /etc/nginx/dhparam.pem;

  location / {
    proxy_pass http://localhost:8000;
    proxy_redirect off;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Ssl on;
  }

  location /static/ {
    root /srv/upcount/app;
  }
}

Link to enabled sites.

sudo ln -s /etc/nginx/sites-available/<application> /etc/nginx/sites-enabled/<application>

Gunicorn + Uvicorn worker

Uvicorn currently in production should be run with Gunicorn based on the Uvicorn deployment guide.

Uvicorn includes a Gunicorn worker class allowing you to run ASGI applications, with all of Uvicorn’s performance benefits, while also giving you Gunicorn’s fully-featured process management.

This allows you to increase or decrease the number of worker processes on the fly, restart worker processes gracefully, or perform server upgrades without downtime.

So make sure your requirements.txt contains both.

Running your app

For a typical Django project, invoking Gunicorn with a Uvicorn worker for production

gunicorn myproject.asgi:application  -w 4 -k uvicorn.workers.UvicornWorke

This would launch the application with 4 workers. If you want to know more about “How many workers?” please read the Gunicorn ducumentation for more details.

Keeping the app running

To keep your application running you can use systemd - a service manager already installed on Ubuntu 18.04.

There are many options but the main reason to use it is because it’s already there and installed. Very well tested and widely used.

Lets create a systemd service configiration file.

sudo nano /etc/systemd/system/upcount.service

[Unit]
Description=Upcount API
After=network.target

[Service]
User=deploy
Group=deploy
WorkingDirectory=/srv/upcount/app
ExecStart=/srv/upcount/env/bin/gunicorn myproject.asgi:application  -w 4 -k uvicorn.workers.UvicornWorker
Restart=always

[Install]
WantedBy=multi-user.target

After adding this configuration, you can start the service

sudo systemctl daemon-reload
sudo systemctl start upcount
sudo systemctl status upcount

PostgreSQL

Install postgres

sudo apt install postgresql

Create a user ‘deploy’ with the interactive createuser command.

sudo -u postgres createuser --interactive

And create a database for the application

sudo -u postgres createdb upcount

Set a password for the created role

sudo -u postgres psql

ALTER USER user_name WITH PASSWORD 'new_password';

Visit your site & continue development!

These steps complete the Django setup. Visit your site and enjoy - start coding your application features!