Categories
Django

Setting up the Server (on Digital Ocean)

The initial setup

Digital Ocean has a pretty nice API which makes it easy to automate the creation of their servers (which they call Droplets. This is nice when you’re trying to work towards automation of the entire process (like I was).

I won’t jump into the automation piece just yet, but once you have your DO account setup (sign up here if you don’t have one), it’s a simple interface to Setup Your Droplet.

I chose the Ubuntu 18.04 LTS image with a $5 server (1GB Ram, 1CPU, 25GB SSD Space, 1000GB Transfer) hosted in their San Francisco data center (SFO21).

We’ve got a server … now what?

We’re going to want to update, upgrade, and install all of the (non-Python) packages for the server. For my case, that meant running the following:

apt-get update
apt-get upgrade
apt-get install python3 python3-pip python3-venv tree postgresql postgresql-contrib nginx

That’s it! We’ve now got a server that is ready to be setup for our Django Project.

In the next post, I’ll walk through how to get your Domain Name to point to the Digital Ocean Server.

  1. SFO2 is disabled for new customers and you will now need to use SFO3 unless you already have resources on SFO2, but if you’re following along you probably don’t. What’s the difference between the two? Nothing 😁
Categories
Django

Writing tests for Django Admin Custom Functionality

I’ve been working on a Django app side project for a while and came across the need to write a custom filter for the Django Admin section.

This was a first for me, and it was pretty straight forward to accomplish the task. I wanted to add a filter on the drop down list so that only certain records would appear.

To do this, I sub-classed the Django Admin SimpleListFilter with the following code:

class EmployeeListFilter(admin.SimpleListFilter):
    title = "Employee"
    parameter_name = "employee"

    def lookups(self, request, model_admin):
        employees = []
        qs = Employee.objects.filter(status__status="Active").order_by("first_name", "last_name")
        for employee in qs:
            employees.append((employee.pk, f"{employee.first_name} {employee.last_name}"))
        return employees

    def queryset(self, request, queryset):
        if self.value():
            qs = queryset.filter(employee__id=self.value())
        else:
            qs = queryset
        return qs

And implemented it like this:

@admin.register(EmployeeO3Note)
class EmployeeO3NoteAdmin(admin.ModelAdmin):
    list_filter = (EmployeeListFilter, "o3_date")

This was, as I said, relatively straight forward to do, but what was less clear to me was how to write tests for this functionality. My project has 100% test coverage, and therefore testing isn’t something I’m unfamiliar with, but in this context, I wasn’t sure where to start.

There are two parts that need to be tested:

  1. lookups
  2. queryset

Additionally, the queryset has two states that need to be tested

  1. With self.value()
  2. Without self.value()

This gives a total of 3 tests to write

The thing that helps me out the most when trying to determine how to write tests is to use the Django Shell in PyCharm. To do this I:

  1. Import necessary parts of Django App
  2. Instantiate the EmployeeListFilter
  3. See what errors I get
  4. Google how to fix the errors
  5. Repeat

This is what the test ended up looking like:

import pytest

from employees.models import EmployeeO3Note
from employees.tests.factories import EmployeeFactory, EmployeeO3NoteFactory, EmployeeStatusFactory
from employees.admin import EmployeeListFilter


ACTIVE_EMPLOYEES = 3
TERMED_EMPLOYEES = 1


@pytest.fixture
def active_employees():
    return EmployeeFactory.create_batch(ACTIVE_EMPLOYEES)


@pytest.fixture
def termed_employees():
    termed_employees = TERMED_EMPLOYEES
    termed = EmployeeStatusFactory(status="Termed")
    return EmployeeFactory.create_batch(termed_employees, status=termed)


@pytest.fixture
def o3_notes_for_all_employees(active_employees, termed_employees):
    all_employees = active_employees + termed_employees
    o3_notes = []
    for i in range(len(all_employees)):
        o3_notes.append(EmployeeO3NoteFactory.create_batch(1, employee=all_employees[i]))
    return o3_notes


@pytest.mark.django_db
def test_admin_filter_active_employee_o3_notes(active_employees):
    employee_list_filter = EmployeeListFilter(request=None, params={}, model=None, model_admin=None)
    assert len(employee_list_filter.lookup_choices) == ACTIVE_EMPLOYEES


@pytest.mark.django_db
def test_admin_query_set_unfiltered_results_o3_notes(o3_notes_for_all_employees):
    total_employees = ACTIVE_EMPLOYEES + TERMED_EMPLOYEES
    employee_list_filter = EmployeeListFilter(request=None, params={}, model=None, model_admin=None)
    assert len(employee_list_filter.queryset(request=None, queryset=EmployeeO3Note.objects.all())) == total_employees


@pytest.mark.django_db
def test_admin_query_set_filtered_results_o3_notes(active_employees, o3_notes_for_all_employees):
    employee_to_test = active_employees[0]
    employee_list_filter = EmployeeListFilter(
        request=None, params={"employee": employee_to_test.pk}, model=None, model_admin=None
    )
    queryset_to_test = employee_list_filter.queryset(request=None, queryset=EmployeeO3Note.objects.all())
	assert len(queryset_to_test.filter(employee__id=employee_to_test.pk)) == 1
Categories
Django

Deploying a Django Site to Digital Ocean – A Series

Previous Efforts

When I first heard of Django I thought it looks like a really interesting, and Pythonic way, to get a website up and running. I spent a whole weekend putting together a site locally and then, using Digital Ocean, decided to push my idea up onto a live site.

One problem that I ran into, which EVERY new Django Developer will run into was static files. I couldn’t get static files to work. No matter what I did, they were just … missing. I proceeded to spend the next few weekends trying to figure out why, but alas, I was not very good (or patient) with reading documentation and gave up.

Fast forward a few years, and while taking the 100 Days of Code on the Web Python course from Talk Python to Me I was able to follow along on a part of the course that pushed up a Django App to Heroku.

I wrote about that effort here. Needless to say, I was pretty pumped. But, I was wondering, is there a way I can actually get a Django site to work on a non-Heroku (PaaS) type infrastructure.

Inspiration

While going through my Twitter timeline I cam across a retweet from TestDrive.io of Matt Segal. He has an amazing walk through of deploying a Django site on the hard level (i.e. using Windows). It’s a mix of Blog posts and YouTube Videos and I highly recommend it. There is some NSFW language, BUT if you can get past that (and I can) it’s a great resource.

This series is meant to be a written record of what I did to implement these recommendations and suggestions, and then to push myself a bit further to expand the complexity of the app.

Articles

A list of the Articles will go here. For now, here’s a rough outline of the planned posts:

The ‘Enhancements’ will be multiple follow up posts (hopefully) as I catalog improvements make to the site. My currently planned enhancements are:

Categories
Django

Django form filters

I’ve been working on a Django Project for a while and one of the apps I have tracks candidates. These candidates have dates of a specific type.

The models look like this:

Candidate

class Candidate(models.Model):
    first_name = models.CharField(max_length=128)
    last_name = models.CharField(max_length=128)
    resume = models.FileField(storage=PrivateMediaStorage(), blank=True, null=True)
    cover_leter = models.FileField(storage=PrivateMediaStorage(), blank=True, null=True)
    email_address = models.EmailField(blank=True, null=True)
    linkedin = models.URLField(blank=True, null=True)
    github = models.URLField(blank=True, null=True)
    rejected = models.BooleanField()
    position = models.ForeignKey(
        "positions.Position",
        on_delete=models.CASCADE,
    )
    hired = models.BooleanField(default=False)

CandidateDate

class CandidateDate(models.Model):
    candidate = models.ForeignKey(
        "Candidate",
        on_delete=models.CASCADE,
    )
    date_type = models.ForeignKey(
        "CandidateDateType",
        on_delete=models.CASCADE,
    )
    candidate_date = models.DateField(blank=True, null=True)
    candidate_date_note = models.TextField(blank=True, null=True)
    meeting_link = models.URLField(blank=True, null=True)

    class Meta:
        ordering = ["candidate", "-candidate_date"]
        unique_together = (
            "candidate",
            "date_type",
        )

CandidateDateType

class CandidateDateType(models.Model):
    date_type = models.CharField(max_length=24)
    description = models.CharField(max_length=255, null=True, blank=True)

You’ll see from the CandidateDate model that the fields candidate and date_type are unique. One problem that I’ve been running into is how to help make that an easier thing to see in the form where the dates are entered.

The Django built in validation will display an error message if a user were to try and select a candidate and date_type that already existed, but it felt like this could be done better.

I did a fair amount of Googling and had a couple of different bright ideas, but ultimately it came down to a pretty simple implementation of the exclude keyword in the ORM

The initial Form looked like this:

class CandidateDateForm(ModelForm):
   class Meta:
        model = CandidateDate
        fields = [
            "candidate",
            "date_type",
            "candidate_date",
            "meeting_link",
            "candidate_date_note",
        ]
        widgets = {
            "candidate": HiddenInput,
        }

I updated it to include a __init__ method which overrode the options in the drop down.

def __init__(self, *args, **kwargs):
    super(CandidateDateForm, self).__init__(*args, **kwargs)
    try:
        candidate = kwargs["initial"]["candidate"]
        candidate_date_set = CandidateDate.objects.filter(candidate=candidate).values_list("date_type", flat=True)
        qs = CandidateDateType.objects.exclude(id__in=candidate_date_set)
        self.fields["date_type"].queryset = qs
    except KeyError:
        pass

Now, with this method the drop down will only show items which can be selected, not all CandidateDateType options.

Seems like a better user experience AND I got to learn a bit about the Django ORM

Categories
Django

Using different .env files

In a Django project I’m working on I use a library called Django-environ which

allows you to utilize 12factor inspired environment variables to configure your Django application.

It’s a pretty sweet library as well. You create a .env file to store your variable that you don’t want in a public repo for your settings.py.

The big issue I have is that my .env file for my local development isn’t what I want on my production server (obviously … never set DEBIG=True in production!)

I had tried to use a different .env file using an assortment of methods, but to no avail. And the documentation wasn’t much of a help for using Multiple env file

It is possible to have multiple env files and select one using environment variables.

Now ENV_PATH=other-env ./manage.py runserver uses other-env while ./manage.py runserver uses .env.

But there’s no example about how to actually set that up 🤦🏻‍♂️1.

In fact, this bit in the documentation reminded me of thisvideo on YouTube.

Instead of trying to figure out the use of multiple .env files I instead used a just recipe in my justfile to get the job done.

# checks the deployment for prod settings; will return error if the check doesn't pass
check:
    cp core/.env core/.env_staging
    cp core/.env_prod core/.env
    -python manage.py check --deploy
    cp core/.env_staging core/.env

OK. What does this recipe do?

First, we copy the development .env file to a .env_staging file to keep the original development settings ‘somewhere’

 cp core/.env core/.env_staging

Next, we copy the .env_prod to the .env so that we can use it when we run -python manage.py check --deploy.

cp core/.env_prod core/.env
-python manage.py check --deploy

Why do we use the -? That allows the justfile to keep going if it runs into an error. Since we’re updating our main .env file I want to make sure it gets restored to its original state … just in case!

Finally, we copy the original contents of the .env file from the .env_staging back to the .env to restore it to its development settings.

Now, I can simply run

just check

And I’ll know if I have passed the 12 factor checking for my Django project or somehow introduced something that makes the check not pass.

I’d like to figure out how to set up multiple .env files, create an example and contribute to the docs … but honestly I have no freaking clue how to do it. If I am able to figure it out, you can bet I’m going to write up a PR for the docs!

  1. I’d like to figure out how to set up multiple .env files, create an example and contribute to the docs … but honestly I have no freaking clue how to do it. If I am able to figure it out, you can bet I’m going to write up a PR for the docs!