Django versus Rails 01 - Getting Started

It is a new year and, well, a new framework: Django! Yes I’ve done some Django work before but its been an age since I’ve done much so I thought I’d write down the things I learn as I learn them.

The Context

I am building a small proof of concept app for an outstandingly wonderful consulting firm, Glass Ivy where we need to:

• Process data with Pandas
• Build a web user interface
• Read and write to a database

Although Glass Ivy has strong Rails credentials, this gives us an opportunity to broaden our skill base and increase our Python / Django credentials.

Step 01: Checking Out the Code

This is a normal thing:

cd (where you want to to go; I use ~/Sync/coding/django)
git clone (a git ssh url)

Step 02: Add .gitignore

As a Rails user, I know the normal things to add to .gitignore but I have no idea for Python so I sourced this from the net:

# Created by https://www.gitignore.io

### OSX ###
.DS_Store
.AppleDouble
.LSOverride

# Icon must end with two \r
Icon


# Thumbnails
._*

# Files that might appear on external disk
.Spotlight-V100
.Trashes

# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk


### Python ###
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]

# C extensions
*.so

# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.cache
nosetests.xml
coverage.xml

# Translations
*.mo
*.pot

# Sphinx documentation
docs/_build/

# PyBuilder
target/


### Django ###
*.log
*.pot
*.pyc
__pycache__/
local_settings.py

.env
db.sqlite3

Step 03: Create a Virtual Environment

Just as the Rails community has rbenv / rvm and so on, Python has venv. So create your environment with:

python3 -m venv env

Step 04: Activate the Virtual Environment

Once it is created, you want to activate it with

source env/bin/activate

Step 05: Installing The Packages

Just as Rails has Gemfile, a Django app has requirements text which can be installed with:

 pip install -r requirements.txt

Note: This is equivalent to bundle install.

Step 06: Creating the Database

To create the database for your Django app, you really are better off using raw sql. Yes there is something called the Django extensions for this but I never could make it work:

create database foo;

Note 1: This is NOT equivalent to rake db:create

Note 2: The file that is the Rails equivalent of config/database.yml is called settings.py and it is located within the “app” directory in your Django application. An app can be thought of either as an application or a context without your overall Django project. By tying database configuration to the app level, Django allows for a “multiple thing” approach to a web app versus the very monolithic Rails application with its (general) single database nature.

Step 07: Running the Migrations

Django like Rails has a migrations engine and one your first steps always needs to be to execute the migrations that specify the database:

python manage.py migrate

Note: This is equivalent to rake db:migrate

The output of running migrations should look something like this:

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, manager, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying manager.0001_initial... OK
  Applying sessions.0001_initial... OK

One very real difference between Django migrations and Rails migrations lies in how migration files are stored. Let’s take the example above of alter_user_first_name_max_length. This is clearly a portion of some filename in your Django app but when you look in your app itself for this, you will not find it. That’s because this particular migration only exists in:

env/lib/python3.10/site-packages/django/contrib/auth/migrations/0012_alter_user_first_name_max_length.py

This is fundamentally different from the Rails approach of db/migrate/timestamp_migration_name.rb. The reason for my pointing this out very, very clearly is that all too often with persistent storage, you need to understand how migrations function to fix something.

Looking at the Database level Result of Running Migrations

If you log into your database after running the migrations above, you’ll see something fairly interesting:

❯ psql source_club
psql (11.18)
Type "help" for help.

source_club=# \dt
                        List of relations
 Schema |                Name                 | Type  |  Owner
--------+-------------------------------------+-------+----------
 public | auth_group                          | table | postgres
 public | auth_group_permissions              | table | postgres
 public | auth_permission                     | table | postgres
 public | auth_user                           | table | postgres
 public | auth_user_groups                    | table | postgres
 public | auth_user_user_permissions          | table | postgres
 public | django_admin_log                    | table | postgres
 public | django_content_type                 | table | postgres
 public | django_migrations                   | table | postgres
 public | django_session                      | table | postgres
 public | manager_customerproduct             | table | postgres
 public | manager_customertosourceclubproduct | table | postgres
 public | manager_pricinganalysisoutput       | table | postgres
 public | manager_sourceclubcustomer          | table | postgres
 public | manager_sourceclubproduct           | table | postgres

Here are a few notable things:

• Tables are generally prefixed.
• There are three prefixes – auth (which presumably comes from an authorization library), django (which is system level) and manager (which is coming from the internal code for this application)
• The migrations above prefixed by manager come from the migration file manager.0001_initial.py. Where as in Rails there is a real focus on one migration, one thing, in Django they seem to package multiple migrations into a single file.
• Table names lack the pluralization found in Rails.
• Rails makes a very big effort to map table names and object classes to human queryable strings i.e. “select * from users” simply feels better than “select * from auth_user” (or at least it does to me).

Step 08: Getting an Enhanced Console like Rails Console

The Rails console is quite powerful and autoloads classes, databases and more. To get the equivalent of the Rails console for Django, we need to return to the Django Extensions mentioned earlier.

Install the Django Extensions with:

pip3 install django-extensions

You then need to add the line:

'django_extensions',

into the list of installed Django Extensions like this:

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'manager.apps.ManagerConfig',
    'django_extensions',
]

You can get then get into the enhanced console with:

python manage.py shell_plus

Step 09: Adding a Management Command

A Django management command is the equivalent of a Rails Rake task. Management commands live below your app directory in the folder management, specifically in the commands folder:

 tree  manager/management
manager/management
├── __init__.py
├── __pycache__
│   ├── __init__.cpython-310.pyc
│   └── __init__.cpython-39.pyc
└── commands
    ├── __init__.py
    ├── __pycache__
    │   └── map_customer_products_to_sourceclub_products.cpython-310.pyc
    ├── _private.py
    ├── clear_data.py
    ├── demo.py
    ├── export_analysis.py
    ├── find_or_add_customer.py
    ├── import_customer_products.py
    ├── import_source_club_mappings.py
    ├── import_sourceclub_products.py
    └── map_customer_products_to_sourceclub_products.py

Here the management commands start with clear_data.py.

An example of a Django management command is below:

# NOTE: Brought in by copy/paste may not all be needed
from tokenize import String
from unicodedata import name
from django.core.management.base import BaseCommand, CommandError
from manager.models import SourceClubCustomer

class Command(BaseCommand):
    
    # python manage.py demo
    def handle(self, *args, **options):
        print("This script lists all the commands for the demo:")
        print("  1. CLEAR ALL DATA")
        print("")
        print("     python manage.py clear_data")
        print("")
        print("  2. IMPORT INVENTORY")
        print("")
        print("     python manage.py import_sourceclub_products  ")
        print("")
        print("  3. IMPORT CUSTOMER DATA")
        print("")
        print("    python manage.py import_customer_products  ")
        print("")
        print("  4. IMPORT INVENTORY MAPPING")
        print("")
        print("     python manage.py import_source_club_mappings   ")
        print("")
        print("  5. EXPORT ANALYSIS")
        print("")
        print("     python manage.py export_analysis.py   ")

What this command does is print out the different tasks the project offers.

Step 10: Adding a New Model with a Migration

The process for adding a new model to an existing Django app is considerably different from Rails. Here’s the process:

1. Go into models.py and add a class to represent the model right down to the schema.

2. Run the command:

   python manage.py makemigrations

This creates a migration file in app_directory/migrations numbered for the migration’s logical sequence number.

1. Run the migration command:

   python manage.py migrate

2. Write any methods needed for the class.

A sample class is shown below:

class FileExport(Timestamped):
     filename = models.CharField(max_length=50, null=False)
     processed_at = models.DateTimeField(auto_now_add=True, null=False)
     export_type = models.CharField(max_length=50, null=False)
     status = models.CharField(max_length=50, null=False)   
     
     def foo():
         return "foo"
       
     
     # export = FileExport.log_export('full_path_and_name_to_export_file', 'customer_data')
     def log_export(filename, export_type):
         new_export = FileExport()
         new_export.filename = filename
         new_export.import_type = export_type
         new_export.status = "exported"
         
         if new_export.save():
             return new_export  

The generated migration file is below:

# Generated by Django 4.1.4 on 2023-01-31 14:55

from django.db import migrations, models


class Migration(migrations.Migration):

    dependencies = [
        ('manager', '0003_alter_customerproduct_source_club_customer_and_more'),
    ]

    operations = [
        migrations.CreateModel(
            name='FileExport',
            fields=[
                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('created_at', models.DateTimeField(auto_now_add=True)),
                ('updated_at', models.DateTimeField(auto_now=True)),
                ('deleted_at', models.DateTimeField(null=True)),
                ('filename', models.CharField(max_length=50)),
                ('processed_at', models.DateTimeField(auto_now_add=True)),
                ('export_type', models.CharField(max_length=50)),
                ('status', models.CharField(max_length=50)),
            ],
            options={
                'abstract': False,
            },
        ),
    ]

Step 11: Class versus Instance Methods in Python

In Ruby class versus instance methods are defined as follows:

class FileExport
  # this is a class method of FileExport called foo
  def self.foo
  end
  
  # this is an instance method of FileExport called bar
  def bar(param1, param2)
  end
end

In Python class versus instance methods are defined as follows:

class FileExport:
  
  # This is a class method
  def foo():
    # code would go here; the pass statement substitutes for a blank body
      pass
      
  # instance methods take self as the first parameter BUT
  #  when the method is called only param1 and param2 
  #  are passed into it 
  def bar(self, param1, param2):
    # code would go here; the pass statement substitutes for a blank body
    pass

Step 12: Adding a Test

Tests for Django live in app_directory/tests. A test for the class bove looks like this:

from django.test import TestCase
from manager.models import FileImport

class FileExportTestCase(TestCase):
    def setUp(self):
        FileExport.objects.create(filename="map.csv", import_type="MappingImport", status="Good")
        
    def test_foo(self):
        """This should return foo"""
        self.assertEqual(FileImport.foo(), 'foo')

    def log_export(self):
        """This should create a new file export object"""
        FileExport.log_export("./data/2022-03-03 12.csv", "customer_data")
        total_rows = FileExport.objects.count()
        # 1 objects in setup plus 1 from this method so 2!!!
        self.assertEqual(total_rows, 2)

Note: The test above has a few characteristics:

• Both tests are for class methods
• The self that is passed in represents not a class versus instance distinction but the test environment as a whole (Rails handles this thru instance variables i.e. variables prefaced with an @ sign)

Tests can be run with:

❯ python manage.py test manager/tests/
Found 17 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
........
Method 0: Check previous mapping table
Method 1: Exact match on manufacturer_item_number
Method 2: Trying name variant: Delta Dental-abcdefghi
Method 2: Trying name variant: Delta Dental abcdefghi
Method 3: Product contains mfg item number: abcdefghi
No matching products found
.........
----------------------------------------------------------------------
Ran 17 tests in 0.106s

OK
Destroying test database for alias 'default'...