Django versus Rails 02 - Working in Python with the Db Layer using ChatGPT to Help Learn

Yesterday’s blog post focused on adapting to Django from a Rails environment perspective. Today I’ll focus on getting used to coding in Python and Django.

Goal: Write a Management Command

My goal for today is to write a management command – something very similar to a Rake task – which:

Takes a string input
Checks it against a table
Returns the result if a match as found
Adds it to the table if a match wasn’t found and returns the result

In the Rails world, I refer to this as a find_or_create action so I named my command accordingly:

project_root
  manager
    management
      commands
        find_or_add_customer.py

In that find_or_add_customer.py file, I have the basics for a Python class:

# NOTE: Brought in by copy/paste may not all be needed
from tokenize import String
from unicodedata import name
from django.core.management.base import BaseCommand, CommandError
from manager.models import SourceClubCustomer

# We probably dont have to have pandas for this project. It feels a little heavy
import pandas as pd

class Command(BaseCommand):
    # customer name input (comes in from a param or web form)
    customer_name = "Foo Company"
    
    # look up customer in db
    
    # if found customer then return it
    
    # if not found then create it
    
    # and then return it

That’s my starting point.

Learning to Query SQL Alchemy

My understanding is that default Django equivalent to ActiveRecord is SQL Alchemy. Now to flesh out the logic, I’ll go into the enhanced Django shell and experiment:

./manage.py shell_plus

Let’s start with a basic count of the rows in the table:

SourceClubCustomer.objects.count()

Note: Two differences from ActiveRecord – the use of .objects instead of .all and the () after count.

Even though we know – right now – we don’t have any rows in our table, we still need to write the equivalent of an ActiveRecord where clause. And while I could have looked this up, I instead decided to try ChatGPT with this prompt:

Query a python Django model for anything matching “foo” in a name field

ChatGPT responded with this:

from myapp.models import MyModel

def my_view(request):
    matching_objects = MyModel.objects.filter(name__contains='foo')
    # do something with the matching objects

NOTE: There is also icontains as an operator which presumably is the case insensitive Postgres like that depends on your table type.

The use of contains makes me think this is an SQL like operation where we really want an exact match so I rewrote my prompt:

Query a python Django model for anything equal to “foo” in a name field

and ChatGPT gave me this:

from myapp.models import MyModel

def my_view(request):
    matching_objects = MyModel.objects.filter(name__exact='foo')
    # do something with the matching objects

It also gave me this:

matching_objects = MyModel.objects.filter(name='foo')

as an alternative (which I already had written experimentally in the shell).

Moving from a QuerySet to an Object

The code above returns a QuerySet which can be verified with:

>>> matching_objects.__class__
<class 'django.db.models.query.QuerySet'>

This is similar to an ActiveRecord collection and you can call a .first() method to get the underlying object:

>>> matching_objects = SourceClubCustomer.objects.filter(name__contains=customer_name).first()

At this point you can also check the class and see:

>>> matching_objects.__class__
<class 'NoneType'>

And this brings us, alas, to creation …

Creating a New Object

So I now need to create a new object but I realized that I just have no idea how do this so, again, ChatGPT:

How do I create a new object in Django based on a name field

from myapp.models import MyModel

def my_view(request):
    new_object = MyModel.objects.create(name='foo')
    # do something with the new object

And as an alternative, ChatGPT gives me:

new_object = MyModel(name='foo')
new_object.save()

Putting It All Together

Putting all of the above together, we get:

class Command(BaseCommand):
    # customer name input (comes in from a param or web form); currently just static
    customer_name = "Foo Company";
    
    # look up customer in db
    customer = SourceClubCustomer.objects.filter(name__contains=customer_name)

    # if found customer then return it
    if customer.count() == 1
        print("Found customer so about to return ")
        return customer.first()
    
    # if not found then create it and return it
    new_customer = SourceClubCustomer(name=customer_name)
    
    if new_customer.save()
        print("Into save routine before returning new customer")
        return new_customer

If you are a regular Python user then you are no doubt roaring with laughter going “That will never work” and since I have no issue documenting my (myriad) flaws, here’s the result of executing the code above.

Note: The way you execute a Django management command is the following:

python manage.py find_or_add_customer

The name of the file containing the command is what you give to Django’s manage.py as an argument. This is, naturally, the equivalent of rake task:action with the difference of one action per file.

❯ python manage.py find_or_add_customer
Traceback (most recent call last):
  File "/Users/sjohnson/Sync/coding/django/sourceclub-product-matcher/manage.py", line 22, in <module>
    main()
  File "/Users/sjohnson/Sync/coding/django/sourceclub-product-matcher/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/Users/sjohnson/Sync/coding/django/sourceclub-product-matcher/env/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/Users/sjohnson/Sync/coding/django/sourceclub-product-matcher/env/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/sjohnson/Sync/coding/django/sourceclub-product-matcher/env/lib/python3.10/site-packages/django/core/management/__init__.py", line 279, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/Users/sjohnson/Sync/coding/django/sourceclub-product-matcher/env/lib/python3.10/site-packages/django/core/management/__init__.py", line 48, in load_command_class
    module = import_module("%s.management.commands.%s" % (app_name, name))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 879, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1017, in get_code
  File "<frozen importlib._bootstrap_external>", line 947, in source_to_code
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/sjohnson/Sync/coding/django/sourceclub-product-matcher/manager/management/commands/find_or_add_customer.py", line 18
    if customer.count() == 1
                            ^
SyntaxError: expected ':'

The first and most glaring errors are obviously the missing : which delimit control structures so this rewrites our code as follows:

class Command(BaseCommand):
    # customer name input (comes in from a param or web form); currently just static
    customer_name = "Foo Company";
    
    # look up customer in db
    customer = SourceClubCustomer.objects.filter(name__contains=customer_name)

    # if found customer then return it
    if customer.count() == 1:
        print("Found customer so about to return ")
        return customer.first()
    
    # if not found then create it and return it
    new_customer = SourceClubCustomer(name=customer_name)
    
    if new_customer.save():
        print("Into save routine before returning new customer")
        return new_customer

Executing the code above illustrates the next error:

❯ python manage.py find_or_add_customer
Traceback (most recent call last):
  File "/Users/sjohnson/Sync/coding/django/sourceclub-product-matcher/manage.py", line 22, in <module>
    main()
  ...
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/sjohnson/Sync/coding/django/sourceclub-product-matcher/manager/management/commands/find_or_add_customer.py", line 20
    return customer.first()
    ^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: 'return' outside function

And this then brings me to an existential crisis – by implementing this as a management “function”, I actually have a conceptual error – I can’t return anything because there isn’t a function. This means I need to re-implement the guts of this in the underlying SourceClubCustomer class as follows:

# source club customers or potential customers
class SourceClubCustomer(Timestamped):
    name = models.CharField(max_length=50)

    # customer = SourceClubCustomer.find_or_create('foo')
    def find_or_create(customer_name):
        # look up customer in db; NOTE did not use the find method above deliberately
        customer = SourceClubCustomer.objects.filter(name__contains=customer_name)

        # if found customer then return it
        if customer.count() == 1:
            print("Found customer so about to return")
            return customer.first()
    
        # if not found then create it and return it
        new_customer = SourceClubCustomer(name=customer_name)
    
        if new_customer.save():
            print("Into save routine before returning new customer")
            return new_customer

And now we have a single find_or_create method which handles everything. Additionally we can implement this – since it is just a method on a Python class in a management command or a Django view or whatever. So our management command now becomes:

# NOTE: Brought in by copy/paste may not all be needed
from tokenize import String
from unicodedata import name
from django.core.management.base import BaseCommand, CommandError
from manager.models import SourceClubCustomer

class Command(BaseCommand):
    
    def handle(self, *args, **options):
        # customer name input (comes in from a param or web form); currently just static
        customer_name = "Foo Company";

        customer = SourceClubCustomer.find_or_create(customer_name)

The def handle is a required part of all management commands that my earlier errors hid. Once I fixed those, I quickly got a:

NotImplementedError: subclasses of BaseCommand must provide a handle() method

A little bit of research and I figured it out.

Defining a New Model

In order to add a new model, we need to add a Python class to the file:

manager/models.py

Let’s add a simple class for tracking imports of data with two fields:

filename
processed_at

This gives us the following class:

# these are the imports that are processed
class FileImport(Timestamped):
     filename = models.CharField(max_length=50)
     created_at = models.DateTimeField(auto_now_add=True)

Now this brings us to the workflow for migrations. From a Rails perspective, we tend create the migration either before the class or concurrently so my natural thought was that I can now run the migration script:

❯ python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, manager, sessions
Running migrations:
  No migrations to apply.
  Your models in app(s): 'manager' have changes that are not yet reflected in a migration, and so won't be applied.
  Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.    

So clearly there is another step here:

python manage.py makemigrations
Migrations for 'manager':
  manager/migrations/0002_fileimport.py
    - Create model FileImport

The file manager/migrations/0002_fileimport.py now needs to be reviewed and edited. Here’s what it looks like:

# Generated by Django 4.1.4 on 2023-01-18 15:54

from django.db import migrations, models


class Migration(migrations.Migration):

    dependencies = [
        ('manager', '0001_initial'),
    ]

    operations = [
        migrations.CreateModel(
            name='FileImport',
            fields=[
                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('created_at', models.DateTimeField(auto_now_add=True)),
                ('updated_at', models.DateTimeField(auto_now=True)),
                ('deleted_at', models.DateTimeField(null=True)),
                ('filename', models.CharField(max_length=50)),
                ('processed_at', models.DateTimeField(auto_now_add=True)),
            ],
            options={
                'abstract': False,
            },
        ),
    ]

My input to the migration was simply filename and processed_at and it has been expanded into a much more extensive data structure. My intent was that the processed_at entry indicates when the file was imported. And while it can be argued that you could simply assume created_at indicates this – particularly given that they will initially have the same value, the processed_at field lends itself to the data being reprocessed – a fairly likely scenario.

Summary

Django and Rails are clearly cousins at heart. Knowledge that you have from one easily applies to the other. Rails, to me, still feels far more elegant but I think that’s the nature of Ruby over Python (and, yes, this is a taste difference).