Blog postings

I have become a big fan of Django data migrations · by mark | 4 Jul 2022, 8:45 p.m.

One of the more irritating things about Django is keeping track of what you have done to the database. Now for things like these blog posts you don't normally care as you create the object, insert it into the database, and render it on demand. But for other things where your database model or some parameter style data is updated you want to be able to control programatically what is going on. 

If you read many resources on django they are long on a feature called fixtures. This attention is a bit of a red herring. You use fixtures to populate a DB right at the beginning of a project, or to take a dump of live data for test purposes due to how the Django test framework works. That's it. If you are migrating or updating data that isn't user generated content you want to use data migrations. These have several benefits:

  • Can be reversed: you can back out changes if you turn out to have messed up
  • Can be reviewed and debugged: changes can be made iteratively until they're right
  • Can be versioned: stick them in git and you can see what has gone on and why, rather than trying to infer what has gone on from database state (you will never do this). 
  • Can basically do fixtures: you can point the data migration at some other source and load it into the DB 
  • Zero downtime: sensible deployment processes will replicate exactly your test and prod DBs (e.g. on Heroku: I do git push heroku master, give it a few minutes, and it is live).

The major downside is that they are a bit complicated to set up but once you get the hang of it they're not bad. 

So what do you need to do? 

Make an empty migration

Easy. At the command prompt utter the incantation

python manage.py makemigrations --empty yourappname

where yourappname is, well, your app name. That's it.

Define functions

In the migration file Django created for you, there is a new Migration instance with an operations member defined. Make this look like 

operations = [migrations.RunPython(forward_func, reverse_func)]

Create a forward function

This is where you create the new objects in your model. Function signature should be 

def forward_func(apps, schema_editor)

To get a model, use my_model = apps.get_model("yourappname","yourmodelname"). Don't import the model and call it directly. It won't work as the migration executes in such a way that imported model objects cannot be used. I can't tell you how to create the new data objects; this depends on your use case. Try my_model.objects.create()

Create a reverse function

This is the reverse of the forward function. Identify the objects created in the forward_func(), and delete() them. 

Actually do the migration

At the command line, do python manage.py migrate . This should work. It will give you a numbered success message if it works. If it does not, fix your migration. Make sure to test both forward and backwards! 

The migrate command takes you forward. To go back, first remember what the numbered success message is (e.g. 0011). Then do python manage.py migrate yourappname 0010. If it doesn't work, fix your reverse function. 

The great thing about this is the ability to move between DB versions in a controlled way. This is just not possible with fixtures as you can't really tell, without agonisingly reconciling the fixture file with the DB row-by-row, whether the fixture is loaded. Data migrations give you the control you need to do updates safely. 

 

Had to say goodbye to my cat after 15 years today · by mark | 25 Jun 2022, 12:39 p.m.

We got Misu fifteen years ago as a kitten, when we were living in Sheffield. Before we were married even. He's been with us ever since and has always been a playful affectionate moggie who wanted nothing more than a piece of tuna and a lap to sit on. 

Recently he has not been himself and his mobility has gone too. Took him to the vets for some tests yesterday and there was a laundry list of issues - mainly end stage renal failure which had caused diabetes as well as osteoarthritis. Treatment would not have been kind; the drugs that would have been used for his joints could not be started until other things were under control and that would have taken several months, by which point it would be close to the end anyway with his kidney disease. Only a few months back he was still full of beans but now he had become an old immobile cat who found lying down painful. There was only one last thing we could do for him. I've never had to have a pet euthanised before, I was surprised (but relieved) at how quick it was. 

We took this picture of Misu and our little Atlas during the height of the 2020 lockdowns. Atlas was knocked down by a car last year and the suddenness of losing him was very upsetting. With Misu we knew it was coming and it was just his time to go. Two wonderful cats who are very missed.

Atlas and Misu, taken during early 2020 lockdown

 

I have joined the gym again and already regret it · by mark | 21 Jun 2022, 7:59 p.m.

When the pandemic hit and we were instructed to remain indoors one of the first things I did, after making myself an expert on various things like Coronaviridae and R0 on Wikipedia, was do some financial planning. Basically I cancelled all the direct debits I could with few survivors such as my mortgage and Netflix. The gym was excised after giving them contractual notice. 

Well. Years pass; I do not get younger; my waistline is imperilled; time to pick up heavy objects and put them down again after first doing some brisk cardiovascular exertion as a warm up.

I do not like exercise. I like having done exercise. Maybe this time I will increase muscle mass. 

It's good for me, I think. 

Most annoying thing is the jacuzzi they have. To save on the leccy they've dialed it down a degree Centrigrade or two. It's tepid. They should make it warmer. 

 

Got the boat from Mallaig to Skye · by mark | 4 Jun 2022, 8:39 a.m. (updated 23 Aug 2022, 6:33 a.m.)

You can drive to Skye. There is a bridge. The alternative is the ferry from Mallaig run by CalMac. It's never been at a decent time before but this time it worked out. 
Signal flags MV Loch Fyne

There were also dolphins! Flying through the ship's wake. Skye itself was glorious too. No dolphin pics but got this looking across to the mainland

Somewhere on Skye

Addendum: if you are planning on doing this route you want to book in advance. One sailing was fully booked and the other was almost so. If we had just turned up and tried to pay at the kiosk we would have had a long wait at least once. Service is once every 60-90 minutes depending on tides and fuelling schedules. 

Sailing takes around 45 minutes. As you can see from the pictures we had a nice calm day and there was no seasickness from anyone on board. In winds it could be far worse. Fare was around £45 return for four people and a car which wasn't too bad - driving that route would take longer and use quite a bit of fuel anyway. Plus you don't see dolphins from the road! 

 

I went up a chairlift at Glencoe · by mark | 2 Jun 2022, 12:47 p.m.

There is a very big mountain you can get a chairlift up. I did it. Best £15 I've spent in a while

Up the chairlift

I'd do that again