Soft Deletion in Django
Giving your users the ability to delete objects from your database is a risky proposition — some types of objects are low risk, but others, no matter how much you warn them of the risks of deleting things, will lead to customer requests. You know the kind — “So, I deleted this thing, but I didn’t mean to and now I really need it back…can you recover it?” Of course, databases don’t work that way. When a thing is deleted, it’s deleted.
So, the dream: be able to easily define some objects to be soft-deleted and others to be hard-deleted so that your developers don’t have to remember as they are interacting with objects which are which, and be able to easily recover deleted objects for your users when they need you to. Soft deletion can help with this.
The Model
Most objects in Django inherit from models.Model. If we define a SoftDeletionModel that our objects inherit from instead, we can give it whatever attributes we want, and trust that all models that inherit this way will have the same behavior, and we only have to remember it at the time we define the model, not every time we use it. It might look something like this:
class SoftDeletionModel(models.Model):
deleted_at = models.DateTimeField(blank=True, null=True)
objects = SoftDeletionManager()
all_objects = SoftDeletionManager(alive_only=False)
class Meta:
abstract = True
def delete(self):
self.deleted_at = timezone.now()
self.save()
def hard_delete(self):
super(SoftDeletionModel, self).delete()The pieces:
deleted_at: this means that all models inheriting from theSoftDeletionModelwill have this attribute available to be set. By default it will be null. I recommend a date instead of a boolean so that you can create a background job that hard-deletes any objects that were “deleted” more than 24 hours/7 days/30 days (whatever the right cadence is for you and your users ) ago — data that users choose to delete should actually be deleted.- We’ll look at
objectsandall_objectsin the next section. This is what makes this so powerful abstract = True: This just means we won’t ever define aSoftDeletionModelobject on its own. More detail from the Django docs here.- The
deletemethod means that whenever you call.delete()on any object that inherits from theSoftDeletionModel, it won’t actually be deleted from the database — itsdeleted_atattribute will be set instead hard_deletegives you the option to really truly delete something from the database if you want to, but is named something other than the usual delete methods to ensure that you have to think about what you’re doing before you do it, and actually mean to do it. This usually won’t be exposed to users, but could only be called by developers from the shell.
The Manager
The objects and all_objects attributes defined on the SoftDeletionModel above reference a Django custom manager.
class SoftDeletionManager(models.Manager):
def __init__(self, *args, **kwargs):
self.alive_only = kwargs.pop('alive_only', True)
super(SoftDeletionManager, self).__init__(*args, **kwargs)
def get_queryset(self):
if self.alive_only:
return SoftDeletionQuerySet(self.model).filter(deleted_at=None)
return SoftDeletionQuerySet(self.model)
def hard_delete(self):
return self.get_queryset().hard_delete()The pieces:
- We initialize with
alive_onlyset to True by default, unless we’ve instantiated the manager with that in thekwargs(by callingall_objectsinstead ofobjects) - We define
get_querysetthat, unless we’re callingall_objectsreturns any object that doesn’t have a value fordeleted_at— you don’t want to be working with things your users think they’ve deleted! Otherwise, just return everything, using theSoftDeletionQuerySet, which we’ll look at below hard_delete, once again allows us to really truly delete a thing.
The QuerySet
Before we jump in, here are the docs on Django QuerySets.
class SoftDeletionQuerySet(QuerySet):
def delete(self):
return super(SoftDeletionQuerySet, self).update(deleted_at=timezone.now())
def hard_delete(self):
return super(SoftDeletionQuerySet, self).delete()
def alive(self):
return self.filter(deleted_at=None)
def dead(self):
return self.exclude(deleted_at=None)The pieces:
delete— bulk deleting a QuerySet bypasses an individual object’s delete method, which is why this is needed here as wellaliveanddeadare just helpers — you may find you don’t need them.hard_delete, as above, actually removes the objects from your database, but does this on a QuerySet instead of an individual object
Putting it all together
Lets say we’ve got a VeryImportantSomething that we want users to think they can delete, but that we want to be able to recover for them in case they do it, and come back with regrets.
class VeryImportantSomething(SoftDeletionModel):
# Define the model just as you would any other Django model
...- If I call
VeryImportantSomething.objects.get(pk=123).delete(), I will not remove the object from the database, but instead set thedeleted_atattribute - If I call
VeryImportantSomething.objects.all()I will actually get allVeryImportantSomethingsthat do not have a value set on theirdeleted_atattribute. Likewise, if I callVeryImportantSomething.objects.get(pk=123), I will get anObjectDoesNotExisterror, as if it weren’t in my database at all. - If I were to call
VeryImportantSomething.all_objects.get(pk=123), however, the object would be returned to me (and I could then setdeleted_atto beNone, and thereby “un-delete” it for my user!
FAQ
I’ve gotten some great feedback on this post, so adding the below to address some common questions and concerns. Thanks to everyone who’s reached out about these things and more — I always love to hear how folks are implementing solutions and helping to make them better!
Django Admin
Because Django admin uses model managers, by default, a model deleted via Django admin that is using the SoftDeletionModel will be soft deleted, and will no longer show up in the list or searchable items.
To override both of these defaults, the following admin class could be defined, and any admin class for a model using the SoftDeletionModel could inherit from it.
class SoftDeletionAdmin(admin.ModelAdmin):
def get_queryset(self, request):
qs = self.model.all_objects
# The below is copied from the base implementation in BaseModelAdmin to prevent other changes in behavior
ordering = self.get_ordering(request)
if ordering:
qs = qs.order_by(*ordering)
return qs def delete_model(self, request, obj):
obj.hard_delete()
Related Objects
The SoftDeletionManager handles querying directly for the soft-deleted model, but imagine the following scenario: if Lesson is also a model in our database, with a foreign key to Course, when I query for Lessons, my results include those whose Course has been deleted.
This is because soft-deleting an object does not result in cascade deletion the way true database-level deletes do. This is intentional — if the purpose of soft deletion is to be able to recover data, much, if not all, of that benefit would be lost if we deleted all the foreign key relationships on a soft-delete — the recovery would be pretty meaningless without those related objects!
However, this querying of Lessons with a deleted Course is a valid concern. Unfortunately, I don’t know of a great way to do this globally, since the solution relies on knowing which fields we need to ensure have not been deleted. If every model in your database is a SoftDeletionModel, I can imagine a solution that leverages Django’s introspection, but I haven’t implemented this myself —I’d love to hear about it if you have!
If you just need to implement this for a couple of models, you can do so with a similar mechanism as we used above, using a custom model Manager. It might look like this:
class LessonManager(models.Manager):
def get_queryset(self):
return super(LessonManager, self).get_queryset().filter(course__deleted_at__isnull=True)And then on the Lesson model itself, you’d add:
objects = PublishedCourseManager()
Uniqueness Constraints
One common issue with any soft-deletion solution will be its impacts on uniqueness constraints. For example, if Courses are required to have a unique title, and a user deletes a Course, they would logically expect to be able to create a new one with the same title as the one they just deleted. However, since that one is actually still in the database, any database-level constraint will still fail.
There’s no perfect solution to this — the best I’ve come up with is to further override the model’s delete method to update the field in question to prevent collisions. As with the solution above, this is not a global solution (though you may be able to build one using introspection, as suggested above — I’d love to hear from you if you do this!). For example we could further override the delete method on Course, like so:
def _regenerate_field_for_soft_deletion(obj, field_name):
timestamp = arrow.utcnow().timestamp
max_length = obj.__class__._meta.get_field(field_name).max_length
slug_suffix = '-deleted-{}'.format(str(timestamp))
new_slug = getattr(obj, field_name) if (len(new_slug) + len(slug_suffix)) > max_length:
cutoff = max_length - len(slug_suffix)
new_slug = obj.slug[:cutoff]
return new_slug + slug_suffix
def delete(self):
# Rename the course to prevent collisions
self.title = _regenerate_field_for_soft_deletion(self, 'title')
# SoftDeletionModel.delete() saves the object, so no need to save it here
return super(Course, self).delete()
The _regenerate_slug_for_soft_deletion method handles any charfield and ensures that the new value won’t overrun the max_length of the field, so can be reused on a variety of models.
Updated 6/13/2020
