Soft Deletes in Marten

Yet more content on new features in Marten leading up to the 1.0 release coming soon. These posts aren’t getting many reads, but they’ll be stuck in Google/Bing for later users, so expect a couple more of these.

As part of the 1.0 release work, Marten gained the capability last week (>0.9.8) to support the concept of “soft deletes.” Instead of just deleting a document completely out of the database, a soft delete would just mark a “deleted” column in the database table to denote that the row is now obsolete. The value of a “soft delete” is simply that you don’t lose any data from the historic record. The downsides are now you’ve got more rows cluttering up your database and you probably need to filter out deleted documents from your queries.

To see this in action, let’s first configure a document type in Marten as soft deleted:

    var store = DocumentStore.For(_ =>
    {
        _.Connection(ConnectionSource.ConnectionString);
        _.Schema.For<User>().SoftDeleted();
    });

By default, Marten does a hard delete of the row, so you’ll need to explicitly opt into soft deletes per document type. There is also a [SoftDeleted] attribute in Marten that you could use to decorate a document type to specify that it should be soft deleted.

When a document type is soft deleted, Marten adds a couple extra fields to the document storage table in the database called “mt_deleted” and “mt_deleted_at” just to track whether and when a document was deleted.

In usage, it’s transparent to the user that you’re doing a soft delete instead of a hard delete:

    // Create a new User document
    var user = new User();
    session.Store(user);
    session.SaveChanges();

    // Mark it deleted
    session.Delete(user);
    session.SaveChanges();

As I said earlier, one of your challenges is to filter out deleted documents in queries. Fortunately, Marten has you covered. As the following acceptance test from Marten shows, deleted documents are automatically filtered out of the Linq query results:

    [Fact]
    public void query_soft_deleted_docs()
    {
        var user1 = new User { UserName = "foo" };
        var user2 = new User { UserName = "bar" };
        var user3 = new User { UserName = "baz" };
        var user4 = new User { UserName = "jack" };

        using (var session = theStore.OpenSession())
        {
            session.Store(user1, user2, user3, user4);
            session.SaveChanges();

            // Deleting 'bar' and 'baz'
            session.DeleteWhere<User>(x => x.UserName.StartsWith("b"));
            session.SaveChanges();

            // no where clause, deleted docs should be filtered out
            session.Query<User>().OrderBy(x => x.UserName).Select(x => x.UserName)
                .ToList().ShouldHaveTheSameElementsAs("foo", "jack");

            var sql = session.Query<User>().OrderBy(x => x.UserName).Select(x => x.UserName).ToCommand().CommandText;
                _output.WriteLine(sql);

            // with a where clause
                session.Query<User>().Where(x => x.UserName != "jack")
                .ToList().Single().UserName.ShouldBe("foo");
        }
    }

Easy peasy. Of course you may want to query against the deleted documents or against all the documents. Marten’s got you covered there too, you can use these two custom Linq extensions in Marten to include deleted documents:

  1. “IDocumentSession.Query<T>().MaybeDeleted()…” will include all documents in the query, regardless of whether or not they have been deleted
  2. “IDocumentSession.Query<T>().IsDeleted()…” will only include documents marked as deleted

Advertisements

6 thoughts on “Soft Deletes in Marten

  1. Damien

    In the MaybeDeleted case, would there be an easy way to distinguish between the deleted and non-deleted documents. I.e. you may want to query all of them, but you may want to apply subtly different behaviour for them. Or if you need to do that, should you do two separate queries – one as normal and one with IsDeleted?

    Reply
    1. jeremydmiller Post author

      Um, yeah, that’s a good question. We don’t have an answer for that one yet, you’d need to do two separate queries. At the worst, you could use a batch query to get them at the same time I guess.

      – Jeremy

      Reply
  2. Marco

    We could extend the DocumentMetaData class with the deleted/deletedAt information (null if the document does not support softdeletes)

    Reply
    1. jeremydmiller Post author

      That’s going to be expensive because that’s a separate connection and round trip for each as it is now. I think we’ll need to gather that information as items are loaded, or change the way metadata fetching works today.

      Reply
  3. mdissel

    We could extend the DocumentMetaData class with the deleted/deletedAt information (and null if the document doesn’t support softdeletes)

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s