Build: Denormalize latest build for projects #12392

stsewd · 2025-08-11T22:50:58Z

Sorting by the last build project is expensive, as the query is done for each project (even if paginated, since the order applies to all projects). For comparison, here are some numbers with data from production

# Ordering by name
145 ms ± 446 μs per loop (mean ± std. dev. of 5 runs, 5 loops each)

# Ordering by max(builds_date)
4.37 s ± 254 ms per loop (mean ± std. dev. of 5 runs, 5 loops each)

The migration is going to take some time to run, I imagine... but should be worth it for a snappier dashboard.

agjohnson

This is a great approach, it should help with query time a lot on the listing view.

It seems like this could be a one way relationship, but I don't think it's necessarily an issue that it's two way either.

Having a ballpark estimate on the data migration in prod would be helpful. If it's far too long, we might weigh other options for updating the project field.

agjohnson · 2025-08-11T23:31:23Z

readthedocs/projects/models.py

+    latest_build = models.OneToOneField(
+        "builds.Build",
+        verbose_name=_("Latest build"),
+        related_name="latest_build_for_project",


Do we want the related relationship at all here? Isn't it always going to be build.project === build.latest_build_for_project?

yeah, but don't think you can omit the related field, django will create one if not given. But I didn't test what would happen if the default was used (it would have collided with Build.project is my guess).

Isn't it always going to be build.project === build.latest_build_for_project?

No. There is going to be only one build where this is true. Most of the builds will have build.latest_build_for_project === None

Also, I don't like too much the related name latest_build_for_project because is not semantically correct since it will return a Project object and not a Build object.

I prefer if we disable the related relationship. Django has an option for this using +:

If you’d prefer Django not to create a backwards relation, set related_name to '+' or end it with '+'

(from https://docs.djangoproject.com/en/5.2/ref/models/fields/#django.db.models.ForeignKey.related_name)

That's why the name ends with _for_project, but I'm fine with disabling the related name

agjohnson · 2025-08-11T23:32:56Z

readthedocs/projects/migrations/0154_set_latest_build.py

+        latest_build = project.builds.order_by("-date").first()
+        if latest_build:
+            project.latest_build = latest_build
+            project.save()


Yeah I'm also guessing this might take a while to migrate. What does this query time look like without updating anything in the database?

that would be a good metric to see how long this would take, yeah, will do that first thing tomorrow.

It might make sense to do the model change first, so we can run the migration without blocking the deploy, and then do the code changes using the model?

I think Eric's idea makes sense. It will require a few extra PRs, and multiple deploys tho.

Like doing the actual filtering after the migration is done? I'm fine doing a Hotfix after the migration is done of it takes too long to run

It took like 10 min in .org without saving, maybe with saving will take 15 min? But fine to just hotfix.

ericholscher

This looks like a great and simple implementation. I was wondering about just doing a latest_build_date, but having a 1to1 to the Build object is likely better so we can actually pull the metadata off it. I really like this pattern, and imagine we could be using it a few other places to really speed up slow queries!

humitos

Looks good to me with some adjustments.

readthedocs/builds/signals_receivers.py

humitos · 2025-08-12T09:27:38Z

readthedocs/projects/migrations/0154_set_latest_build.py

+        latest_build = project.builds.order_by("-date").first()
+        if latest_build:
+            project.latest_build = latest_build
+            project.save()


I think Eric's idea makes sense. It will require a few extra PRs, and multiple deploys tho.

humitos · 2025-08-12T09:31:55Z

readthedocs/projects/models.py

+    latest_build = models.OneToOneField(
+        "builds.Build",
+        verbose_name=_("Latest build"),
+        related_name="latest_build_for_project",


Isn't it always going to be build.project === build.latest_build_for_project?

No. There is going to be only one build where this is true. Most of the builds will have build.latest_build_for_project === None

humitos · 2025-08-12T09:35:06Z

readthedocs/projects/models.py

+    latest_build = models.OneToOneField(
+        "builds.Build",
+        verbose_name=_("Latest build"),
+        related_name="latest_build_for_project",


Also, I don't like too much the related name latest_build_for_project because is not semantically correct since it will return a Project object and not a Build object.

I prefer if we disable the related relationship. Django has an option for this using +:

If you’d prefer Django not to create a backwards relation, set related_name to '+' or end it with '+'

(from https://docs.djangoproject.com/en/5.2/ref/models/fields/#django.db.models.ForeignKey.related_name)

ref #12392

Build: Denormalize latest build for projects

d370d9a

stsewd requested a review from a team as a code owner August 11, 2025 22:50

stsewd requested a review from agjohnson August 11, 2025 22:50

auto-assign bot assigned stsewd Aug 11, 2025

stsewd requested a review from ericholscher August 11, 2025 22:51

agjohnson approved these changes Aug 11, 2025

View reviewed changes

ericholscher approved these changes Aug 12, 2025

View reviewed changes

humitos approved these changes Aug 12, 2025

View reviewed changes

stsewd added 2 commits August 12, 2025 09:11

Updates from review

7573702

Don't use new field yet

752feff

stsewd merged commit 6697275 into main Aug 12, 2025
2 of 4 checks passed

stsewd deleted the denormalize-latest-build branch August 12, 2025 14:19

stsewd mentioned this pull request Aug 12, 2025

Project: use new field for sorting #12398

Merged

stsewd added a commit that referenced this pull request Aug 12, 2025

Project: use new field for sorting (#12398)

aca2139

ref #12392

Uh oh!

Build: Denormalize latest build for projects #12392

Build: Denormalize latest build for projects #12392

Uh oh!

Conversation

stsewd commented Aug 11, 2025

Uh oh!

agjohnson left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stsewd Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ericholscher left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

humitos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

stsewd Aug 12, 2025 •

edited

Loading

ericholscher left a comment •

edited

Loading