I'm debugging sitemap generation, and finally have the following snippet:
items=Event.objects.all()
for limit in (1000, 10000, 20000):
events=items[:limit]
print(f'{limit=}')
start=time()
for obj in events.values('id', 'slug', 'date_updated'):
s='/events/{0[slug]}_{0[id]}_{0[date_updated]}/'.format(obj)
print(f' Time spent: {time()-start:.2f}')
start=time()
for obj in events.only('slug', 'date_updated'):
s=f'/events/{obj.slug}_{obj.pk}_{obj.date_updated}/'
print(f' Time spent: {time()-start:.2f}')
And I have the following output:
limit=1000
Time spent: 0.26
Time spent: 9.80
limit=10000
Time spent: 0.66
Time spent: 85.09
limit=20000
Time spent: 1.18
Time spent: 177.20
I see in the doc that QuerySet.values() is recommended for few fields and when you don't need the complete Model behaviour, which is indeed the case here. But is the overhead of creating Model instances (and whatever more there is under the hood) indeed SO huge?
Or it's something about my setup and (mis)configuration. When it's doing models, I see both python and mysql processes consume CPU. And it's inexplicable, because same query when I run it in mysql client takes half a second.
