Repository navigation
feat: Implement unified pagination and add template versions support - #1308
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1308 +/- ##
==========================================
+ Coverage 66.81% 66.82% +0.01%
==========================================
Files 284 282 -2
Lines 18633 18493 -140
Branches 235 235
==========================================
- Hits 12449 12358 -91
+ Misses 4903 4871 -32
+ Partials 1281 1264 -17
Continue to review full report at Codecov.
|
kylecarbs
left a comment
There was a problem hiding this comment.
Looks good to me! Just two minor comments 😎
There was a problem hiding this comment.
@BrunoQuaresma just want to make sure you see this and get a chance to discuss it if needed
There was a problem hiding this comment.
Does this mean we send either after_id or limit and offset? I think typescript will complain if we leave anything out of this type, so can we send 0 or empty string for the ones we're not using?
There was a problem hiding this comment.
@presleyp The way this is (currently) implemented means you can set any combination of them (including setting all or none). I don't know if we want to keep it that way though.
@Emyrk do you know if there's a way we can make the generator express the above? (E.g. would setting ? all of these, like readonly after_id?: string, work?)
There was a problem hiding this comment.
Yeah I'm concerned that if the backend expects "just omit the ones you don't want to use" but the frontend enforces "these are all required" then we might have trouble sending well formed requests.
There was a problem hiding this comment.
Yes I agree, that would be super inconvenient! I see pointers and a few other null types are marked as optional by the TypeScript generator, but not others. I could take a look at improving that. Perhaps using the json omitempty option (Go specific) could also be marked optional in TypeScript.
There was a problem hiding this comment.
I don't think both after_id and offset make sense together. And, I'll argue that we should drop offset entirely in favor of after_id.
The problem with offset is that it is not safe to race conditions that add or delete rows. Say we have {a0, a1, a2, b0, b1, b2} as our data. I paginate with limit=3 offset=0 and get {a0, a1, a2}. Now someone else deletes a1. Then I call paginate with limit=3 offset=3. I get {b1, b2}!
You might argue that in some use cases, it doesn't matter if we duplicate or omit data, so why not give both options? The problem is that it's really hard to be sure we've communicated the danger about offset to potential users of this API. They might not read the docstrings. This API might end up being offered to end users outside of Coder. That offset is so tempting because the code is easier to write than after_id, but we're setting users up to shoot themselves in the foot.
There was a problem hiding this comment.
I agree with this. It's also why I suggested separating the options (offset and after_id) into two structs in my PR description. But since adding both was deliberate (#1057 (comment)) I don't think we should remove it in this PR (at least not for the users endpoint). Personally I don't see much of a usability difference between the two and would prefer only having after_id (or another solution entirely).
There was a problem hiding this comment.
I don't understand the use case for using both AfterID and Offset. I would have thought they should be mutually exclusive.
There was a problem hiding this comment.
I agree, I'd be in favor of dropping offset if there's no use-case for it.
There was a problem hiding this comment.
I found some context in an older PR: #1057 (comment)
There was a problem hiding this comment.
@presleyp @BrunoQuaresma Can the frontend just use cursor based pagination? How much harder would it be to support over offset
There was a problem hiding this comment.
I wouldn't argue for offset-based pagination over cursor-based pagination based on how hard it is to implement on the frontend, but...we don't need pagination on the frontend in CE so we are currently ignoring it, sending no limit or offset params, which I believe works because the offset-based pagination defaults are the same as if it wasn't paginated. Could cursor-based pagination be similarly ignored?
There was a problem hiding this comment.
Cursor-based pagination is similarly ignored if not set in the implementation I'm reading.
There was a problem hiding this comment.
I don't think both after_id and offset make sense together. And, I'll argue that we should drop offset entirely in favor of after_id.
The problem with offset is that it is not safe to race conditions that add or delete rows. Say we have {a0, a1, a2, b0, b1, b2} as our data. I paginate with limit=3 offset=0 and get {a0, a1, a2}. Now someone else deletes a1. Then I call paginate with limit=3 offset=3. I get {b1, b2}!
You might argue that in some use cases, it doesn't matter if we duplicate or omit data, so why not give both options? The problem is that it's really hard to be sure we've communicated the danger about offset to potential users of this API. They might not read the docstrings. This API might end up being offered to end users outside of Coder. That offset is so tempting because the code is easier to write than after_id, but we're setting users up to shoot themselves in the foot.
There was a problem hiding this comment.
This locks us into having a UUID for every database table that we want to paginate. Have we fully considered the consequences?
There was a problem hiding this comment.
I had the same thought, but at least all existing tables seem to be using UUID for id.
@coder/backend Does anyone else have input on this? Steven seems to be on PTO so can't check in with him.
There was a problem hiding this comment.
This can easily be changed to a string or something if we ever want to support different ID types in the future so I think it's fine for now
There was a problem hiding this comment.
What happens if the after_id doesn't exist? Either because user modified a URL, or because someone deleted the row?
There was a problem hiding this comment.
No results, which is acceptable in the former scenario. As for the latter, good catch. Not sure if this has been thought of.
04b3b1f to
13fc406
Compare
…gs (#1318) * feat: Add support for json omitempty to apitypings * feat: Express embedded structs via extends in TypeScript * Handle unembedding via json field in apitypings * Add scripts/apitypings/main.go to Makefile
This PR adds support for pagination to the
/api/v2/templates/:id/versionsendpoint whilst breaking out some of the common functionality that can be shared between endpoints that will support pagination. As such, the/api/v2/usersendpoint also received a small refactor.After a discussion with the team, I settled on using embedded structs for the pagination options. We may want to consider using pointers
&codersdk.Pagination{}for being explicit when they're not set, but since the zero values work well it should compose better in more use cases (e.g.req := codersdk.MyRequest{}; req.Limit = 10). Among other API designs considered were functional options and and optional arguments inside optional structs.I decided to omit support for
searchin the template versions endpoint since I was uncertain if this will be needed. If we want to support it, we may want to consider support for defining the column/field to be searched (e.g. should search be applied to name, description or both).Questions:
@Emyrk Currently using both
offsetandafter_idare supported, as per your design. What are your thoughts about splitting these up? We could for instance create two separate types:type PaginateOffset structandtype PaginateID struct, an endpoint can choose to support either or both by embedding (example):Benefits of splitting them up would be to allow an endpoint to use a lesser feature set, but maybe there's a use-case for always supporting both?
Misc thoughts:
codersdkit would make sense to me to rename all.*Requeststructs to.*Paramssince they define the parameters that can be used for the Client method(s). My reasoning is that we're doing a request by invoking the client method, the params only define the arguments.Fixes #53