feat: use pagination to get all files in pull request #67
Conversation
WalkthroughThe code refactors the logic for retrieving changed files in a GitHub action to use paginated API calls for both pull request and push events, ensuring all pages of results are processed. Test mocks and test cases are updated to reflect the new pagination logic and the use of the Changes
Sequence Diagram(s)sequenceDiagram
participant GitHub Action
participant Octokit API
GitHub Action->>Octokit API: paginate(pulls.listFiles, {per_page: 100})
Octokit API-->>GitHub Action: [files_page_1, files_page_2, ...]
GitHub Action->>GitHub Action: filterFiles(files from all pages)
GitHub Action-->>GitHub Action: Return filtered file list
sequenceDiagram
participant GitHub Action
participant Octokit API
GitHub Action->>Octokit API: paginate(repos.compareCommits, {per_page: 100})
Octokit API-->>GitHub Action: [files_page_1, files_page_2, ...]
GitHub Action->>GitHub Action: filterFiles(files from all pages)
GitHub Action-->>GitHub Action: Return filtered file list
Assessment against linked issues
Assessment against linked issues: Out-of-scope changesNo out-of-scope changes were found. 📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
⏰ Context from checks skipped due to timeout of 90000ms (1)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Pull Request Overview
Adds pagination to ensure all changed files are retrieved for both pull requests and commit comparisons, replacing the default 30-item limit.
- Replaced single-page fetch calls with
octokit.paginateforcompareCommitsandpulls.listFiles - Introduced a reusable
filterFileshelper to apply file-pattern filtering - Updated tests to stub
paginateand verifyper_page: 100is passed
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/git.ts | Use octokit.paginate with per_page: 100 and filterFiles |
| src/git.test.ts | Stub paginate in tests and assert correct per_page parameter |
Comments suppressed due to low confidence (1)
src/git.test.ts:30
- Consider adding a test case that simulates multiple pages returned by
paginateto ensure the results are correctly aggregated across pages.
paginate: sandbox
Adds pagination to gather all files from pull requests and commit comparisons. Currently, the action only checks the first 30, which is the default value of the `per_page` query parameter.
hill-giant
force-pushed
the
feat/paginate
branch
from
35e577a to
2d4bbe8
Compare
src/git.ts
Outdated
| return octokit.paginate( | ||
| octokit.rest.repos.compareCommits, | ||
| { | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| base, | ||
| head, | ||
| per_page: PAGE_SIZE | ||
| }, | ||
| (response) => filterFiles(response.data.files || []) |
There was a problem hiding this comment.
I realized this isn't necessary for this endpoint. Will update.
Removed pagination for comparing commits because it does nothing in this context.
|
Rate limits might be a concern. I can also add Octokit's throttling plugin. |
hill-giant
changed the title
|
To be clear, the intent of adding this functionality is so that this action does not potentially miss any lint errors. Even if the page size were increased to the maximum value of 100, the action will fail to find the lint error in the 101st file. If there are concerns around the action detecting files that shouldn't be linted, like when users check in their node_modules directory, then I could probably add support for excluding paths. If the team doesn't want to fix this issue with pagination, then the action should at the very least log some kind of warning if it runs against a pull request with a greater number of files than it supports. Currently, it just silently fails. |
1 participant
Adds pagination to gather all files from pull requests and commit comparisons. Currently, the action only checks the first 30, which is the default value of the
per_pagequery parameter.Resolves #66