Skip to content

Generic cache layer#938

Merged
MichaelMure merged 9 commits intomasterfrom
cache-reorg
Dec 23, 2022
Merged

Generic cache layer#938
MichaelMure merged 9 commits intomasterfrom
cache-reorg

Conversation

@MichaelMure
Copy link
Contributor

@MichaelMure MichaelMure commented Nov 28, 2022

This PR brings a massive refactor/rewrite of the cache layer, making its handling of entities generic. This means that it becomes way easier to add more entities, easier to improve caching techniques, easier to manage memory, and avoid sticky code in duplicated code.

Major changes include:

  • repo: reduced interface for full-text indexing, avoid the direct dependency with Bleve
  • repo: introduce batched indexing, improve performance
  • repo: return recognizable errors on "not found", to be able to distinguish from general errors
  • repo: introduce support for muti-entities push/pull, and thus having to do only a single auth (fix git-bug push makes two git push #218, fix Bad output on 'git bug pull' #867)
  • cache: introduce CachedEntityBase, base type for caching wrapper around an entity
  • cache: introduce SubCache, generic cache handling for entities
  • cache: introduce withSnapshot, generic efficient snapshot handling
  • cache: emit BuildEvent when building the cache instead of printing, leaving full control for UIs to express those things in a meaningful way
  • cache: each entity now has its own namespace in the cache: backend.Bugs().New(...) ... , reducing the functions names and making things cleaner

Generally, this untangle quite a lot of code and is a big step forward for git-bug.

Also, introduce ErrWaitGroup, a variant of sync.WaitGroup/errgroup.Group that takes func() error instead of func(), run all of them to completion (unless the go context expire), and assemble all the errors with the go1.20 multierror support. This IMHO should be included in the standard library.

@MichaelMure MichaelMure force-pushed the cache-reorg branch 2 times, most recently from b8c757d to 6cb4e5a Compare December 19, 2022 15:24
Additionally, remove and concentrate quite a lot of complexity from the cache layer
into a "per app" single site where to configure how indexing is done.
@MichaelMure
Copy link
Contributor Author

MichaelMure commented Dec 21, 2022

Fix #218

@MichaelMure
Copy link
Contributor Author

fix #867

@MichaelMure MichaelMure marked this pull request as ready for review December 21, 2022 22:21
@MichaelMure MichaelMure force-pushed the cache-reorg branch 3 times, most recently from aac377e to a77000a Compare December 23, 2022 00:38
@MichaelMure MichaelMure merged commit 0a5a0ec into master Dec 23, 2022
@MichaelMure MichaelMure deleted the cache-reorg branch December 23, 2022 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bad output on 'git bug pull' git-bug push makes two git push

1 participant