@shawkins I added this to 3.3 milestone for now.

To summarize it to my understanding there are two cases when running multiple instances of operators is happening and/or desirable, and leader election makes sure only one of them is actively reconciling, thus other than leader instances don't execute reconcilers:

minimize downtime in following cases:
- There is an updated version of operator being released, and deployment first creates the new version of the operator pod then stops the old one. (For now to handle this scenario use recreate deployment strategy)
- Minimise downtime of an operator crash, so there are multiple instances running all the time. However there are multiple strategies are this. So if an operator not the leader, should it populate the caches? and just not reconcile the events.
make sure fail-over operators are be provisioned on the cluster, so multiple instance are provisioned, therefore in case the active operator's pod is crashed, it cannot happen that a new instance is not provision because of cluster resources are not available.

In summary there is one design question:

Should the non-leader operator instances activate event sources, and just don't trigger reconciliation until elected as leader? Or just basically start the operator when it is elected as leader. Both has pros and cons, when event sources are activated will consume resources (polling in some cases possibly, cache resources in memory), on the other hand will minimize downtime, in case of the syncing the caches on startup takes long time.

Leader election #411

Description

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions