Skip to content

Operator Error messages about concurrent modifications #825

Closed

Description

@malexmave

🐞 Bug report

The operator keeps throwing error messages about "the object has been modified; please apply your changes to the latest version and try again". Seems like there is some concurrent modification going on somewhere? We should track this down.

Full error message with a bit of context around it:

2021-11-18T08:33:44.189+0100	INFO	controllers.execution.Scan	Matching ParseDefinition Found	{"scan_parse": "default/nmap-localhost", "ParseDefinition": "nmap-xml"}
2021-11-18T08:33:49.217+0100	INFO	controllers.execution.Scan	Found ScanCompletionHooks	{"ScanCompletionHooks": 1}
2021-11-18T08:33:49.227+0100	INFO	controllers.execution.Scan	Processing hook	{"hook": {"hookName":"cascading-scans","state":"Pending","priority":0,"type":"ReadOnly"}, "jobType": "read-only-hook"}
2021-11-18T08:33:49.231+0100	INFO	controllers.execution.Scan	Created job for hook	{"hook": {"hookName":"cascading-scans","state":"InProgress","jobName":"cascading-scans-nmap-localhost-fbt7r","priority":0,"type":"ReadOnly"}}
2021-11-18T08:33:49.239+0100	INFO	controllers.execution.Scan	Processing hook	{"hook": {"hookName":"cascading-scans","state":"Pending","priority":0,"type":"ReadOnly"}, "jobType": "read-only-hook"}
2021-11-18T08:33:49.252+0100	ERROR	controllers.execution.Scan	Unable to update Scan status	{"error": "Operation cannot be fulfilled on scans.execution.securecodebox.io \"nmap-localhost\": the object has been modified; please apply your changes to the latest version and try again"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/mmaass/go/pkg/mod/github.com/go-logr/zapr@v0.2.0/zapr.go:132
github.com/secureCodeBox/secureCodeBox/operator/controllers/execution/scans.(*ScanReconciler).executeHooks
	/Users/mmaass/Documents/Code/SecureCodeBox/operator/controllers/execution/scans/hook_reconciler.go:147
github.com/secureCodeBox/secureCodeBox/operator/controllers/execution/scans.(*ScanReconciler).Reconcile
	/Users/mmaass/Documents/Code/SecureCodeBox/operator/controllers/execution/scans/scan_controller.go:109
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:298
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99
2021-11-18T08:33:49.252+0100	ERROR	controller-runtime.manager.controller.scan	Reconciler error	{"reconciler group": "execution.securecodebox.io", "reconciler kind": "Scan", "name": "nmap-localhost", "namespace": "default", "error": "Operation cannot be fulfilled on scans.execution.securecodebox.io \"nmap-localhost\": the object has been modified; please apply your changes to the latest version and try again"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/mmaass/go/pkg/mod/github.com/go-logr/zapr@v0.2.0/zapr.go:132
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:302
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99
2021-11-18T08:33:49.252+0100	INFO	controllers.execution.Scan	Processing hook	{"hook": {"hookName":"cascading-scans","state":"InProgress","jobName":"cascading-scans-nmap-localhost-fbt7r","priority":0,"type":"ReadOnly"}, "jobType": "read-only-hook"}
2021-11-18T08:33:49.263+0100	INFO	controllers.execution.Scan	Processing hook	{"hook": {"hookName":"cascading-scans","state":"InProgress","jobName":"cascading-scans-nmap-localhost-fbt7r","priority":0,"type":"ReadOnly"}, "jobType": "read-only-hook"}

It seems to happen during cascading scan hook execution if I interpret this correctly.

Activity

  1. Weltraumschaf commented on Jul 20, 2023

    @Weltraumschaf
    Member

    This is a somehow "expected" error which is handled internally in the reconciler. We should change it to WARN level and remove the stack trace.

  2. moved this from Backlog to Todo in secureCodeBoxon Feb 7, 2025
  3. moved this from Todo to In Progress in secureCodeBoxon Feb 11, 2025
  4. self-assigned this
    on Feb 11, 2025
  5. moved this from In Progress to Done in secureCodeBoxon Feb 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions