Operator E2E test sporadically failing due to the timing issue #4953
Description
Operator E2E test sporadically failing due to the timing issue based on the availability of the pods status. We have to expect reliable execution on every PR to avoid the inconsistency.
For Example:
https://github.com/feast-dev/feast/actions/runs/12896520988/job/35959688148?pr=4930
Here is the error we are seeing.
STEP: deploying the Simple Feast Custom Resource to Kubernetes @ 01/21/25 21:59:21.019
running: kubectl apply -f test/testdata/feast_integration_test_crs/v1alpha1_default_featurestore.yaml -n default
[FAILED] in [It] - /home/runner/work/feast/feast/infra/feast-operator/test/e2e/e2e_test.go:161 @ 01/21/25 21:59:21.129
STEP: Uninstalling the feast CRD @ 01/21/25 21:59:21.129
running: kubectl delete deployment feast-operator-controller-manager -n feast-operator-system
• [FAILED] [466.659 seconds]
controller Operator E2E Tests [It] Should be able to deploy and run a default feature store CR successfully
/home/runner/work/feast/feast/infra/feast-operator/test/e2e/e2e_test.go:100
[FAILED] Error occurred while checking FeatureStore simple-feast-setup is having remote registry or not.
Error: kubectl get featurestore command returned empty output
Unexpected error:
<*errors.errorString | 0xc0002a65b0>:
kubectl get featurestore command returned empty output
{
s: "kubectl get featurestore command returned empty output",
}
occurred
In [It] at: /home/runner/work/feast/feast/infra/feast-operator/test/e2e/e2e_test.go:161 @ 01/21/25 21:59:21.129
------------------------------
S [SKIPPED] [0.000 seconds]
controller Operator E2E Tests [It] Should be able to deploy and run a feature store with remote registry CR successfully
/home/runner/work/feast/feast/infra/feast-operator/test/e2e/e2e_test.go:118
[SKIPPED] Spec skipped because an earlier spec in an ordered container failed
In [It] at: /home/runner/work/feast/feast/infra/feast-operator/test/e2e/e2e_test.go:118 @ 01/21/25 21:59:21.18
------------------------------
Summarizing 1 Failure:
[FAIL] controller Operator E2E Tests [It] Should be able to deploy and run a default feature store CR successfully
/home/runner/work/feast/feast/infra/feast-operator/test/e2e/e2e_test.go:161
Ran 1 of 2 Specs in [46](https://github.com/feast-dev/feast/actions/runs/12896520988/job/35959688148?pr=4930#step:7:47)6.659 seconds
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 1 Skipped
--- FAIL: TestE2E (466.66s)
FAIL
FAIL github.com/feast-dev/feast/infra/feast-operator/test/e2e 466.665s
FAIL
make: *** [Makefile:121: test-e2e] Error 1
Error: Process completed with exit code 2.