Presigned URL for upload request expires on long scan when uploading results #682
Description
🐞 Bug report
Describe the bug
I'm using the zap-advanced helm chart to run a somewhat deep (authenticated) scan of a particularly large API.
After a long scan of 15 hours, the zap-advanced pod crashes with an error code from the lurker container, see logs attached.
Steps To Reproduce
- Start a large enough scan that will take longer than the presignedurl duration (which I believe is 12 hours but I don't understand golang too much).
- Wait for it to finalize and attempt to upload its results.
- lurker container is unable to upload results to minio as the request has expired
Expected behavior
The lurker to get a 200 status response for uploading the results.
I would also be happy with an option to configure duration of presignedurls
System (please complete the following information):
- secureCodeBox 3.1.0
- Kubernetes Version:
{Major:"1", Minor:"20+", GitVersion:"v1.20.9-gke.1001", GitCommit:"1fe18c314ed577f6047d2712a9d1c8e498e22381", GitTreeState:"clean", BuildDate:"2021-08-23T23:06:28Z", GoVersion:"go1.15.13b5", Compiler:"gc", Platform:"linux/amd64"}
Screenshots / Logs
2021/09/28 03:13:20 Starting lurker
2021/09/28 03:13:20 Waiting for main container 'zap-advanced-scan' to complete
2021/09/28 03:13:20 After scan is completed file '/home/securecodebox/results/zap-results.xml' will be uploaded to 'securecodebox-operator-minio.somenamespace.svc.cluster.local'
2021/09/28 03:13:20 Waiting for maincontainer to exit.
2021/09/28 17:15:09 Main Container exited. Lurker will end as well.
2021/09/28 17:15:09 Uploading result files.
2021/09/28 17:15:09 Uploading /home/securecodebox/results/zap-results.xml
2021/09/28 17:15:09 File has a size of 1190810 bytes
2021/09/28 17:15:09 File upload returned non 2xx status code (403)
2021/09/28 17:15:09 Failed Request:
2021/09/28 17:15:09 HTTP/1.1 403 Forbidden
Connection: close
Content-Length: 408
Accept-Ranges: bytes
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
Date: Tue, 28 Sep 2021 17:15:09 GMT
Server: MinIO/RELEASE.2020-09-26T03-44-56Z
Vary: Origin
X-Amz-Request-Id: 16A90BCBD04372D9
X-Xss-Protection: 1; mode=block
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>scan-35b36f4f-1d02-46f7-a189-ac514abc1383/zap-results.xml</Key><BucketName>securecodebox</BucketName><Resource>/securecodebox/scan-35b36f4f-1d02-46f7-a189-ac514abc1383/zap-results.xml</Resource><RequestId>16A90BCBD04372D9</RequestId><HostId>4549e6c4-f23c-4732-b8a5-135105c6516b</HostId></Error>
2021/09/28 17:15:09 Lurker failed to upload scan result file. File upload returned non 2xx status code (403)
Additional context
- I am running an authenticated scan with custom Graal.js scripts for both authentication and session management
- I have configured a single forced user.
- The ajax spider is disabled and it imports the API definition from an OpenAPI endpoint.
- I am using the default included installation of minio with default values
Metadata
Metadata
Assignees
Labels
Type
Projects
Status