feat(seaweedfs): add SeaweedFS object storage provider - #14160

Adds SeaweedFS as a CloudStack object-storage provider using S3/IAM APIs and SeaweedFS quota support.

Changes:

• Implements provider lifecycle, bucket, IAM, quota, and usage operations.
• Adds Maven and Spring module registration.
• Adds provider and driver tests.

Suppressed comments (4)

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:430

• On any transient per-bucket S3 failure, this inserts 0. BucketApiServiceImpl treats returned values as authoritative and persists bucket and object-store usage, so a temporary timeout resets usage to zero and under-reports capacity. Propagate the failure for the whole store (or skip the store update) instead of publishing zero.

            } catch (AmazonClientException e) {
                logger.warn("Failed to get usage for bucket {}: {}", bucket.getName(), e.getMessage());
                bucketUsage.put(bucket.getName(), 0L);

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:271

• HttpClient.newHttpClient() and this request have no connect or request timeout. A stalled SeaweedFS endpoint can block the synchronous bucket create/update API worker indefinitely. Use a shared client with configured connect and request timeouts, preferably using the provider's existing timeout configuration.

            java.net.http.HttpClient client = java.net.http.HttpClient.newHttpClient();
            java.net.http.HttpResponse<String> response = client.send(reqBuilder.build(),
                    java.net.http.HttpResponse.BodyHandlers.ofString());

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:261

• The request adds Content-Length to the SDK headers at line 243, then copies every header into java.net.http.HttpRequest.Builder. Java's HttpClient rejects Content-Length as a restricted header, so this loop can throw before the request is sent. Filter transport-managed headers (at least Content-Length, and Host if present) and let HttpClient generate them while signing the same body.

            for (java.util.Map.Entry<String, String> entry : request.getHeaders().entrySet()) {
                if (entry.getKey() != null && entry.getValue() != null) {
                    reqBuilder.header(entry.getKey(), entry.getValue());
                }

plugins/storage/object/seaweedfs/src/test/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImplTest.java:280

• The common setUp already populates storeDetailsMap with the S3 URL and both credentials, so this test never exercises the missing-configuration guard; it passes only because the real HTTP call fails. Clear or override the details map and assert the validation message before any network call.

        // No S3 URL/credentials configured — should throw with a clear message
        assertThrows(CloudRuntimeException.class, () -> driver.setBucketQuota(bucketTO, TEST_STORE_ID, 10));

• Files reviewed: 10/10 changed files
• Comments generated: 6
• Review effort level: Lite

Suppressed comments (10)

client/pom.xml:669

• The existing Add Object Storage UI hard-codes its provider list and has no SeaweedFS entry (ui/src/views/infra/AddObjectStorage.vue:130), so adding this dependency still leaves the provider unavailable through the supported UI. Add the provider to the UI (the default URL fields already match this lifecycle) or explicitly limit the feature to API registration.

        <dependency>
            <groupId>org.apache.cloudstack</groupId>
            <artifactId>cloud-plugin-storage-object-seaweedfs</artifactId>
            <version>${project.version}</version>
        </dependency>

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:520

• initialize always persists a resolved s3Url in the object-store details, so this branch will return that stale value after updateObjectStore changes ObjectStoreVO.url. The generic update path then calls listBuckets() against the old endpoint and can accept an unreachable new URL; subsequent bucket operations continue using the old endpoint. Synchronize the detail URL when the store URL changes or use the current store URL for this provider.

        Map<String, String> storeDetails = _storeDetailsDao.getDetails(storeId);
        String s3Url = storeDetails.get(SeaweedFSObjectStoreUtil.STORE_DETAILS_KEY_S3_URL);
        if (s3Url == null || s3Url.isEmpty()) {
            ObjectStoreVO store = _storeDao.findById(storeId);
            s3Url = store.getUrl();
        }

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:170

• When the stored key is missing, this path creates a replacement and persists it only in account details. Existing BucketVO rows still contain the old pair; createBucketResponse exposes those row values, so previously created buckets continue to hand clients invalid credentials after rotation. Update all buckets for this store/account with the new key pair, as the Cloudian driver does.

        // Persist the credentials in the account details
        details.put(SeaweedFSObjectStoreUtil.KEY_ACCESS_KEY, key.getAccessKeyId());
        details.put(SeaweedFSObjectStoreUtil.KEY_SECRET_KEY, key.getSecretAccessKey());
        _accountDetailsDao.persist(accountId, details);

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:500

• ListObjectsV2 reports only current versions. For a versioned bucket, noncurrent object versions are omitted even though they still consume storage, so usage and BucketVO.size are undercounted after overwrites or deletes. Use a version-aware listing or a SeaweedFS usage endpoint when versioning is enabled.

                    for (com.amazonaws.services.s3.model.S3ObjectSummary summary : result.getObjectSummaries()) {
                        size += summary.getSize();
                    }

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:458

• CloudStack's createBucket API makes quota required and calls setQuota even for a value of 0, so a SeaweedFS deployment without this extension receives a 404 here and the surrounding create flow rolls the bucket back. That contradicts the description that bucket CRUD still works without the extension; either make zero quota a no-op/handle unsupported servers, or make the extension an explicit hard prerequisite for bucket creation.

        SeaweedFSObjectStoreUtil.setBucketQuotaViaS3Extension(s3Url, accessKey, secretKey, bucket.getName(), size, getS3ExtensionHttpClient());

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:164

• Two concurrent createUser calls for the same account can both observe no stored key and no IAM keys, then each create and persist a different key. The last persistence wins, leaving the other key unmanaged; later calls see the persisted key and never enter deleteUnmanagedAccessKeys, so the orphan can accumulate and eventually hit IAM key limits. Serialize per-account provisioning or re-list and reconcile keys after creation.

        CreateAccessKeyResult result = iamClient.createAccessKey(
                new CreateAccessKeyRequest().withUserName(userName));

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:74

• s3:* on Resource: "*" is not a restricted per-account policy. The credentials persisted into each account's bucket record/response can read and write every bucket and, with SeaweedFS's new s3:PutBucketQuota action, change any tenant's quota; the deny only covers bucket creation/deletion. Scope account credentials to that account's buckets/object actions and keep quota/admin mutations on a separate service credential.

        "        \"s3:*\"\n" +
        "      ],\n" +
        "      \"Resource\": \"*\"\n" +

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:328

• Only status codes >= 400 are treated as failures, so a 3xx response is reported as a successful quota update even though HttpClient.newHttpClient() does not follow redirects by default and the mutation was not applied. Accept only the 2xx success range here.

            if (response.statusCode() >= 400) {

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:58

• AccountDetailsDao is account-scoped—its lookup and persistence APIs accept only accountId—while this driver provisions credentials for a specific storeId. If one account uses two SeaweedFS pools, the second provisioning overwrites these fixed keys; a later bucket creation in the first pool then reads and publishes the second pool's credentials. Namespace the keys by store ID or use store-scoped credential storage.

    public static final String KEY_ACCESS_KEY = "swfs_AccessKey";
    public static final String KEY_SECRET_KEY = "swfs_SecretKey";

plugins/storage/object/seaweedfs/src/test/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImplTest.java:345

• setUp() always stubs getDetails(TEST_STORE_ID) with a valid URL and credentials, so this test does not exercise the missing-configuration path. It instead creates a real HttpClient and attempts a network request, allowing the test to pass for the wrong reason. Clear or override storeDetailsMap before the assertion so the exception is caused by missing credentials.

    public void testSetBucketQuotaNoS3ConfigThrows() {
        BucketTO bucketTO = mock(BucketTO.class);
        when(bucketTO.getName()).thenReturn(TEST_BUCKET_NAME);
        // No S3 URL/credentials configured — should throw with a clear message
        assertThrows(CloudRuntimeException.class, () -> driver.setBucketQuota(bucketTO, TEST_STORE_ID, 10));

• Files reviewed: 11/11 changed files
• Comments generated: 4
• Review effort level: Lite

Suppressed comments (8)

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:172

• Persisting the new key before updating BucketVOs creates a one-way inconsistency: if updateAccountBucketCredentials fails for any existing bucket, the key remains stored while some bucket rows still contain old credentials. The next invocation returns at lines 155-157 without retrying the repair, so those buckets can continue exposing invalid credentials. Make persistence and bucket updates transactional, or reconcile all bucket rows on the reuse path.

        _accountDetailsDao.persist(accountId, details);

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:554

• When iamUrl is omitted, initialize() stores it as the initial s3Url, but updateObjectStore only changes ObjectStoreVO.url. After a URL update, this accessor keeps IAM provisioning on the old endpoint while S3 operations use the new one, so account or bucket creation can fail or modify the wrong SeaweedFS instance. Track whether iamUrl was explicit or update this detail with the URL change.

    protected String getIAMUrl(long storeId) {
        Map<String, String> storeDetails = _storeDetailsDao.getDetails(storeId);
        return storeDetails.get(SeaweedFSObjectStoreUtil.STORE_DETAILS_KEY_IAM_URL);

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:207

• iamAccessKeyExists only matches the key ID, so an IAM key whose status is Inactive is treated as usable. If an administrator disables the stored key, createUser returns true and new bucket records continue using credentials that cannot authenticate. Check AccessKeyMetadata.getStatus() and rotate/replace inactive keys before reusing them.

                if (accessKeyId.equals(metadata.getAccessKeyId())) {
                    return true;

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:157

• The reuse check validates only the access-key ID. If the secret-key detail is missing or corrupted, this returns success and never repairs the credential pair, leaving newly created or existing bucket records unusable. Require both stored values before reusing the key; otherwise treat the pair as invalid and rotate it without preserving the unusable key.

        String storedAccessKeyId = details.get(accessKeyDetailKey);
        if (storedAccessKeyId != null && iamAccessKeyExists(iamClient, userName, storedAccessKeyId)) {
            logger.debug("Reusing existing IAM access key {} for user {}", storedAccessKeyId, userName);
            return true;

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:213

• This catch treats every AmazonClientException from ListAccessKeys as evidence that the stored key is absent. A timeout, authentication failure, or temporary IAM outage therefore sends createUser into the replacement path, which can overwrite the persisted credentials, invalidate bucket records, or hit the IAM key limit. Propagate or retry lookup failures and rotate only after a successful listing that omits the stored ID.

        } catch (AmazonClientException e) {
            logger.warn("Failed to list IAM access keys for user {}: {}", userName, e.getMessage());
        }
        return false;

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:99

• These IAM credentials are persisted for the account and exposed as bucket credentials, so this policy is the tenant boundary. Allowing s3:* on Resource: "*" gives every account access to every bucket in the SeaweedFS pool (including PutBucketQuota); denying only create/delete does not isolate accounts. Scope the policy to the account's bucket ARNs and update it as buckets are created or deleted, as the MinIO driver does.

        "      \"Action\": [\n" +
        "        \"s3:*\"\n" +
        "      ],\n" +
        "      \"Resource\": \"*\"\n" +

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:341

• path always starts with /, so URI.resolve(path) replaces any path prefix in s3Url. For an endpoint behind a reverse proxy such as https://host/object-s3, the outgoing quota request is sent to /bucket instead of /object-s3/bucket, so it reaches the wrong route (and can fail signature verification). Build the outgoing URI while preserving the endpoint's existing path.

            java.net.URI fullUri = endpointUri.resolve(path);
            if (! queryString.isEmpty()) {
                fullUri = java.net.URI.create(fullUri.toString() + "?" + queryString);

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:251

• Negative quotas are treated as a disable request here, but updateBucketQuota persists the caller's negative value and computes resource accounting from it. Changing a 5-GiB bucket to -1 therefore frees 6 GiB and stores -1 even though the server quota was cleared. Reject negative values; only zero should disable.

        if (sizeGiB <= 0) {

• Files reviewed: 12/12 changed files
• Comments generated: 2
• Review effort level: Lite

Suppressed comments (6)

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:429

• When a post-create step fails, createBucket throws before BucketApiServiceImpl sets bucketCreated=true, so the service will not perform its own cleanup. If this delete fails and the exception is only logged, the service removes the CloudStack row while the remote bucket remains orphaned, permanently consuming the name and storage with no retry path. Preserve retryable cleanup state or propagate enough state for the caller to retry/record the orphan.

                s3client.deleteBucket(bucketName);
                logger.info("Cleanup of bucket {} succeeded", bucketName);
            } catch (AmazonClientException cleanupEx) {
                logger.error("Cleanup of bucket {} also failed", bucketName, cleanupEx);
            }

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:440

• Policy-revocation failures are swallowed here. If the original PutUserPolicy was accepted but the client lost the response, the account policy can still contain this bucket ARN; the caller then invokes deleteBucket, which removes the row before another policy refresh. If IAM remains unavailable, that second call fails after row removal, leaving stale tenant access (and unreleased resource reservations) with no CloudStack state to retry. Make this cleanup durable/retryable instead of losing the failure.

            } catch (Exception policyEx) {
                logger.warn("Failed to revoke IAM policy for bucket {} after cleanup: {}", bucketName, policyEx.getMessage());
            }

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:732

• ListObjectsV2 returns only the current version of each key. Once CloudStack enables versioning, noncurrent versions still consume SeaweedFS storage but are omitted here, so the scheduled usage scan underreports BucketVO.size/ObjectStoreVO.usedSize and can make capacity accounting inaccurate. Use version-aware listing or a SeaweedFS usage source when versioning is enabled.

        // Fallback: list objects per bucket via S3. This is O(total objects)
        // and does not scale to large deployments; configure metricsUrl for

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:271

• Both createUser and createBucket already hold this store/account lock before calling this helper, so this acquireIamLock call re-enters the same GlobalLock. GlobalLock.lock() emits a WARN for every re-entrant acquisition (framework/db/src/main/java/com/cloud/utils/db/GlobalLock.java:124-132), making normal bucket provisioning look like lock contention and polluting management-server logs. Split the locked and lock-free policy-refresh paths, or let callers retain lock ownership without reacquiring it.

    protected void updateAccountIAMPolicy(AmazonIdentityManagement iamClient, long storeId, long accountId, String excludeBucket) {
        GlobalLock lock = acquireIamLock(storeId, accountId);
        if (lock == null) {
            throw new CloudRuntimeException("Failed to acquire IAM lock for store " + storeId + " account " + accountId);

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:370

• The AWS SDK v1 signer already combines request.getEndpoint().getPath() with request.getResourcePath() when building the canonical URI. Prefixing endpointPath into signedResourcePath therefore signs /object-s3/object-s3/bucket for an endpoint such as https://host/object-s3, while the outgoing URI contains /object-s3/bucket; quota updates through path-prefixed reverse proxies will fail with SignatureDoesNotMatch. Keep the request resource path unprefixed and only prepend endpointPath when constructing fullUri.

            }

ui/src/views/infra/AddObjectStorage.vue:130

• Selecting SeaweedFS renders the generic form, which submits only url, accesskey, and secretkey; it never sends the provider's supported s3Url or iamUrl overrides. The lifecycle then defaults both endpoints to url, so a deployment with a separate IAM endpoint cannot be registered through this UI even though the backend explicitly supports that configuration. Add SeaweedFS-specific endpoint fields/serialization, or otherwise make the UI's supported endpoint model explicit.

      providers: ['MinIO', 'Ceph', 'Cloudian HyperStore', 'SeaweedFS', 'Simulator'],

• Files reviewed: 15/15 changed files
• Comments generated: 1
• Review effort level: Lite

Suppressed comments (3)

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:518

• This removes the CloudStack BucketVO before the IAM policy refresh can succeed. If updateAccountIAMPolicy then hits a transient IAM failure, deleteCheckedBucket never reaches its resource-limit/allocated-size cleanup or its normal row removal, so the bucket ID cannot be retried while the old account policy may still grant this now-reusable ARN. Keep the row until the policy update succeeds, or roll back the row and accounting on failure while coordinating the concurrent policy rebuild.

        for (BucketVO bvo : _bucketDao.listByObjectStoreIdAndAccountId(storeId, accountId)) {
            if (bucketName.equals(bvo.getName())) {
                _bucketDao.remove(bvo.getId());

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:439

• If this policy refresh fails, the exception is only logged and the original failure is rethrown, leaving the deleted bucket ARN in the account's IAM policy. If the name is later reused by another account, the old account's credentials can still access the new bucket; do not silently accept a failed revocation—retry/reconcile it or otherwise block reuse until the grant is removed.

            try {
                AmazonIdentityManagement iamClient = getIAMClient(storeId);
                updateAccountIAMPolicy(iamClient, storeId, accountId, bucketName);
            } catch (Exception policyEx) {
                logger.warn("Failed to revoke IAM policy for bucket {} after cleanup: {}", bucketName, policyEx.getMessage());

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:482

• The exported SeaweedFS Prometheus series is SeaweedFS_s3_bucket_size_bytes, but this parser looks for seaweed_s3_bucket_size_bytes. Prometheus metric names are case-sensitive, so configuring metricsUrl will yield no bucket samples and usage will be reported incorrectly. Use the actual exported name (or make the prefix configurable).

    public static final String METRIC_BUCKET_SIZE_BYTES = "seaweed_s3_bucket_size_bytes";

• Files reviewed: 15/15 changed files
• Comments generated: 2
• Review effort level: Lite

Suppressed comments (4)

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:555

• A managed bucket's sample can be present but use a valid Prometheus numeric format that Long.parseLong cannot consume (for example, a floating-point or scientific-notation value). Swallowing that error leaves the bucket at the preinitialized 0 and makes getAllBucketsUsage return successfully, so the driver skips its S3 fallback and overwrites real usage with zero. Treat this as a scrape failure (or parse the full Prometheus number format) so the existing fallback is used.

                } catch (NumberFormatException ignored) {
                    // Skip unparseable metric values
                }

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:311

• Swallowing every 404/405 when sizeGiB == 0 also makes clearing an existing quota report success when the extension is unavailable. After an update from a positive quota, CloudStack will lower its DB/resource accounting while SeaweedFS retains the old quota/read-only state, leaving the two systems inconsistent. Only tolerate this for a known initial no-quota create, or propagate the error for quota clears.

            if (sizeGiB == 0 && e.getMessage() != null
                    && (e.getMessage().contains("status 404") || e.getMessage().contains("status 405"))
                    && !e.getMessage().contains("NoSuchBucket")) {
                org.apache.logging.log4j.LogManager.getLogger(SeaweedFSObjectStoreUtil.class)
                        .warn("SeaweedFS quota extension not available for bucket {}; skipping quota disable (quota is already off by default)", bucketName);
                return;

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:517

• The documentation above allows metricsUrl to point at a Prometheus server, but this always appends /metrics; a Prometheus server's /metrics exposes its own metrics rather than the scraped SeaweedFS bucket series. That response is still HTTP 200, so the driver returns zero for every bucket instead of falling back. Restrict this setting to a SeaweedFS exporter URL or query Prometheus's HTTP API when a server URL is supplied.

            java.net.URI uri = java.net.URI.create(metricsUrl + "/metrics");

ui/src/views/infra/AddObjectStorage.vue:92

• These SeaweedFS-specific labels and placeholders are hard-coded in English, while the surrounding object-storage form uses $t(...) and locale keys. This leaves the new provider configuration untranslated for non-English users; add locale entries and bind the labels/placeholders through the localization mechanism.

          <a-form-item name="s3Url" ref="s3Url" label="S3 Endpoint URL (optional)">

• Files reviewed: 15/15 changed files
• Comments generated: 1
• Review effort level: Lite

Suppressed comments (5)

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:557

• The lock is released here before BucketApiServiceImpl removes the BucketVO. A concurrent createUser/createBucket can acquire the lock after this block, observe the still-present row, and publish a policy that re-adds this bucket; the row is then removed, leaving a stale ARN grant. Since bucket names are reusable, the old account could access a later account's bucket with the same name. Keep row removal inside the same lock scope or otherwise serialize deletion with policy refresh.

        } finally {
            iamLock.unlock();
            iamLock.releaseRef();

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:439

• Although this returns the updated BucketVO, BucketApiServiceImpl.createBucket discards that return value and ultimately returns its original bucket to CreateBucketCmd. The response generator therefore sees null or stale accessKey, secretKey, and bucketURL immediately after a successful create, even though these fields are populated in the database here and are part of BucketResponse. Propagate the returned BucketVO into the API response/state update.

            // Return the updated BucketVO (not the stale input bucket) so
            // BucketApiServiceImpl.createBucket does not overwrite the
            // persisted credentials with the stale values.
            return bucketVO;

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/driver/SeaweedFSObjectStoreDriverImpl.java:774

• SeaweedFS refreshes bucket_size_bytes only on the S3 instance holding its distributed s3.leader lock. If metricsUrl points to a load-balanced service, this request can hit a non-leader with stale or empty gauges; the parser then accepts those values and bypasses the S3 fallback, so CloudStack can under-report usage. Use a stable leader endpoint or an aggregated/freshness-checked metrics source.

            try {
                return SeaweedFSObjectStoreUtil.parseBucketUsageFromMetrics(
                        metricsUrl, bucketNames, getS3ExtensionHttpClient());

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:563

• This check also matches Prometheus # HELP and # TYPE metadata lines, so a response with no bucket samples is accepted. The result was prefilled with zeroes and is then returned without the S3 fallback, causing every managed bucket to be reported as 0 bytes during an exporter response that only contains the metric declaration. Require an actual sample line (or otherwise treat a sample-less family as a scrape failure).

            if (!body.contains(METRIC_BUCKET_SIZE_BYTES)) {

plugins/storage/object/seaweedfs/src/main/java/org/apache/cloudstack/storage/datastore/util/SeaweedFSObjectStoreUtil.java:547

• Appending /metrics directly makes a configured URL ending in / request //metrics. That can return a redirect or 404; HttpClient does not follow redirects here, so the driver silently falls back to the O(total objects) S3 scan on every usage poll and loses the scalability benefit of metricsUrl. Normalize trailing slashes before appending the endpoint path.

            java.net.URI uri = java.net.URI.create(metricsUrl + "/metrics");

• Files reviewed: 16/16 changed files
• Comments generated: 0 new
• Review effort level: Lite