Skip to content

Commit 987fcbd

Browse files
abhinandanprateekyadvr
abhinandanprateek
authored and
yadvr
committed
CLOUDSTACK-8592: Implement Quota service
Quota service while allowing for scalability will make sure that the cloud is not exploited by attacks, careless use and program errors. To address this problem, we propose to employ a quota-enforcement service that allows resource usage within certain bounds as defined by policies and available quotas for various entities. Quota service extends the functionality of usage server to provide a measurement for the resources used by the accounts and domains using a common unit referred to as cloud currency in this document. It can be configured to ensure that your usage won’t exceed the budget allocated to accounts/domain in cloud currency. It will let user know how much of the cloud resources he is using. It will help the cloud admins, if they want, to ensure that a user does not go beyond his allocated quota. Per usage cycle if a account is found to be exceeding its quota then it is locked. Locking an account means that it will not be able to initiat e a new resource allocation request, whether it is more storage or an additional ip. Needless to say quota service as well as any action on the account is configurable. Changes from Github code review: - Added marvin test for quota plugin API - removed unused commented code - debug messages in debug enabled check - checks for nulls, fixed access to member variables and feature - changes based on PR comments - unit tests for UsageTypes - unit tests for all Cmd classes - unit tests for all service and manager impls - try-catch-finally or try-with-resource in dao impls for failsafe db switching - remove dead code - add missing quota calculation case (regression fixed) - replace tabs with spaces in pom.xmls - quota: though default value for quota_calculated is 0, the usage server makes it null while entering usage entries. Flipping the condition so as to acocunt for that. - quotatypes: fix NPE in quota type - quota framework test fixes - made statement period configurable - changed default email templates to reflect the fact that exhausted quota may not result in a locked account - added quotaUpdateCmd that refreshes quota balances and sends alerts and statements - report quotaSummary command returns quota balance, quota usage and state for all account - made UI framework changes to allow for text area input in edit views - process usage entries that have greater than 0 usage - orocess quota entries only if tariff is non zero - if there are credit entries but no balance entry create a dummy balance entry - remove any credit entries that are before the last balance entry when displaying balance statement - on a rerun the last balance is now getting added FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Quota+Service+-+FS PR: apache#768 Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
1 parent f30fbe9 commit 987fcbd

110 files changed

Lines changed: 10350 additions & 81 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎api/src/org/apache/cloudstack/api/ApiConstants.java‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -273,6 +273,7 @@ public class ApiConstants {
273273
public static final String VIRTUAL_MACHINE_ID_IP = "vmidipmap";
274274
public static final String VIRTUAL_MACHINE_COUNT = "virtualmachinecount";
275275
public static final String USAGE_ID = "usageid";
276+
public static final String USAGE_TYPE = "usagetype";
276277

277278
public static final String VLAN = "vlan";
278279
public static final String VLAN_RANGE = "vlanrange";

‎api/src/org/apache/cloudstack/api/BaseCmd.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ public static enum HTTPMethod {
9595
GET, POST, PUT, DELETE
9696
}
9797
public static enum CommandType {
98-
BOOLEAN, DATE, FLOAT, INTEGER, SHORT, LIST, LONG, OBJECT, MAP, STRING, TZDATE, UUID
98+
BOOLEAN, DATE, FLOAT, DOUBLE, INTEGER, SHORT, LIST, LONG, OBJECT, MAP, STRING, TZDATE, UUID
9999
}
100100

101101
private Object _responseObject;

‎api/src/org/apache/cloudstack/api/command/admin/usage/GetUsageRecordsCmd.java‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,30 @@ public Long getProjectId() {
111111
public String getUsageId() {
112112
return usageId;
113113
}
114+
public void setAccountName(String accountName) {
115+
this.accountName = accountName;
116+
}
117+
118+
public void setDomainId(Long domainId) {
119+
this.domainId = domainId;
120+
}
121+
122+
public void setEndDate(Date endDate) {
123+
this.endDate = endDate == null ? null : new Date(endDate.getTime());
124+
}
125+
126+
public void setStartDate(Date startDate) {
127+
this.startDate = startDate == null ? null : new Date(startDate.getTime());
128+
}
129+
130+
public void setAccountId(Long accountId) {
131+
this.accountId = accountId;
132+
}
133+
134+
public void setUsageId(String usageId) {
135+
this.usageId = usageId;
136+
}
137+
114138

115139
/////////////////////////////////////////////////////
116140
/////////////// API Implementation///////////////////

‎api/src/org/apache/cloudstack/usage/UsageTypes.java‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
import org.apache.cloudstack.api.response.UsageTypeResponse;
2323

2424
public class UsageTypes {
25+
/* Any changes here should also reflect in cloud_usage.quota_mapping table */
2526
public static final int RUNNING_VM = 1;
2627
public static final int ALLOCATED_VM = 2; // used for tracking how long storage has been allocated for a VM
2728
public static final int IP_ADDRESS = 3;

‎api/test/org/apache/cloudstack/api/command/test/UsageCmdTest.java‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
package org.apache.cloudstack.api.command.test;
1818

1919
import java.util.ArrayList;
20+
import java.util.Date;
2021
import java.util.List;
2122

2223
import junit.framework.TestCase;
@@ -70,4 +71,22 @@ public void testExecuteEmptyResult() {
7071

7172
}
7273

74+
@Test
75+
public void testCrud() {
76+
getUsageRecordsCmd.setDomainId(1L);
77+
assertTrue(getUsageRecordsCmd.getDomainId().equals(1L));
78+
79+
getUsageRecordsCmd.setAccountName("someAccount");
80+
assertTrue(getUsageRecordsCmd.getAccountName().equals("someAccount"));
81+
82+
Date d = new Date();
83+
getUsageRecordsCmd.setStartDate(d);
84+
getUsageRecordsCmd.setEndDate(d);
85+
assertTrue(getUsageRecordsCmd.getStartDate().equals(d));
86+
assertTrue(getUsageRecordsCmd.getEndDate().equals(d));
87+
88+
getUsageRecordsCmd.setUsageId("someId");
89+
assertTrue(getUsageRecordsCmd.getUsageId().equals("someId"));
90+
}
91+
7392
}

‎client/WEB-INF/classes/resources/messages.properties‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1423,6 +1423,47 @@ label.show.advanced.settings=Show advanced settings
14231423
label.delete.OpenDaylight.device=Delete OpenDaylight Controller
14241424
label.polling.interval.sec=Polling Interval (in sec)
14251425
label.quiet.time.sec=Quiet Time (in sec)
1426+
label.usage.type=Usage Type
1427+
label.usage.unit=Unit
1428+
label.quota.value=Quota Value
1429+
label.quota.description=Quota Description
1430+
label.quota.configuration=Quota Configuration
1431+
label.quota.configure=Configure Quota
1432+
label.quota.remove=Remove Quota
1433+
label.quota.totalusage=Total Usage
1434+
label.quota.balance=Balance
1435+
label.quota.minbalance=Min Balance
1436+
label.quota.enforcequota=Enforce Quota
1437+
label.quota.summary=Summary
1438+
label.quota.fullsummary=All Accounts
1439+
label.quota.tariff=Tariff
1440+
label.quota.state=State
1441+
label.quota.startdate=Start Date
1442+
label.quota.enddate=End Date
1443+
label.quota.total=Total
1444+
label.quota.startquota=Start Quota
1445+
label.quota.endquota=End Quota
1446+
label.quota.type.name=Usage Type
1447+
label.quota.type.unit=Usage Unit
1448+
label.quota.usage=Quota Consumption
1449+
label.quota.add.credits=Add Credits
1450+
label.quota.email.template=Email Template
1451+
label.quota.statement=Statement
1452+
label.quota.statement.balance=Quota Balance
1453+
label.quota.statement.quota=Quota Usage
1454+
label.quota.statement.tariff=Quota Tariff
1455+
label.quota.tariff.value=Tariff Value
1456+
label.quota.tariff.edit=Edit Tariff
1457+
label.quota.tariff.effectivedate=Effective Date
1458+
label.quota.date=Date
1459+
label.quota.dates=Update Dates
1460+
label.quota.credit=Credit
1461+
label.quota.credits=Credits
1462+
label.quota.value=Quota Value
1463+
label.quota.statement.bydates=Statement
1464+
label.quota.email.subject=Subject
1465+
label.quota.email.body=Body
1466+
label.quota.email.lastupdated=Last Update
14261467
label.destroy.vm.graceperiod=Destroy VM Grace Period
14271468
label.SNMP.community=SNMP Community
14281469
label.SNMP.port=SNMP Port

‎client/pom.xml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,11 @@
256256
<artifactId>cloud-framework-ipc</artifactId>
257257
<version>${project.version}</version>
258258
</dependency>
259+
<dependency>
260+
<groupId>org.apache.cloudstack</groupId>
261+
<artifactId>cloud-framework-quota</artifactId>
262+
<version>${project.version}</version>
263+
</dependency>
259264
<dependency>
260265
<groupId>org.apache.cloudstack</groupId>
261266
<artifactId>cloud-framework-rest</artifactId>
@@ -366,6 +371,11 @@
366371
<artifactId>cloud-plugin-network-globodns</artifactId>
367372
<version>${project.version}</version>
368373
</dependency>
374+
<dependency>
375+
<groupId>org.apache.cloudstack</groupId>
376+
<artifactId>cloud-plugin-database-quota</artifactId>
377+
<version>${project.version}</version>
378+
</dependency>
369379
</dependencies>
370380
<build>
371381
<plugins>

‎client/tomcatconf/commands.properties.in‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -787,3 +787,14 @@ addGloboDnsHost=1
787787
### volume/template post upload
788788
getUploadParamsForVolume=15
789789
getUploadParamsForTemplate=15
790+
791+
### Quota Service
792+
quotaStatement=15
793+
quotaBalance=15
794+
quotaSummary=15
795+
quotaUpdate=1
796+
quotaTariffList=15
797+
quotaTariffUpdate=1
798+
quotaCredits=1
799+
quotaEmailTemplateList=1
800+
quotaEmailTemplateUpdate=1

‎engine/schema/src/com/cloud/upgrade/dao/Upgrade452to460.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -340,4 +340,4 @@ private void updateSystemVmTemplates(final Connection conn) {
340340
}
341341
s_logger.debug("Updating System Vm Template IDs Complete");
342342
}
343-
}
343+
}

‎engine/schema/src/com/cloud/upgrade/dao/Upgrade461to470.java‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@
2323

2424
import java.io.File;
2525
import java.sql.Connection;
26+
import java.sql.PreparedStatement;
27+
import java.sql.SQLException;
2628

2729
public class Upgrade461to470 implements DbUpgrade {
2830
final static Logger s_logger = Logger.getLogger(Upgrade461to470.class);
@@ -51,8 +53,23 @@ public File[] getPrepareScripts() {
5153
return new File[] {new File(script)};
5254
}
5355

56+
public void alterAddColumnToCloudUsage(final Connection conn) {
57+
final String alterTableSql = "ALTER TABLE `cloud_usage`.`cloud_usage` ADD COLUMN `quota_calculated` tinyint(1) DEFAULT 0 NOT NULL COMMENT 'quota calculation status'";
58+
try (PreparedStatement pstmt = conn.prepareStatement(alterTableSql)) {
59+
pstmt.executeUpdate();
60+
s_logger.info("Altered cloud_usage.cloud_usage table and added column quota_calculated");
61+
} catch (SQLException e) {
62+
if (e.getMessage().contains("quota_calculated")) {
63+
s_logger.warn("cloud_usage.cloud_usage table already has a column called quota_calculated");
64+
} else {
65+
throw new CloudRuntimeException("Unable to create column quota_calculated in table cloud_usage.cloud_usage", e);
66+
}
67+
}
68+
}
69+
5470
@Override
5571
public void performDataMigration(Connection conn) {
72+
alterAddColumnToCloudUsage(conn);
5673
}
5774

5875
@Override

0 commit comments

Comments
 (0)