Skip to content

Commit 545e89c

Browse files
DaanHooglandDaan HooglandsureshanapartiRodrigoDLopezGutoVeronezi
authored
Mshost stats (apache#5588)
* ms stats thread added * initial data collection for management server * empty list management server metrics command * bean copy into MS metrics object * ms status VO * further API and DB plumbing * minimal metrics response in API * remove commented, refactor data collection plumbing * javadocs * surpress stacktrace on expected error * update status experiment * ms status publish framework added * review comment addressed * static data to DB and API, /proc/ reading * addressing review comments * ui for ms details * small ui adjustment * beanCopy * agentcount response and system parameter * labels * package-lock * add version strings to regular list API * add shutdown time to DB * add last start and last stop to regular list response * distro info in regular response/session count added * metrics as details * add heap used and remove details map * thread-statusses * move db upgrade to 4.17 * sysmem * procmem * ui demo comments applied * javadoc * get conf and log file locations * loginfo * cpuLoadStats * no.remote * extra spaces removed * clusterlistener * add unit to kb value * revert accidental rename * silly fqcn removed * get mem info from bean is possible * refactor long sequence for readability * registerListener * listUsageMetrics and isDbLocal * rats * local usage and db or not * minimal listDbMetrics * db vars and stats * cleanup and #queries queried * db stats calculation * rat * remove list response wrapper from sinlge details-lists responses * rudimentary metrics view * metrics table cleanup * table makeup, collection dates * move component to appropriate location * capitalisation removed * rebase error resolved * rename deamon to daemon * small style comments applied * another merge issue * naming comments and boot time * stop/start prefixed with server * layout-fix * listMSMetrics test and test refactor * usage metrics test * db metrics test * extra validations * Update ui/public/locales/en.json Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com> * descriptions of loadaverages and replica's * collection time on top * cpu load on metrics overview * DbStatsCollection * some parameter description texts * labels adjusted * new output 'kernelversion' and log info cleanup * labels * Update api/src/main/java/com/cloud/server/ManagementServerHostStats.java Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/response/DbMetricsResponse.java Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com> * Update framework/cluster/src/main/java/com/cloud/cluster/dao/ManagementServerHostDao.java Co-authored-by: Rodrigo D. Lopez <19981369+RodrigoDLopez@users.noreply.github.com> * Update framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java Co-authored-by: Rodrigo D. Lopez <19981369+RodrigoDLopez@users.noreply.github.com> * Update api/src/main/java/org/apache/cloudstack/api/response/ManagementServerResponse.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update api/src/main/java/org/apache/cloudstack/api/response/ManagementServerResponse.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update engine/schema/src/main/java/com/cloud/host/dao/HostDao.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update server/src/main/java/com/cloud/server/StatsCollector.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update framework/cluster/src/main/java/com/cloud/cluster/dao/ManagementServerHostDao.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update server/src/main/java/com/cloud/server/StatsCollector.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update server/src/main/java/com/cloud/server/StatsCollector.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update server/src/main/java/com/cloud/server/StatsCollector.java Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java * Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java * some (more) refactorring suggestions applied * human readable memory sizes * rat * actual collection time instead of query time, improved descriptions * merge errors fixed * optional metric values * javadoc and logging * names of jmx vars have changed * vue3-compatibility * new output parameter type * lower retention default * vue3 fixes * polish comments * polish comments 2, the reckoning * note on usage servers * merge conflict errors * pollish * conditional assertion to deal with simulator restart Co-authored-by: Daan Hoogland <dahn@onecht.net> Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com> Co-authored-by: Rodrigo D. Lopez <19981369+RodrigoDLopez@users.noreply.github.com> Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com> Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
1 parent 9a53bad commit 545e89c

56 files changed

Lines changed: 36408 additions & 615 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎api/src/main/java/com/cloud/server/ManagementServerHostStats.java‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
//
2+
// Licensed to the Apache Software Foundation (ASF) under one
3+
// or more contributor license agreements. See the NOTICE file
4+
// distributed with this work for additional information
5+
// regarding copyright ownership. The ASF licenses this file
6+
// to you under the Apache License, Version 2.0 (the
7+
// "License"); you may not use this file except in compliance
8+
// with the License. You may obtain a copy of the License at
9+
//
10+
// http://www.apache.org/licenses/LICENSE-2.0
11+
//
12+
// Unless required by applicable law or agreed to in writing,
13+
// software distributed under the License is distributed on an
14+
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
// KIND, either express or implied. See the License for the
16+
// specific language governing permissions and limitations
17+
// under the License.
18+
//
19+
package com.cloud.server;
20+
21+
import java.util.Date;
22+
23+
/**
24+
* management server related stats
25+
*/
26+
public interface ManagementServerHostStats {
27+
Date getCollectionTime();
28+
29+
Date getSystemBootTime();
30+
31+
long getManagementServerHostId();
32+
33+
String getManagementServerHostUuid();
34+
35+
long getSessions();
36+
37+
double getCpuUtilization();
38+
39+
long getTotalJvmMemoryBytes();
40+
41+
double getFreeJvmMemoryBytes();
42+
43+
long getProcessJvmMemoryBytes();
44+
45+
long getJvmUptime();
46+
47+
long getJvmStartTime();
48+
49+
int getAvailableProcessors();
50+
51+
double getLoadAverage();
52+
53+
long getTotalInit();
54+
55+
long getTotalUsed();
56+
57+
long getMaxJvmMemoryBytes();
58+
59+
long getTotalCommitted();
60+
61+
long getPid();
62+
63+
String getJvmName();
64+
65+
String getJvmVendor();
66+
67+
String getJvmVersion();
68+
69+
String getOsDistribution();
70+
71+
int getAgentCount();
72+
73+
long getHeapMemoryUsed();
74+
75+
long getHeapMemoryTotal();
76+
77+
int getThreadsBlockedCount();
78+
79+
int getThreadsTotalCount();
80+
81+
int getThreadsDaemonCount();
82+
83+
int getThreadsRunnableCount();
84+
85+
int getThreadsTerminatedCount();
86+
87+
int getThreadsWaitingCount();
88+
89+
long getSystemMemoryTotal();
90+
91+
long getSystemMemoryFree();
92+
93+
long getSystemMemoryUsed();
94+
95+
long getSystemMemoryVirtualSize();
96+
97+
String getLogInfo();
98+
99+
/**
100+
* @return in mega hertz
101+
*/
102+
double getSystemTotalCpuCycles();
103+
104+
double[] getSystemLoadAverages();
105+
106+
long[] getSystemCyclesUsage();
107+
108+
boolean isDbLocal();
109+
110+
boolean isUsageLocal();
111+
112+
String getKernelVersion();
113+
}

‎api/src/main/java/org/apache/cloudstack/api/ApiConstants.java‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -239,10 +239,17 @@ public class ApiConstants {
239239
public static final String IS_RECURSIVE = "isrecursive";
240240
public static final String ISO_FILTER = "isofilter";
241241
public static final String ISO_GUEST_OS_NONE = "None";
242+
public static final String JAVA_DISTRIBUTION = "javadistribution";
243+
public static final String JAVA_VERSION = "javaversion";
242244
public static final String JOB_ID = "jobid";
243245
public static final String JOB_STATUS = "jobstatus";
246+
public static final String KEEPALIVE_ENABLED = "keepaliveenabled";
247+
public static final String KERNEL_VERSION = "kernelversion";
244248
public static final String LABEL = "label";
245249
public static final String LASTNAME = "lastname";
250+
public static final String LAST_BOOT = "lastboottime";
251+
public static final String LAST_SERVER_START = "lastserverstart";
252+
public static final String LAST_SERVER_STOP = "lastserverstop";
246253
public static final String LEVEL = "level";
247254
public static final String LENGTH = "length";
248255
public static final String LIMIT_CPU_USE = "limitcpuuse";
@@ -260,7 +267,6 @@ public class ApiConstants {
260267
public static final String MIGRATION_TYPE = "migrationtype";
261268
public static final String MEMORY = "memory";
262269
public static final String MODE = "mode";
263-
public static final String KEEPALIVE_ENABLED = "keepaliveenabled";
264270
public static final String NAME = "name";
265271
public static final String METHOD_NAME = "methodname";
266272
public static final String NETWORK_DOMAIN = "networkdomain";
@@ -273,6 +279,7 @@ public class ApiConstants {
273279
public static final String NEW_END_IP = "newendip";
274280
public static final String NUM_RETRIES = "numretries";
275281
public static final String OFFER_HA = "offerha";
282+
public static final String OS_DISTRIBUTION = "osdistribution";
276283
public static final String IS_SYSTEM_OFFERING = "issystem";
277284
public static final String IS_DEFAULT_USE = "defaultuse";
278285
public static final String OLD_FORMAT = "oldformat";

‎api/src/main/java/org/apache/cloudstack/api/response/ManagementServerResponse.java‎

Lines changed: 101 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,9 @@
2222
import org.apache.cloudstack.api.ApiConstants;
2323
import org.apache.cloudstack.api.BaseResponse;
2424
import org.apache.cloudstack.api.EntityReference;
25+
import org.apache.cloudstack.management.ManagementServerHost.State;
26+
27+
import java.util.Date;
2528

2629
@EntityReference(value = ManagementServerHost.class)
2730
public class ManagementServerResponse extends BaseResponse {
@@ -35,12 +38,80 @@ public class ManagementServerResponse extends BaseResponse {
3538

3639
@SerializedName(ApiConstants.STATE)
3740
@Param(description = "the state of the management server")
38-
private ManagementServerHost.State state;
41+
private State state;
3942

4043
@SerializedName(ApiConstants.VERSION)
4144
@Param(description = "the version of the management server")
4245
private String version;
4346

47+
@SerializedName(ApiConstants.JAVA_DISTRIBUTION)
48+
@Param(description = "the java distribution name running the management server process")
49+
private String javaDistribution;
50+
51+
@SerializedName(ApiConstants.JAVA_VERSION)
52+
@Param(description = "the version of the java distribution running the management server process")
53+
private String javaVersion;
54+
55+
@SerializedName(ApiConstants.OS_DISTRIBUTION)
56+
@Param(description = "the name of the OS distribution running on the management server")
57+
private String osDistribution;
58+
59+
@SerializedName(ApiConstants.LAST_SERVER_START)
60+
@Param(description = "the last time this Management Server was started")
61+
private Date lastServerStart;
62+
63+
@SerializedName(ApiConstants.LAST_SERVER_STOP)
64+
@Param(description = "the last time this Management Server was stopped")
65+
private Date lastServerStop;
66+
67+
@SerializedName(ApiConstants.LAST_BOOT)
68+
@Param(description = "the last time the host on which this Management Server runs was booted")
69+
private Date lastBoot;
70+
71+
@SerializedName(ApiConstants.KERNEL_VERSION)
72+
@Param(description = "the running OS kernel version for this Management Server")
73+
private String kernelVersion;
74+
75+
public String getId() {
76+
return this.id;
77+
}
78+
79+
public String getName() {
80+
return this.name;
81+
}
82+
83+
public State getState() {
84+
return state;
85+
}
86+
87+
public String getVersion() {
88+
return version;
89+
}
90+
91+
public String getJavaVersion() {
92+
return javaVersion;
93+
}
94+
95+
public String getJavaDistribution() {
96+
return javaDistribution;
97+
}
98+
99+
public String getOsDistribution() {
100+
return osDistribution;
101+
}
102+
103+
public Date getLastServerStart() {
104+
return lastServerStart;
105+
}
106+
107+
public Date getLastServerStop() {
108+
return lastServerStop;
109+
}
110+
111+
public Date getLastBoot() {
112+
return lastBoot;
113+
}
114+
44115
public void setId(String id) {
45116
this.id = id;
46117
}
@@ -49,11 +120,39 @@ public void setName(String name) {
49120
this.name = name;
50121
}
51122

52-
public void setState(ManagementServerHost.State state) {
123+
public void setState(State state) {
53124
this.state = state;
54125
}
55126

56127
public void setVersion(String version) {
57128
this.version = version;
58129
}
130+
131+
public void setJavaDistribution(String javaDistribution) {
132+
this.javaDistribution = javaDistribution;
133+
}
134+
135+
public void setJavaVersion(String javaVersion) {
136+
this.javaVersion = javaVersion;
137+
}
138+
139+
public void setOsDistribution(String osDistribution) {
140+
this.osDistribution = osDistribution;
141+
}
142+
143+
public void setLastServerStart(Date lastServerStart) {
144+
this.lastServerStart = lastServerStart;
145+
}
146+
147+
public void setLastServerStop(Date lastServerStop) {
148+
this.lastServerStop = lastServerStop;
149+
}
150+
151+
public void setLastBoot(Date lastBoot) {
152+
this.lastBoot = lastBoot;
153+
}
154+
155+
public void setKernelVersion(String kernelVersion) {
156+
this.kernelVersion = kernelVersion;
157+
}
59158
}

‎api/src/main/java/org/apache/cloudstack/management/ManagementServerStatus.java‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
// Licensed to the Apache Software Foundation (ASF) under one
2+
// or more contributor license agreements. See the NOTICE file
3+
// distributed with this work for additional information
4+
// regarding copyright ownership. The ASF licenses this file
5+
// to you under the Apache License, Version 2.0 (the
6+
// "License"); you may not use this file except in compliance
7+
// with the License. You may obtain a copy of the License at
8+
//
9+
// http://www.apache.org/licenses/LICENSE-2.0
10+
//
11+
// Unless required by applicable law or agreed to in writing,
12+
// software distributed under the License is distributed on an
13+
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
// KIND, either express or implied. See the License for the
15+
// specific language governing permissions and limitations
16+
// under the License.
17+
package org.apache.cloudstack.management;
18+
19+
import org.apache.cloudstack.api.InternalIdentity;
20+
21+
import java.util.Date;
22+
23+
public interface ManagementServerStatus extends InternalIdentity {
24+
long getId();
25+
26+
String getMsId();
27+
28+
Date getLastJvmStart();
29+
30+
Date getLastJvmStop();
31+
32+
Date getLastSystemBoot();
33+
34+
String getOsDistribution();
35+
36+
String getJavaName();
37+
38+
String getJavaVersion();
39+
40+
Date getUpdated();
41+
42+
Date getCreated();
43+
44+
Date getRemoved();
45+
}

‎client/src/main/webapp/WEB-INF/web.xml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,9 @@
2828
<listener>
2929
<listener-class>org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener</listener-class>
3030
</listener>
31+
<listener>
32+
<listener-class>com.cloud.api.ApiSessionListener</listener-class>
33+
</listener>
3134
<context-param>
3235
<param-name>contextConfigLocation</param-name>
3336
<param-value>classpath:META-INF/cloudstack/webApplicationContext.xml</param-value>

‎engine/orchestration/src/main/java/com/cloud/agent/manager/ConnectedAgentAttache.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
import com.cloud.utils.nio.Link;
2727

2828
/**
29-
* ConnectedAgentAttache implements an direct connection to this management server.
29+
* ConnectedAgentAttache implements a direct connection to this management server.
3030
*/
3131
public class ConnectedAgentAttache extends AgentAttache {
3232
private static final Logger s_logger = Logger.getLogger(ConnectedAgentAttache.class);

‎engine/schema/src/main/java/com/cloud/host/dao/HostDao.java‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,4 +142,11 @@ public interface HostDao extends GenericDao<HostVO, Long>, StateDao<Status, Stat
142142
HostVO findByName(String name);
143143

144144
List<HostVO> listHostsWithActiveVMs(long offeringId);
145+
146+
/**
147+
* Retrieves the number of hosts/agents this {@see ManagementServer} has responsibility over.
148+
* @param msid the id of the {@see ManagementServer}
149+
* @return the number of hosts/agents this {@see ManagementServer} has responsibility over
150+
*/
151+
int countByMs(long msid);
145152
}

‎engine/schema/src/main/java/com/cloud/host/dao/HostDaoImpl.java‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,7 @@ public class HostDaoImpl extends GenericDaoBase<HostVO, Long> implements HostDao
119119
protected SearchBuilder<HostVO> UnmanagedApplianceSearch;
120120
protected SearchBuilder<HostVO> MaintenanceCountSearch;
121121
protected SearchBuilder<HostVO> HostTypeCountSearch;
122+
protected SearchBuilder<HostVO> ResponsibleMsCountSearch;
122123
protected SearchBuilder<HostVO> HostTypeZoneCountSearch;
123124
protected SearchBuilder<HostVO> ClusterStatusSearch;
124125
protected SearchBuilder<HostVO> TypeNameZoneSearch;
@@ -179,6 +180,10 @@ public void init() {
179180
HostTypeCountSearch.and("type", HostTypeCountSearch.entity().getType(), SearchCriteria.Op.EQ);
180181
HostTypeCountSearch.done();
181182

183+
ResponsibleMsCountSearch = createSearchBuilder();
184+
ResponsibleMsCountSearch.and("managementServerId", ResponsibleMsCountSearch.entity().getManagementServerId(), SearchCriteria.Op.EQ);
185+
ResponsibleMsCountSearch.done();
186+
182187
HostTypeZoneCountSearch = createSearchBuilder();
183188
HostTypeZoneCountSearch.and("type", HostTypeZoneCountSearch.entity().getType(), SearchCriteria.Op.EQ);
184189
HostTypeZoneCountSearch.and("dc", HostTypeZoneCountSearch.entity().getDataCenterId(), SearchCriteria.Op.EQ);
@@ -1287,6 +1292,13 @@ public List<HostVO> listHostsWithActiveVMs(long offeringId) {
12871292
}
12881293
}
12891294

1295+
@Override
1296+
public int countByMs(long msid) {
1297+
SearchCriteria<HostVO> sc = ResponsibleMsCountSearch.create();
1298+
sc.setParameters("managementServerId", msid);
1299+
return getCount(sc);
1300+
}
1301+
12901302
@Override
12911303
public List<HostVO> listAllHostsByType(Host.Type type) {
12921304
SearchCriteria<HostVO> sc = TypeSearch.create();

0 commit comments

Comments
 (0)