server: fix scale for dummy template ingested vm #5099

shwstppr · 2021-06-10T12:45:32Z

Description

For vm import, dummy template can be used - https://docs.cloudstack.apache.org/en/latest/adminguide/virtual_machines.html#dummy-template
Named as system-default-vm-import-dummy-template.iso, the template is marked as removed in the CloudStack DB.
For scale operation while trying to find vm template, server needs to find in all templates including removed.

Fixes #5096

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Using cmk

Resize volume without changes:

(localcloud) SBCM5> > list unmanagedinstances clusterid=0c83cffa-e9f3-4cb7-b22d-0976a2fba552 
{
  "count": 1,
  "unmanagedinstance": [
    {
      "clusterid": "0c83cffa-e9f3-4cb7-b22d-0976a2fba552",
      "cpucorepersocket": 1,
      "cpunumber": 1,
      "cpuspeed": 0,
      "disk": [
        {
          "capacity": 2147483648,
          "controller": "ide",
          "controllerunit": 0,
          "datastorehost": "10.0.32.4",
          "datastorename": "32e98b5afe8b384da139b425a15956ac",
          "datastorepath": "/acs/primary/pr5024-t907-vmware-67u3/pr5024-t907-vmware-67u3-esxi-pri1",
          "datastoretype": "NFS",
          "id": "8-3001",
          "imagepath": "[32e98b5afe8b384da139b425a15956ac] i-2-3-VM/ROOT-3.vmdk",
          "label": "Hard disk 1",
          "position": 1
        }
      ],
      "hostid": "80b3de19-49ca-4550-b5cf-06e6c5d77944",
      "memory": 512,
      "name": "i-2-3-VM",
      "nic": [
        {
          "adaptertype": "E1000",
          "id": "Network adapter 1",
          "macaddress": "02:00:2a:21:00:01",
          "networkname": "cloud.guest.1570.200.1-vSwitch1",
          "vlanid": 1570
        }
      ],
      "osdisplayname": "CentOS 4/5 or later (64-bit)",
      "osid": "centos64Guest",
      "powerstate": "PowerOn"
    }
  ]
}
(localcloud) SBCM5> > import unmanagedinstance clusterid=0c83cffa-e9f3-4cb7-b22d-0976a2fba552 name="i-2-3-VM" serviceofferingid=5b9d15f7-0a59-4b7d-9f37-4c405fbf8e72 nicnetworklist[0].nic="Network adapter 1" nicnetworklist[0].network=d0109853-cf2c-4e5e-a966-2ed34ed6199b
{
  "virtualmachine": {
    "account": "admin",
    "affinitygroup": [],
    "cpunumber": 1,
    "cpuspeed": 500,
    "created": "2021-06-10T12:55:27+0000",
    "details": {
      "dataDiskController": "osdefault",
      "deployvm": "true",
      "nicAdapter": "E1000",
      "rootDiskController": "ide"
    },
    "displayname": "i-2-3-VM",
    "displayvm": true,
    "domain": "ROOT",
    "domainid": "b00e1fa4-c9e5-11eb-aeb3-1e00700002b4",
    "guestosid": "b013d752-c9e5-11eb-aeb3-1e00700002b4",
    "haenable": false,
    "hostid": "80b3de19-49ca-4550-b5cf-06e6c5d77944",
    "hostname": "10.0.35.43",
    "hypervisor": "VMware",
    "id": "91844bb9-16a7-4037-a545-121efed8f035",
    "instancename": "i-2-3-VM",
    "isdynamicallyscalable": false,
    "memory": 512,
    "name": "i-2-3-VM",
    "nic": [
      {
        "broadcasturi": "vlan://1570",
        "extradhcpoption": [],
        "id": "740f2181-a689-4ffd-b4eb-298088b9efde",
        "isdefault": true,
        "isolationuri": "vlan://1570",
        "macaddress": "02:00:2a:21:00:01",
        "networkid": "d0109853-cf2c-4e5e-a966-2ed34ed6199b",
        "networkname": "l2",
        "secondaryip": [],
        "traffictype": "Guest",
        "type": "L2"
      }
    ],
    "osdisplayname": "CentOS 5.0 (64-bit)",
    "ostypeid": "b013d752-c9e5-11eb-aeb3-1e00700002b4",
    "passwordenabled": false,
    "rootdeviceid": 0,
    "rootdevicetype": "ROOT",
    "securitygroup": [],
    "serviceofferingid": "5b9d15f7-0a59-4b7d-9f37-4c405fbf8e72",
    "serviceofferingname": "Small Instance",
    "state": "Running",
    "tags": [],
    "templatedisplaytext": "VM Import Default Template",
    "templateid": "0b053a68-6b29-47f6-95da-253d832f367e",
    "templatename": "system-default-vm-import-dummy-template.iso",
    "userid": "c60c6d16-c9e5-11eb-aeb3-1e00700002b4",
    "username": "admin",
    "zoneid": "518549fc-5421-499b-b945-c3b8107ae620",
    "zonename": "pr5024-t907-vmware-67u3"
  }
}
(localcloud) SBCM5> > list volumes id=8affd803-133a-4897-b8f5-98a9d40929d7 
{
  "count": 1,
  "volume": [
    {
      "account": "admin",
      "chaininfo": "{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[32e98b5afe8b384da139b425a15956ac] i-2-3-VM/ROOT-3.vmdk\",\"[32e98b5afe8b384da139b425a15956ac] 641ad3fee8d33f5d9532e594cb7b7183/641ad3fee8d33f5d9532e594cb7b7183.vmdk\"]}",
      "clusterid": "0c83cffa-e9f3-4cb7-b22d-0976a2fba552",
      "clustername": "p1-c1",
      "created": "2021-06-10T12:55:27+0000",
      "destroyed": false,
      "deviceid": 0,
      "displayvolume": true,
      "domain": "ROOT",
      "domainid": "b00e1fa4-c9e5-11eb-aeb3-1e00700002b4",
      "hypervisor": "VMware",
      "id": "8affd803-133a-4897-b8f5-98a9d40929d7",
      "isextractable": false,
      "isodisplaytext": "VM Import Default Template",
      "isoid": "0b053a68-6b29-47f6-95da-253d832f367e",
      "isoname": "system-default-vm-import-dummy-template.iso",
      "name": "ROOT-4",
      "path": "ROOT-3",
      "podid": "db876331-6a87-4637-8a67-e8f1e41f426d",
      "podname": "Pod1",
      "provisioningtype": "thin",
      "quiescevm": false,
      "serviceofferingdisplaytext": "Small Instance",
      "serviceofferingid": "5b9d15f7-0a59-4b7d-9f37-4c405fbf8e72",
      "serviceofferingname": "Small Instance",
      "size": 2147483648,
      "state": "Ready",
      "storage": "pr5024-t907-vmware-67u3-esxi-pri1",
      "storageid": "32e98b5a-fe8b-384d-a139-b425a15956ac",
      "storagetype": "shared",
      "tags": [],
      "templatedisplaytext": "VM Import Default Template",
      "templateid": "0b053a68-6b29-47f6-95da-253d832f367e",
      "templatename": "system-default-vm-import-dummy-template.iso",
      "type": "ROOT",
      "virtualmachineid": "91844bb9-16a7-4037-a545-121efed8f035",
      "vmdisplayname": "i-2-3-VM",
      "vmname": "i-2-3-VM",
      "vmstate": "Running",
      "zoneid": "518549fc-5421-499b-b945-c3b8107ae620",
      "zonename": "pr5024-t907-vmware-67u3"
    }
  ]
}
(localcloud) SBCM5> > resize volume id=8affd803-133a-4897-b8f5-98a9d40929d7 size=10
{
  "accountid": "c60b36b0-c9e5-11eb-aeb3-1e00700002b4",
  "cmd": "org.apache.cloudstack.api.command.admin.volume.ResizeVolumeCmdByAdmin",
  "completed": "2021-06-10T12:57:31+0000",
  "created": "2021-06-10T12:57:31+0000",
  "jobid": "9a55f792-53fe-4ede-aa43-b3589d2f8882",
  "jobprocstatus": 0,
  "jobresult": {
    "errorcode": 530,
    "errortext": "Command failed due to Internal Server Error"
  },
  "jobresultcode": 530,
  "jobresulttype": "object",
  "jobstatus": 2,
  "userid": "c60c6d16-c9e5-11eb-aeb3-1e00700002b4"
}

NPE observed:

2021-06-10 12:57:31,821 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-28:ctx-26c1b20d job-33) (logid:9a55f792) Unexpected exception while executing org.apache.cloudstack.api.command.admin.volume.ResizeVolumeCmdByAdmin
java.lang.NullPointerException
        at com.cloud.storage.VolumeApiServiceImpl.resizeVolume(VolumeApiServiceImpl.java:928)
        at com.cloud.storage.VolumeApiServiceImpl.resizeVolume(VolumeApiServiceImpl.java:192)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
        at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
        at com.sun.proxy.$Proxy217.resizeVolume(Unknown Source)
        at org.apache.cloudstack.api.command.user.volume.ResizeVolumeCmd.execute(ResizeVolumeCmd.java:191)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
        at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

After changes:

(localcloud) SBCM5> > list unmanagedinstances clusterid=0c83cffa-e9f3-4cb7-b22d-0976a2fba552
{
  "count": 1,
  "unmanagedinstance": [
    {
      "clusterid": "0c83cffa-e9f3-4cb7-b22d-0976a2fba552",
      "cpucorepersocket": 1,
      "cpunumber": 1,
      "cpuspeed": 0,
      "disk": [
        {
          "capacity": 2147483648,
          "controller": "pvscsi",
          "controllerunit": 0,
          "datastorehost": "10.0.32.4",
          "datastorename": "3d56ca3dc1143cfdb7d129b5b64d73b2",
          "datastorepath": "/acs/primary/pr5024-t907-vmware-67u3/pr5024-t907-vmware-67u3-esxi-pri2",
          "datastoretype": "NFS",
          "id": "11-2000",
          "imagepath": "[3d56ca3dc1143cfdb7d129b5b64d73b2] i-2-5-VM/ROOT-5_2.vmdk",
          "label": "Hard disk 1",
          "position": 0
        }
      ],
      "hostid": "80b3de19-49ca-4550-b5cf-06e6c5d77944",
      "memory": 512,
      "name": "i-2-5-VM",
      "nic": [
        {
          "adaptertype": "E1000",
          "id": "Network adapter 1",
          "macaddress": "02:00:30:f3:00:02",
          "networkname": "cloud.guest.1570.200.1-vSwitch1",
          "vlanid": 1570
        }
      ],
      "osdisplayname": "CentOS 4/5 or later (64-bit)",
      "osid": "centos64Guest",
      "powerstate": "PowerOn"
    }
  ]
}
(localcloud) SBCM5> > import unmanagedinstance clusterid=0c83cffa-e9f3-4cb7-b22d-0976a2fba552 name="i-2-5-VM" serviceofferingid=5b9d15f7-0a59-4b7d-9f37-4c405fbf8e72 nicnetworklist[0].nic="Network adapter 1" nicnetworklist[0].network=d0109853-cf2c-4e5e-a966-2ed34ed6199b
{
  "virtualmachine": {
    "account": "admin",
    "affinitygroup": [],
    "cpunumber": 1,
    "cpuspeed": 500,
    "created": "2021-06-10T14:22:49+0000",
    "details": {
      "dataDiskController": "osdefault",
      "deployvm": "true",
      "nicAdapter": "E1000",
      "rootDiskController": "pvscsi"
    },
    "displayname": "i-2-5-VM",
    "displayvm": true,
    "domain": "ROOT",
    "domainid": "b00e1fa4-c9e5-11eb-aeb3-1e00700002b4",
    "guestosid": "b013d752-c9e5-11eb-aeb3-1e00700002b4",
    "haenable": false,
    "hostid": "80b3de19-49ca-4550-b5cf-06e6c5d77944",
    "hostname": "10.0.35.43",
    "hypervisor": "VMware",
    "id": "652b7235-7a21-4483-92a7-c8d7124f1e05",
    "instancename": "i-2-5-VM",
    "isdynamicallyscalable": false,
    "memory": 512,
    "name": "i-2-5-VM",
    "nic": [
      {
        "broadcasturi": "vlan://1570",
        "extradhcpoption": [],
        "id": "cb89dd9b-6eff-4add-859b-a8e445ceaa66",
        "isdefault": true,
        "isolationuri": "vlan://1570",
        "macaddress": "02:00:30:f3:00:02",
        "networkid": "d0109853-cf2c-4e5e-a966-2ed34ed6199b",
        "networkname": "l2",
        "secondaryip": [],
        "traffictype": "Guest",
        "type": "L2"
      }
    ],
    "osdisplayname": "CentOS 5.0 (64-bit)",
    "ostypeid": "b013d752-c9e5-11eb-aeb3-1e00700002b4",
    "passwordenabled": false,
    "rootdeviceid": 0,
    "rootdevicetype": "ROOT",
    "securitygroup": [],
    "serviceofferingid": "5b9d15f7-0a59-4b7d-9f37-4c405fbf8e72",
    "serviceofferingname": "Small Instance",
    "state": "Running",
    "tags": [],
    "templatedisplaytext": "VM Import Default Template",
    "templateid": "0b053a68-6b29-47f6-95da-253d832f367e",
    "templatename": "system-default-vm-import-dummy-template.iso",
    "userid": "c60c6d16-c9e5-11eb-aeb3-1e00700002b4",
    "username": "admin",
    "zoneid": "518549fc-5421-499b-b945-c3b8107ae620",
    "zonename": "pr5024-t907-vmware-67u3"
  }
}
(localcloud) SBCM5> > stop virtualmachine id=652b7235-7a21-4483-92a7-c8d7124f1e05 filter=id,state
{}
(localcloud) SBCM5> > list volumes id=1deb7425-88bf-47f2-b3e1-4228e6a5e87e filter=id,size
{
  "count": 1,
  "volume": [
    {
      "id": "1deb7425-88bf-47f2-b3e1-4228e6a5e87e",
      "size": 2147483648
    }
  ]
}
(localcloud) SBCM5> > resize volume id=1deb7425-88bf-47f2-b3e1-4228e6a5e87e size=10
{
  "volume": {
    "account": "admin",
    "chaininfo": "{\"diskDeviceBusName\":\"scsi0:0\",\"diskChain\":[\"[3d56ca3dc1143cfdb7d129b5b64d73b2] i-2-5-VM/ROOT-5_2.vmdk\"]}",
    "clusterid": "0c83cffa-e9f3-4cb7-b22d-0976a2fba552",
    "clustername": "p1-c1",
    "created": "2021-06-10T14:22:49+0000",
    "destroyed": false,
    "deviceid": 0,
    "displayvolume": true,
    "domain": "ROOT",
    "domainid": "b00e1fa4-c9e5-11eb-aeb3-1e00700002b4",
    "hypervisor": "VMware",
    "id": "1deb7425-88bf-47f2-b3e1-4228e6a5e87e",
    "isextractable": false,
    "isodisplaytext": "VM Import Default Template",
    "isoid": "0b053a68-6b29-47f6-95da-253d832f367e",
    "isoname": "system-default-vm-import-dummy-template.iso",
    "name": "ROOT-6",
    "path": "ROOT-5_2",
    "podid": "db876331-6a87-4637-8a67-e8f1e41f426d",
    "podname": "Pod1",
    "provisioningtype": "thin",
    "quiescevm": false,
    "serviceofferingdisplaytext": "Small Instance",
    "serviceofferingid": "5b9d15f7-0a59-4b7d-9f37-4c405fbf8e72",
    "serviceofferingname": "Small Instance",
    "size": 10737418240,
    "state": "Ready",
    "storage": "pr5024-t907-vmware-67u3-esxi-pri2",
    "storageid": "3d56ca3d-c114-3cfd-b7d1-29b5b64d73b2",
    "storagetype": "shared",
    "tags": [],
    "templatedisplaytext": "VM Import Default Template",
    "templateid": "0b053a68-6b29-47f6-95da-253d832f367e",
    "templatename": "system-default-vm-import-dummy-template.iso",
    "type": "ROOT",
    "virtualmachineid": "652b7235-7a21-4483-92a7-c8d7124f1e05",
    "vmdisplayname": "i-2-5-VM",
    "vmname": "i-2-5-VM",
    "vmstate": "Stopped",
    "zoneid": "518549fc-5421-499b-b945-c3b8107ae620",
    "zonename": "pr5024-t907-vmware-67u3"
  }
}

GabrielBrascher

LGTM

For vm import, dummy template can be used - https://docs.cloudstack.apache.org/en/latest/adminguide/virtual_machines.html#dummy-template Named as system-default-vm-import-dummy-template.iso, the template is marked as removed in the CloudStack DB. For scale operation while trying to find vm template, server needs to find in all templates including removed. Fixes apache#5096 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2021-06-10T13:37:48Z

@blueorangutan package

blueorangutan · 2021-06-10T13:39:03Z

@shwstppr a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-06-10T14:03:57Z

Packaging result: ✔️ centos7 ✔️ centos8 ✔️ debian. SL-JID 199

shwstppr · 2021-06-11T04:43:16Z

Fix added in #5095

GabrielBrascher approved these changes Jun 10, 2021

View reviewed changes

shwstppr force-pushed the fix-scale-ingested-vm branch from 2710f59 to 3bf4b14 Compare June 10, 2021 12:49

shwstppr added this to the 4.15.1.0 milestone Jun 10, 2021

shwstppr changed the base branch from master to 4.15 June 10, 2021 13:37

apache deleted a comment from blueorangutan Jun 10, 2021

shwstppr mentioned this pull request Jun 10, 2021

Failed to scale between Service Offerings with the same root disk size #5095

Merged

12 tasks

shwstppr closed this Jun 11, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

server: fix scale for dummy template ingested vm #5099

server: fix scale for dummy template ingested vm #5099

Uh oh!

shwstppr commented Jun 10, 2021 •

edited

Loading

Uh oh!

GabrielBrascher left a comment

Uh oh!

shwstppr commented Jun 10, 2021

Uh oh!

blueorangutan commented Jun 10, 2021

Uh oh!

blueorangutan commented Jun 10, 2021

Uh oh!

shwstppr commented Jun 11, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

server: fix scale for dummy template ingested vm #5099

server: fix scale for dummy template ingested vm #5099

Uh oh!

Conversation

shwstppr commented Jun 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

GabrielBrascher left a comment

Choose a reason for hiding this comment

Uh oh!

shwstppr commented Jun 10, 2021

Uh oh!

blueorangutan commented Jun 10, 2021

Uh oh!

blueorangutan commented Jun 10, 2021

Uh oh!

shwstppr commented Jun 11, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shwstppr commented Jun 10, 2021 •

edited

Loading