Added action in UI for syncStoragePool API for DatastoreCluster type primary storages.

Fixes apache#5086

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

…disk size (apache#5095)

* Cover a case where resizing root disk failed; add isNotPossibleToResize method.

* remove format from resize validation

* Revert if-conditional changes that removed ImageFormat.ISO validation

* Add JUnit tests for VolumeApiServiceImpl.isNotPossibleToResize

* Fix checkstyle of test Class

* Use _templateDao.findByIdIncludingRemoved instead of _templateDao.findById

* Prevent null serviceOfferingView and Mock findByIdIncludingRemoved instead of findById

Modifications in TranslationMenu.vue as well

Fixes apache#5107

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>

…ated over physical nic instead of linux bridge (apache#5078)

* vxlan: arp does not work between hosts as multicast group is communicated over physical nic instead of linux bridge

when linux bridge is setup (refer to http://docs.cloudstack.apache.org/projects/archived-cloudstack-getting-started/en/latest/networking/vxlan.html#configure-product-to-use-vxlan-plugin) and used as the kvm traffic label of physical networks, the vms on different hosts cannot reach each other.

(1) does not work:
```
/usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan.sh -v 1001 -p eth1 -b brvx-1001 -o add
```
"bridge fdb" shows
```
00:00:00:00:00:00 dev vxlan1001 dst 239.0.3.233 via eth1 self permanent
```

(2) this works:
```
/usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan.sh -v 1001 -p cloudbr1 -b brvx-1001 -o add
```
"bridge fdb" shows
```
00:00:00:00:00:00 dev vxlan1001 dst 239.0.3.233 via cloudbr1 self permanent
```

* vxlan: fix issue if kvm network label is not set

…nts. (apache#5037)

- Added connection manager to the gateway client.
 - Renew the client session on '401 Unauthorized' response.
 - Refactored the gateway client calls, for GET and POST methods.
 - Consume the http entity content after login/(re)authentication and close the content stream if exists.
 - Updated storage pool client connection timeout configuration 'storage.pool.client.timeout' to non-dynamic.
 - Added storage pool client max connections configuration 'storage.pool.client.max.connections' (default: 100) to specify the maximum connections for the ScaleIO storage pool client.
 - Updated unit tests.
and blocked the attach volume operation for uploaded volume on ScaleIO/PowerFlex storage pool

…5113)

Co-authored-by: Daan Hoogland <dahn@onecht.net>

…ache#5105)

This PR fixes the issue that nic has wrong gateway after updating vm nic.

Steps to reproduce the issue
(1) create shared network (in advanced zone or advanced zone with sg)
(2) create new shared network (with same startip/endip/netmask, but different gateway).
(3) create a vm in new network
(4) stop vm and update vm nic ip address

Expected result:
The vm has correct gateway and netmask (of second network)

Actual result:
The vm has wrong gateway and netmask (of first network)

…tualMachine (apache#4819)

On API `resetSSHKeyForVirtualMachine`, ACS also regenerates VM password when it uses a template with `Password Enabled` as true; there is already anAPI to reset VM password, therefore, the reset SSH keys API should not reset the VM SSH password as well.

Besides running a meaningless process, the VM's password regeneration slows down the main process and may cause a confusion in operations due to password change in the VM without being explicity requested.

Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>

This PR intends to improve logging on agent start to facilitate troubleshooting.

Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>

* fix of detach volume while OS is in boot state

* check that the VM could be accessed trough ssh

…ates) (apache#5129)

* ui: Notify vm password on reinstall of VM (for password enabled templates)

* refactor

…riate. (apache#5131)

…bytes) (apache#5150)

This reverts change in commit

apache@065fa1c#diff-c78f7f699938cf529cbcef26bda2ed93630b58f336986fd0753fbfa1a6adbc43L103-R103

…ache#5137)

* Change logrotate interval to hourly

The logrotate config says interval as hourly but it relies
on timer service to be invoked but in timer the frequency
is mentioned as 12h. So it wont be invoked every hour.

So change the frequency to hourly

* Add change to vpc router

TooltipButton was not added as Vue component. It was leading to error in page load.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

…#5153)

* db: Cleanup unused tables

* Removing volume_host_ref references

* Removing template_host_ref references

* fix space issue

* Fix fk constraint

* Removing certificate table

* Revert "Removing certificate table"

This reverts commit fa24e64.

* Addressing comments

…che#5130)

This PR fixes the deletion will be handled by DefaultSnapshotStrategy::deleteSnapshot apache#4797

…pache#5165)

* Prevent starting a VM in destroyed state (or any state but Stopped)

* Document cidrlist param deprecation in CreatePortForwardingRule and CreateFirewallRule

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Enhance log messages with hostName

* Use host.toString() on most of host logs.

* Remove redundant "Host" in logs and enhance logs

* duplicated "for"

* Adopt String.format, and enhance code

* Address reviews enhancing log messages

Update server/src/main/java/com/cloud/resource/ResourceManagerImpl.java
    -- server/src/main/java/com/cloud/vm/UserVmManagerImpl.java
    -- server/src/main/java/com/cloud/resource/RollingMaintenanceManagerImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Fix String.format issue and change log message from debug to warn

* Fix checkstyle issue

* Fix string.format log

* Address review: enhance logs

* Enhance log of hosts in maintenance avoid list

* Remove "VM" on logs as vm.toString() already appends VM-<details>

* Add more details of the VM when postStateTransitionEvent

* Address reviewer and enhance VMInstanceVO.toString()

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

…rce (apache#5182)

* ui: refactor labels with tooltip in forms

Adds new Vue component TooltipLabel for easier re-use

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* moved component in widgets

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* moved TooltipButton in widgets

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* missing change in previous commit

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>

* form improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* tooltip icon class fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update ui/src/views/storage/UploadLocalVolume.vue

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* Update ui/src/views/storage/CreateVolume.vue

Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>

* ui: refactor get api params in forms

Refactor code getting api params for APIs in UI forms.
Added a new util plugin in plugins.js

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* apiserver : Ensure required parameters are not empty

* supported services are not required as they can be empty

* Removing redudndant validators

…apache#5144)

* server: fix failed to apply userdata when enable static nat

* server: fix cannot expunge vm as applyUserdata fails

* configdrive: fix ISO is not recognized when plug a new nic

* configdrive: detach and attach configdrive ISO as it is changed when plug a new nic or migrate vm

* configdrive test: (1) password file does not exists in recreated ISO; (2) vm hostname should be changed after migration

* configdrive: use centos55 template with sshkey and configdrive support

* configdrive: disklabel is 'config-2' for configdrive ISO

* configdrive: use copy for configdrive ISO and move for other template/volume/iso

* configdrive: use public-keys.txt

* configdrive test: fix (1) update_template ; (2) ssh into vm by keypair

Fixes apache#5177

Fixes apache#5192

* vmware: fix migrate vm with volume

Recent forward merge of 4.15 branch accidentally brought a bug in VM relocation method for VMware while trying to find datastore for the migrated volume.
This PR fixes it by using either of available target or source host.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* refactor: migrate with storage host capability check

Refactors Boolean HypervisorCapabilitiesDao::isStorageMotionSupported to boolean HypervisorCapabilitiesDao::isStorageMotionSupported for simplifying callers.
Refactors log messages.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* simplify

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* review comments addressed

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* var rename

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

New NICs shouldn't added to VM with network from admin account, same as deployment.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* systemvmtemplate: use latest LTS kernel from buster-ports

Use the latest v5.10 Linux kernel (LTS) from buster-ports

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 2dd1650276e7fa31fc3b920b67be8396647c279d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update configure_grub.sh

* changes

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix failing build

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update install_systemvm_packages.sh

Currently there is no disk IO driver configuration for VMs running on KVM. That's OK for most the cases; however, recently there have been added some quite interesting optimizations with the IO driver io_uring.

Note that IO URING requires:

Qemu >= 5.0, and
Libvirt >= 6.3.0.
By using io_uring we can see a massive I/O performance improvement within Virtual Machines running from Local and/or NFS storage.

This implementation enhances the KVM disk configuration by adding workflow for setting the disk IO drivers. Additionally, if the Qemu and Libvirt versions matches with the required for having io_uring we are going to set it on the VM. If there is no support for such driver we keep it as it is nowadays, without any IO driver configured.

Fixes: apache#4883

On newer libvirt/qemu it seems PCI hot-plugging could be an issue as
seen in:

https://www.suse.com/support/kb/doc/?id=000019383
https://bugs.launchpad.net/nova/+bug/1836065

This was found to be true on ARM64/aarch64 platform (tested on
RaspberryPi4). As per the default machine doc, it advises to
pre-allocate PCI controllers on the machine and pcie-to-pci-bridge based
controller for legacy PCI models:
https://libvirt.org/pci-hotplug.html#x86_64-q35

This patch introduces the concept as a workaround until a proper fix is
done (ideally in the upstream libvirt/qemu projects). Until then client
code can add 32 PCI controllers and a pcie-to-pci-bridge controller for
aarch64 platforms.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Merge the pull request of full clone backend api