Fix direct download URL checks #7693

nvazquez · 2023-06-26T22:00:46Z

Description

This PR fixes the URL check for direct downloads, in the case of HTTPS URLs the certificates were not loaded into the SSL context

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Tested on KVM with HTTPS, HTTP and metalink templates

nvazquez · 2023-06-26T22:01:02Z

@blueorangutan package

blueorangutan · 2023-06-26T22:02:03Z

@nvazquez a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-06-26T22:53:49Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6341

nvazquez · 2023-06-27T01:07:49Z

@blueorangutan test

blueorangutan · 2023-06-27T01:08:04Z

@nvazquez a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2023-06-27T01:29:25Z

[SF] Trillian Build Failed (tid-6870)

kiranchavala · 2023-06-27T07:10:27Z

@blueorangutan test

blueorangutan · 2023-06-27T07:12:03Z

@kiranchavala a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2023-06-27T19:39:11Z

[SF] Trillian test result (tid-6875)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43331 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7693-t6875-kvm-centos7.zip
Smoke tests completed. 101 look OK, 7 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_migrate_VM_and_root_volume	`Error`	77.66	test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks	`Error`	58.57	test_vm_life_cycle.py
test_10_attachAndDetach_iso	`Error`	0.06	test_vm_life_cycle.py
test_06_download_detached_volume	`Error`	301.86	test_volumes.py
test_13_migrate_volume_and_change_offering	`Error`	125.80	test_volumes.py
ContextSuite context=TestIpv6Vpc>:setup	`Error`	0.00	test_vpc_ipv6.py
ContextSuite context=TestVPCRedundancy>:setup	`Error`	0.00	test_vpc_redundant.py
ContextSuite context=TestVPCNics>:setup	`Error`	0.00	test_vpc_router_nics.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup	`Error`	0.00	test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setup	`Error`	0.00	test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup	`Error`	0.00	test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setup	`Error`	0.00	test_vpc_vpn.py
test_disable_oobm_ha_state_ineligible	`Error`	0.37	test_hostha_kvm.py

core/src/main/java/org/apache/cloudstack/direct/download/HttpsDirectTemplateDownloader.java

kiranchavala

LGTM

Tested the issue with http, https and metalink template links and the directdownload link is working fine

Before the fix

2023-06-27 05:42:41,470 WARN [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-5:null) (logid:96df14c1) Error downloading template 226 due to: Error on HTTPS request: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

After the fix

2023-06-27 11:02:09,364 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:c6e0ab18) Processing command: org.apache.cloudstack.agent.directdownload.HttpsDirectDownloadCommand


2023-06-27 11:02:09,364 DEBUG [storage.resource.StorageSubsystemCommandHandlerBase] (agentRequest-Handler-3:null) (logid:c6e0ab18) Executing command HttpsDirectDownloadCommand: [{"url":"https://ftp.yz.yamagata-u.ac.jp/pub/linux/fedora-projects/fedora/linux/releases/37/Cloud/x86_64/images/Fedora-Cloud-Base-37-1.7.x86_64.qcow2","templateId":204,"destData":{"origUrl":"https://ftp.yz.yamagata-u.ac.jp/pub/linux/fedora-projects/fedora/linux/releases/37/Cloud/x86_64/images/Fedora-Cloud-Base-37-1.7.x86_64.qcow2","uuid":"0a5a5582-a6d2-4566-af24-206568bcf354","id":204,"format":"QCOW2","accountId":2,"hvm":true,"displayText":"fedora3","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"3c9ef251-b75d-3d2b-88ee-0122246e9f7e","name":"pr7693-t6872-kvm-centos7-kvm-pri1","id":1,"poolType":"NetworkFilesystem","host":"10.0.32.4","path":"/acs/primary/pr7693-t6872-kvm-centos7/pr7693-t6872-kvm-centos7-kvm-pri1","port":2049,"url":"NetworkFilesystem://10.0.32.4/acs/primary/pr7693-t6872-kvm-centos7/pr7693-t6872-kvm-centos7-kvm-pri1/?ROLE=Primary&STOREUUID=3c9ef251-b75d-3d2b-88ee-0122246e9f7e","isManaged":false}},"name":"204-2-54b96105-91a3-3d81-a34f-85692f07c72a","size":5368709120,"hypervisorType":"KVM","bootable":false,"uniqueName":"204-2-54b96105-91a3-3d81-a34f-85692f07c72a","directDownload":true,"deployAsIs":false},"destPool":{"uuid":"3c9ef251-b75d-3d2b-88ee-0122246e9f7e","name":"pr7693-t6872-kvm-centos7-kvm-pri1","id":1,"poolType":"NetworkFilesystem","host":"10.0.32.4","path":"/acs/primary/pr7693-t6872-kvm-centos7/pr7693-t6872-kvm-centos7-kvm-pri1","port":2049,"url":"NetworkFilesystem://10.0.32.4/acs/primary/pr7693-t6872-kvm-centos7/pr7693-t6872-kvm-centos7-kvm-pri1/?ROLE=Primary&STOREUUID=3c9ef251-b75d-3d2b-88ee-0122246e9f7e","isManaged":false},"headers":{},"connectTimeout":5000,"soTimeout":5000,"connectionRequestTimeout":5000,"templateSize":5368709120,"format":"QCOW2","wait":10800,"bypassHostMaintenance":false}].

2023-06-27 11:02:09,364 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-3:null) (logid:c6e0ab18) Verifying temporary location for downloading the template exists on the host

2023-06-27 11:02:10,542 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-3:null) (logid:c6e0ab18) Checking for free space on the host for downloading the template with physical size: 492830720 and virtual size: 5368709120

2023-06-27 11:02:10,543 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:c6e0ab18) Executing: /bin/bash -c df --output=avail /var/lib/libvirt/images -B 1 | tail -1

2023-06-27 11:02:10,546 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:c6e0ab18) Executing while with timeout : 3600000


2023-06-27 11:02:10,558 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:c6e0ab18) Successfully refreshed pool 3c9ef251-b75d-3d2b-88ee-0122246e9f7e Capacity: (1.9990 TB) 2197949513728 Used: (1.4938 TB) 1642465329152 Available: (517.33 GB) 555484184576

2023-06-27 11:02:10,558 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:c6e0ab18) Executing: /bin/bash -c sed -n '/keystore.passphrase/p' '/etc/cloudstack/agent/agent.properties' 2>/dev/null  | sed 's/keystore.passphrase=//g' 2>/dev/null

2023-06-27 11:02:10,561 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:c6e0ab18) Executing while with timeout : 3600000

2023-06-27 11:02:10,565 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:c6e0ab18) Execution is successful.

2023-06-27 11:02:10,585 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-3:null) (logid:c6e0ab18) Trying to download template

2023-06-27 11:02:11,677 INFO  [direct.download.DirectTemplateDownloaderImpl] (agentRequest-Handler-3:null) (logid:c6e0ab18) Downloading template 204 from https://ftp.yz.yamagata-u.ac.jp/pub/linux/fedora-projects/fedora/linux/releases/37/
Cloud/x86_64/images/Fedora-Cloud-Base-37-1.7.x86_64.qcow2 to: /var/lib/libvirt/images/template/2/204/Fedora-Cloud-Base-37-1.7.x86_64.qcow2

2023-06-27 11:03:54,376 INFO  [direct.download.DirectTemplateDownloaderImpl] (agentRequest-Handler-3:null) (logid:c6e0ab18) No checksum provided, skipping checksum validation

2023-06-27 11:03:54,379 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:c6e0ab18) Executing: /bin/bash -c file /var/lib/libvirt/images/template/2/204/Fedora-Cloud-Base-37-1.7.x86_64.qcow2 | awk -F' ' '{print $2}'
2023-06-27 11:03:54,382 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:c6e0ab18) Executing while with timeout : 3600000
2023-06-27 11:03:54,388 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:c6e0ab18) Execution is successful.
2023-06-27 11:03:54,389 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:c6e0ab18) Executing: /bin/bash -c mv /var/lib/libvirt/images/template/2/204/Fedora-Cloud-Base-37-1.7.x86_64.qcow2 /mnt/3c9ef251-b75d-3d2b-88ee-0122246e9f7e/a76d363c-ba0f-47e3-bfd3-10448127ec1d

shwstppr

LGTM

Though the effective change/improvement is to load custom and system-trusted SSL certificates while checking accessibility and getting the size for a direct-download QCOW2 template, a lot of refactoring was needed to achieve that.

Tested with a HTTPS template URL and can see it getting registered without errors.

weizhouapache

code lgtm

codecov · 2023-06-28T19:28:14Z

Codecov Report

Merging #7693 (eff5ce9) into 4.18 (985f0ec) will increase coverage by 0.00%.
The diff coverage is 0.00%.

@@            Coverage Diff            @@
##               4.18    #7693   +/-   ##
=========================================
  Coverage     13.00%   13.01%           
- Complexity     9013     9015    +2     
=========================================
  Files          2719     2719           
  Lines        256854   256836   -18     
  Branches      40048    40045    -3     
=========================================
+ Hits          33413    33418    +5     
+ Misses       219255   219231   -24     
- Partials       4186     4187    +1

Impacted Files	Coverage Δ
...r/kvm/resource/wrapper/LibvirtCheckUrlCommand.java	`22.22% <0.00%> (+8.88%)`	⬆️
...ud/hypervisor/kvm/storage/KVMStorageProcessor.java	`5.02% <0.00%> (+0.04%)`	⬆️
...n/java/com/cloud/storage/VolumeApiServiceImpl.java	`15.28% <0.00%> (ø)`

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

rohityadavcloud · 2023-06-30T14:37:10Z

@blueorangutan package

blueorangutan · 2023-06-30T14:38:03Z

@rohityadavcloud a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-06-30T15:32:29Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 6374

rohityadavcloud · 2023-07-03T07:32:47Z

@blueorangutan test

blueorangutan · 2023-07-03T07:34:03Z

@rohityadavcloud a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

rohityadavcloud

LGTM didn't test it, need to pass smoketests

DaanHoogland

new files don't end with a new line. Can you add those?

DaanHoogland · 2023-07-03T12:12:01Z

core/src/main/java/org/apache/cloudstack/direct/download/HttpsDirectTemplateDownloader.java

+        }
+    }
+
+    private SSLContext getSSLContext() {


there are nested try statements in this method. I think it can do with some restructuring.

DaanHoogland · 2023-07-03T12:12:40Z

core/src/main/java/org/apache/cloudstack/direct/download/HttpsDirectTemplateDownloader.java

+    private SSLContext getSSLContext() {
+        try {
+            KeyStore customKeystore  = KeyStore.getInstance("jks");
+            try (FileInputStream instream = new FileInputStream(new File("/etc/cloudstack/agent/cloud.jks"))) {


can this move to a separate method?

DaanHoogland · 2023-07-03T12:13:00Z

core/src/main/java/org/apache/cloudstack/direct/download/HttpsDirectTemplateDownloader.java

+            KeyStore defaultKeystore = KeyStore.getInstance(KeyStore.getDefaultType());
+            String relativeCacertsPath = "/lib/security/cacerts".replace("/", File.separator);
+            String filename = System.getProperty("java.home") + relativeCacertsPath;
+            try (FileInputStream is = new FileInputStream(filename)) {


can this load to a separate method as well?

DaanHoogland · 2023-07-03T12:20:05Z

core/src/main/java/org/apache/cloudstack/direct/download/HttpsMultiTrustManager.java

+            return Iterables.getFirst(Iterables.filter(
+                    Arrays.asList(factory.getTrustManagers()), X509TrustManager.class), null);
+        } catch (NoSuchAlgorithmException | KeyStoreException e) {
+            e.printStackTrace();


why not logging, but error output?

blueorangutan · 2023-07-03T20:19:49Z

[SF] Trillian test result (tid-6937)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44508 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7693-t6937-kvm-centos7.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_migrate_VM_and_root_volume	`Error`	79.71	test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks	`Error`	53.34	test_vm_life_cycle.py

DaanHoogland · 2023-07-06T08:22:40Z

@rohityadavcloud , that was an premature merge. I have some comments out that should be addressed!

DaanHoogland · 2023-07-06T08:22:56Z

@rohityadavcloud , that was an premature merge. I have some comments out that should be addressed!

cc @nvazquez

weizhouapache · 2023-08-31T08:11:38Z

this could be fixed by #7923

Fix direct download URL checks

eff5ce9

nvazquez added type:bug component:kvm component:templates labels Jun 26, 2023

nvazquez added this to the 4.18.1.0 milestone Jun 26, 2023

nvazquez requested review from DaanHoogland, rohityadavcloud, shwstppr and weizhouapache June 26, 2023 22:00

boring-cyborg bot added component:agent component:storage labels Jun 26, 2023

weizhouapache reviewed Jun 28, 2023

View reviewed changes

core/src/main/java/org/apache/cloudstack/direct/download/HttpsDirectTemplateDownloader.java Show resolved Hide resolved

kiranchavala approved these changes Jun 28, 2023

View reviewed changes

shwstppr approved these changes Jun 28, 2023

View reviewed changes

weizhouapache closed this Jun 28, 2023

weizhouapache reopened this Jun 28, 2023

weizhouapache approved these changes Jun 28, 2023

View reviewed changes

rohityadavcloud marked this pull request as ready for review June 30, 2023 14:36

rohityadavcloud approved these changes Jul 3, 2023

View reviewed changes

DaanHoogland reviewed Jul 3, 2023

View reviewed changes

rohityadavcloud merged commit c733a23 into apache:4.18 Jul 6, 2023

rohityadavcloud deleted the fix-direct-download-url-check-ca branch July 6, 2023 08:17

This was referenced Aug 31, 2023

Failed to register direct-download template in 4.18.1.0-RC1 #7929

Closed

kvm: fix error when enable SSL for kvm agent #7923

Merged

Fix direct download URL checks #7693

Fix direct download URL checks #7693

Uh oh!

Conversation

nvazquez commented Jun 26, 2023

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

nvazquez commented Jun 26, 2023

Uh oh!

blueorangutan commented Jun 26, 2023

Uh oh!

blueorangutan commented Jun 26, 2023

Uh oh!

nvazquez commented Jun 27, 2023

Uh oh!

blueorangutan commented Jun 27, 2023

Uh oh!

blueorangutan commented Jun 27, 2023

Uh oh!

kiranchavala commented Jun 27, 2023

Uh oh!

blueorangutan commented Jun 27, 2023

Uh oh!

blueorangutan commented Jun 27, 2023

Uh oh!

Uh oh!

kiranchavala left a comment

Choose a reason for hiding this comment

Uh oh!

shwstppr left a comment

Choose a reason for hiding this comment

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Jun 28, 2023

Codecov Report

Uh oh!

rohityadavcloud commented Jun 30, 2023

Uh oh!

blueorangutan commented Jun 30, 2023

Uh oh!

blueorangutan commented Jun 30, 2023

Uh oh!

rohityadavcloud commented Jul 3, 2023

Uh oh!

blueorangutan commented Jul 3, 2023

Uh oh!

rohityadavcloud left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland Jul 3, 2023

Choose a reason for hiding this comment

Uh oh!

DaanHoogland Jul 3, 2023

Choose a reason for hiding this comment

Uh oh!

DaanHoogland Jul 3, 2023

Choose a reason for hiding this comment

Uh oh!

DaanHoogland Jul 3, 2023

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Jul 3, 2023

Uh oh!

DaanHoogland commented Jul 6, 2023

Uh oh!

DaanHoogland commented Jul 6, 2023

Uh oh!

weizhouapache commented Aug 31, 2023

Uh oh!