systemvmtemplate: use latest LTS kernel from buster-ports #5073

rohityadavcloud · 2021-06-03T17:29:15Z

Use the latest v5.10 Linux kernel (LTS) from buster-ports and fixes systemvmtemplate build issue on main branch.

Signed-off-by: Rohit Yadav rohit.yadav@shapeblue.com

Use the latest v5.10 Linux kernel (LTS) from buster-ports Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com> (cherry picked from commit 2dd1650276e7fa31fc3b920b67be8396647c279d) Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

weizhouapache · 2021-06-04T06:19:08Z

@rhtyd good.
maybe consider #3900 and set a smaller template size ?

rohityadavcloud · 2021-06-04T10:58:38Z

@weizhouapache I think it would be too much maintenance work for gain to maintain separate systemvmtemplate for each hypervisor. These days users can download 100MBs more in few extra seconds.

weizhouapache · 2021-06-04T11:13:55Z

tools/appliance/systemvmtemplate/scripts/apt_upgrade.sh

  apt-get autoclean
  apt-get clean
+  reboot
 }


@rhtyd might 'reboot' be an issue in packer processing ?

No see template.json

weizhouapache · 2021-06-04T11:27:00Z

tools/appliance/systemvmtemplate/scripts/configure_grub.sh


 function configure_grub() {
+  # Remove old kernel
+  apt-get remove -y --purge linux-image-amd64 linux-image-4.19.0-16-amd64 || true


@rhtyd two potential issues in my opinion
(1) the versions are different in debian releases (debian 10.9.0: kernel 4.19.0.16.debian 10.8.0: kernel 4.19.0.14,etc)
(2) might there be more than 2 versions ? for example, one new kernel by apt-get -q -y dist-upgrade, another new kernel by apt-get -q -y dist-upgrade -t buster-backports ?

I see, we can explore some general way of removing kernels that are not in use.

weizhouapache · 2021-06-04T11:27:43Z

@weizhouapache I think it would be too much maintenance work for gain to maintain separate systemvmtemplate for each hypervisor. These days users can download 100MBs more in few extra seconds.

@rhtyd yes, agree. let's use same template for hypervisors.

rohityadavcloud · 2021-06-07T10:06:28Z

tools/appliance/systemvmtemplate/template.json

-        "scripts/apt_upgrade.sh",
+        "scripts/apt_upgrade.sh"
+      ],
+      "expect_disconnect": true


@weizhouapache this handles for reboot case. You can read more about it here https://www.packer.io/docs/provisioners/shell

@rhtyd good. thanks for the info.

rohityadavcloud · 2021-06-07T10:09:42Z

tools/appliance/systemvmtemplate/scripts/configure_grub.sh


 function configure_grub() {
+  # Remove old kernel
+  apt-get remove -y --purge linux-image-amd64 linux-image-4* || true


@weizhouapache would this help, essentially saying remove all the old 4.x kernels. As the new/LTS Linux kernel are in the 5.x series - https://www.kernel.org/category/releases.html

@rhtyd ok, I will test it.

rohityadavcloud · 2021-06-17T07:57:54Z

Is this good to be merged @weizhouapache @Pearl1594

weizhouapache

@rhtyd yes, it is good to me.

Will we wait for debian 11 which is planned to be released in July ?

rohityadavcloud · 2021-06-17T08:16:33Z

@weizhouapache yes we should update to Debian 11 if/when it comes before 4.16. In the meanwhile we can use the same Linux kernel backport same as Debian 11.

weizhouapache · 2021-06-17T08:24:08Z

@weizhouapache yes we should update to Debian 11 if/when it comes before 4.16. In the meanwhile we can use the same Linux kernel backport same as Debian 11.

@rhtyd ok, let's merge it when Pearl approves.

the codename of debian11 is 'Bullseye', cool.
bullseye-backports already exists, good.
http://ftp.debian.org/debian/dists/bullseye-backports/

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

rohityadavcloud · 2021-06-17T10:45:15Z

@blueorangutan package

blueorangutan · 2021-06-17T10:46:03Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-06-17T11:15:41Z

Packaging result: ✔️ centos7 ✔️ centos8 ✔️ debian. SL-JID 264

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

rohityadavcloud · 2021-06-17T11:24:24Z

tools/appliance/systemvmtemplate/scripts/install_systemvm_packages.sh

  [[ -f /bin/vhd-util ]] && return

-  wget --no-check-certificate https://github.com/shapeblue/cloudstack-nonoss/raw/master/vhd-util -O /bin/vhd-util
+  wget --no-check-certificate https://github.com/shapeblue/cloudstack-nonoss/raw/main/vhd-util -O /bin/vhd-util


jfyi @sureshanaparti - fixed this

noted, thanks @rhtyd

rohityadavcloud · 2021-06-17T14:10:45Z

@blueorangutan test matrix

blueorangutan · 2021-06-17T14:11:03Z

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

rohityadavcloud · 2021-06-17T15:17:51Z

@blueorangutan test matrix

blueorangutan · 2021-06-17T15:18:03Z

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

blueorangutan · 2021-06-18T04:20:49Z

Trillian test result (tid-985)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44329 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5073-t985-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 87 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	526.36	test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Error`	526.37	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	463.68	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Failure`	504.96	test_vpc_redundant.py

blueorangutan · 2021-06-18T06:29:57Z

Trillian test result (tid-988)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45596 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5073-t988-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 86 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_volume_usage	`Error`	82.34	test_usage.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	605.14	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	525.84	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Failure`	538.46	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Error`	538.48	test_vpc_redundant.py

weizhouapache · 2021-07-15T09:27:38Z

@rhtyd good. are you going to build new systemvm template and upload to download.cloudstack.org ?

rohityadavcloud · 2021-07-15T11:24:24Z

@weizhouapache the current 4.16 systemvmtemplate from this PR has been already on download.cloudstack.org for weeks now, all PRs being tested against that systemvmtemplate

weizhouapache · 2021-07-15T11:25:47Z

@weizhouapache the current 4.16 systemvmtemplate from this PR has been already on download.cloudstack.org for weeks now, all PRs being tested against that systemvmtemplate

@rhtyd nice, good to know. thanks

rohityadavcloud added the component:systemvm-template label Jun 3, 2021

rohityadavcloud added this to the 4.16.0.0 milestone Jun 3, 2021

rohityadavcloud mentioned this pull request Jun 3, 2021

systemvmtemplate: reduce template by using cloud kernel #5070

Closed

5 tasks

rohityadavcloud requested a review from weizhouapache June 3, 2021 17:35

weizhouapache reviewed Jun 4, 2021

View reviewed changes

rohityadavcloud commented Jun 7, 2021

View reviewed changes

Update configure_grub.sh

db3d1f7

rohityadavcloud commented Jun 7, 2021

View reviewed changes

rohityadavcloud requested a review from weizhouapache June 7, 2021 10:09

weizhouapache mentioned this pull request Jun 7, 2021

systemvm with overlayfs #5083

Closed

12 tasks

rohityadavcloud requested a review from Pearl1594 June 16, 2021 03:25

weizhouapache approved these changes Jun 17, 2021

View reviewed changes

rohityadavcloud added 2 commits June 17, 2021 15:58

Merge remote-tracking branch 'origin/main' into 416-systemvmtemplate

53e4408

changes

d66051a

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

fix failing build

4c8f726

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

rohityadavcloud commented Jun 17, 2021

View reviewed changes

rohityadavcloud added the Severity:BLOCKER label Jun 17, 2021

rohityadavcloud requested a review from sureshanaparti June 17, 2021 13:04

apache deleted a comment from blueorangutan Jun 17, 2021

Update install_systemvm_packages.sh

c905bd5

rohityadavcloud merged commit cc27c70 into apache:main Jul 15, 2021

systemvmtemplate: use latest LTS kernel from buster-ports #5073

systemvmtemplate: use latest LTS kernel from buster-ports #5073

Uh oh!

Conversation

rohityadavcloud commented Jun 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

weizhouapache commented Jun 4, 2021

Uh oh!

rohityadavcloud commented Jun 4, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

weizhouapache commented Jun 4, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rohityadavcloud commented Jun 17, 2021

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

rohityadavcloud commented Jun 17, 2021

Uh oh!

weizhouapache commented Jun 17, 2021

Uh oh!

rohityadavcloud commented Jun 17, 2021

Uh oh!

blueorangutan commented Jun 17, 2021

Uh oh!

blueorangutan commented Jun 17, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rohityadavcloud commented Jun 17, 2021

Uh oh!

blueorangutan commented Jun 17, 2021

Uh oh!

rohityadavcloud commented Jun 17, 2021

Uh oh!

blueorangutan commented Jun 17, 2021

Uh oh!

blueorangutan commented Jun 18, 2021

Uh oh!

blueorangutan commented Jun 18, 2021

Uh oh!

weizhouapache commented Jul 15, 2021

Uh oh!

rohityadavcloud commented Jul 15, 2021

Uh oh!

weizhouapache commented Jul 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

rohityadavcloud commented Jun 3, 2021 •

edited

Loading