-
Notifications
You must be signed in to change notification settings - Fork 72
Adjust the wwan, watchdog and alsa-loopback test cases. (BugFix) #821
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Actually, the alsa-loopback-automated test case does the same thing with alsa-loopback. When we test iot project machines, the alsa-loopback-automated needs to be tested. I guess we could do the alsa-loopback-automated only in the futuer.
The watchdog-automated's test items have included watchdog-maunal, so I remove the watchdog-manual test case.
|
Please add a description to this, I have no idea what this is trying to do nor why it is removing those tests |
Hi Massimiliano, I updated my description. I am sorry for my bad. |
pieqq
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I rarely see jobs being removed, but I agree with your analysis: these manual jobs have been superseded by automated jobs, so there is no point in running them!
I left a few remarks inline.
Also, I think it would be wise if other QA engineers review these changes. I will add them.
64ea166 to
6e79354
Compare
The mobilebroadband-cert-automated test case is only part of the wwan test plan. We hope to replace mobilebroadband-cert-automated with wwan-automated. The relevant test plans we currently use include: client-cert-desktop-22-04-automated client-cert-desktop-20-04-automated client-cert-desktop-18-04-automated I have replaced mobilebroadband-cert-automated in them with wwan-automated, and added after-sudpend-wwan- automated to ensure that wwan is tested before and after suspend. In addition, for mobilebroadband-cert-manual in the Manual test plan, when the SIM is inserted into the prototype in the desktop, the system should automatically recognize wwan and SIM card, and we do not need to perform manual testing. So I deleted mobilebroadband-cert-manual in the relevant test plans. At the same time, since wwan-automated already contains wwan-manual cases, I deleted the wwan-manual in these test plans: client-cert-desktop-22-04-manual client-cert-desktop-20-04-manual client-cert-desktop-18-04-manual
6e79354 to
63d0a93
Compare
|
Following Pierre's suggestion, I adjusted the relevant commits. Please review again. Thanks |
zongminl
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some changes would have to be done in wwan/scan-networks-manual, wwan/gsm-connection-interrupted-manual and their corresponding after-suspend jobs, for the reason they still depend on the removed job wwan/check-sim-present-manual
of test cases.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #821 +/- ##
==========================================
+ Coverage 35.25% 35.66% +0.40%
==========================================
Files 302 302
Lines 34173 34233 +60
Branches 5907 5915 +8
==========================================
+ Hits 12048 12208 +160
+ Misses 21559 21459 -100
Partials 566 566
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
/canonical/self-hosted-runners/run-workflows 82bd889 |
…onical#821) * The commit is for the alsa-loopback test case delete. Actually, the alsa-loopback-automated test case does the same thing with alsa-loopback. When we test iot project machines, the alsa-loopback-automated needs to be tested. I guess we could do the alsa-loopback-automated only in the futuer. * This commit to modify the watchdog test cases. The watchdog-automated's test items have included watchdog-maunal, so I remove the watchdog-manual test case. * Modify the wwan related test cases. The mobilebroadband-cert-automated test case is only part of the wwan test plan. We hope to replace mobilebroadband-cert-automated with wwan-automated. The relevant test plans we currently use include: client-cert-desktop-22-04-automated client-cert-desktop-20-04-automated client-cert-desktop-18-04-automated I have replaced mobilebroadband-cert-automated in them with wwan-automated, and added after-sudpend-wwan- automated to ensure that wwan is tested before and after suspend. In addition, for mobilebroadband-cert-manual in the Manual test plan, when the SIM is inserted into the prototype in the desktop, the system should automatically recognize wwan and SIM card, and we do not need to perform manual testing. So I deleted mobilebroadband-cert-manual in the relevant test plans. At the same time, since wwan-automated already contains wwan-manual cases, I deleted the wwan-manual in these test plans: client-cert-desktop-22-04-manual client-cert-desktop-20-04-manual client-cert-desktop-18-04-manual * Revert wwan test-plan.pxu. Keep the necessary dependence of test cases.
…onical#821) * The commit is for the alsa-loopback test case delete. Actually, the alsa-loopback-automated test case does the same thing with alsa-loopback. When we test iot project machines, the alsa-loopback-automated needs to be tested. I guess we could do the alsa-loopback-automated only in the futuer. * This commit to modify the watchdog test cases. The watchdog-automated's test items have included watchdog-maunal, so I remove the watchdog-manual test case. * Modify the wwan related test cases. The mobilebroadband-cert-automated test case is only part of the wwan test plan. We hope to replace mobilebroadband-cert-automated with wwan-automated. The relevant test plans we currently use include: client-cert-desktop-22-04-automated client-cert-desktop-20-04-automated client-cert-desktop-18-04-automated I have replaced mobilebroadband-cert-automated in them with wwan-automated, and added after-sudpend-wwan- automated to ensure that wwan is tested before and after suspend. In addition, for mobilebroadband-cert-manual in the Manual test plan, when the SIM is inserted into the prototype in the desktop, the system should automatically recognize wwan and SIM card, and we do not need to perform manual testing. So I deleted mobilebroadband-cert-manual in the relevant test plans. At the same time, since wwan-automated already contains wwan-manual cases, I deleted the wwan-manual in these test plans: client-cert-desktop-22-04-manual client-cert-desktop-20-04-manual client-cert-desktop-18-04-manual * Revert wwan test-plan.pxu. Keep the necessary dependence of test cases.
…onical#821) * The commit is for the alsa-loopback test case delete. Actually, the alsa-loopback-automated test case does the same thing with alsa-loopback. When we test iot project machines, the alsa-loopback-automated needs to be tested. I guess we could do the alsa-loopback-automated only in the futuer. * This commit to modify the watchdog test cases. The watchdog-automated's test items have included watchdog-maunal, so I remove the watchdog-manual test case. * Modify the wwan related test cases. The mobilebroadband-cert-automated test case is only part of the wwan test plan. We hope to replace mobilebroadband-cert-automated with wwan-automated. The relevant test plans we currently use include: client-cert-desktop-22-04-automated client-cert-desktop-20-04-automated client-cert-desktop-18-04-automated I have replaced mobilebroadband-cert-automated in them with wwan-automated, and added after-sudpend-wwan- automated to ensure that wwan is tested before and after suspend. In addition, for mobilebroadband-cert-manual in the Manual test plan, when the SIM is inserted into the prototype in the desktop, the system should automatically recognize wwan and SIM card, and we do not need to perform manual testing. So I deleted mobilebroadband-cert-manual in the relevant test plans. At the same time, since wwan-automated already contains wwan-manual cases, I deleted the wwan-manual in these test plans: client-cert-desktop-22-04-manual client-cert-desktop-20-04-manual client-cert-desktop-18-04-manual * Revert wwan test-plan.pxu. Keep the necessary dependence of test cases.
Following QA changes in #821¹ regarding the replacement of mobilebroadband-cert-.* nested part for their wwan-.* counterparts, the same is applied for: - client-cert-desktop-24-04 (it was missed in #821 due to bad timing) - client-cert-odm-desktop-2?-04 (to match what's being tested by the QA team) client-cert-odm-desktop-18-04 was not included due to the fact that Ubuntu 18.04 has seen EOL and therefore its related test plan is not actively maintained anymore. ¹ #821
Replace mobilebroadband-cert-automated with wwan-automated nested part, as the wwan-automated test plan includes better coverage for WWAN-related tests. Add after-suspend-wwan-automated counterpart to make sure there are no issues with WWAN after resuming from suspend. This is a follow-up to work done by QA in the certification-client desktop test plans (see #821) Fix CER-2738
…(bugfix) (#1496) * Switch from mobilebroadband to wwan nested parts for cert test plans Following QA changes in #821¹ regarding the replacement of mobilebroadband-cert-.* nested part for their wwan-.* counterparts, the same is applied for: - client-cert-desktop-24-04 (it was missed in #821 due to bad timing) - client-cert-odm-desktop-2?-04 (to match what's being tested by the QA team) client-cert-odm-desktop-18-04 was not included due to the fact that Ubuntu 18.04 has seen EOL and therefore its related test plan is not actively maintained anymore. ¹ #821 * Add after-suspend-wwan-manual for homogeneity * Remove superseded jobs from [after-suspend-]wwan-manual nested parts wwan/detect-manual and wwan/check-sim-present-manual have an automated version available. wwan/gsm-connection-interrupted-manual depends on the manual versions of the detection and SIM verification jobs, and is therefore removed as well, as we have an automated connection job.
Replace mobilebroadband-cert-automated with wwan-automated nested part, as the wwan-automated test plan includes better coverage for WWAN-related tests. Add after-suspend-wwan-automated counterpart to make sure there are no issues with WWAN after resuming from suspend. This is a follow-up to work done by QA in the certification-client desktop test plans (see #821) Fix CER-2738
Use wwan-automated nested part in SRU test plan Replace mobilebroadband-cert-automated with wwan-automated nested part, as the wwan-automated test plan includes better coverage for WWAN-related tests. Add after-suspend-wwan-automated counterpart to make sure there are no issues with WWAN after resuming from suspend. This is a follow-up to work done by QA in the certification-client desktop test plans (see #821) Fix CER-2738
Uh oh!
There was an error while loading. Please reload this page.