Ubuntu Server Hardware Certification Test Case Guide¶
Introduction¶
This guide describes the jobs performed in Ubuntu server certification testing. A job can be anything executed by Checkbox, typically these are either information gathering jobs or test cases.
Test cases follow the format Category/TestName such as ethernet/detect. Some job names may simply be a TestName without the category designator. These are typically jobs that gather and/or attach hardware info for the submission file.
Jobs are grouped into three categories:
- mandatory_include
Items that Checkbox will run every time. These jobs can not be skipped.
- include
Items that Checkbox may run. These jobs can be skipped depending on the presence or absence of certain components, or software.
- bootstrap_include
Items that Checkbox will run before the final test list is created. These jobs accomplish tasks such as gathering initial system information that is used to determine which test cases are applicable to the SUT.
This guide is based on the server-full-20.04.pxu list used for full Server Certification. Other lists in the canonical-certification-server UI are either subsets of this list, or not applicable to 20.04 certification, such as the 18.04 lists.
Tests¶
bootstrap include:¶
- device
No description available
- fwts
No description available
include:¶
- benchmarks/disk/hdparm-cache-read_{name}
This test runs hdparm timing of cache reads as a benchmark for {name}
- benchmarks/disk/hdparm-read_{name}
This test runs hdparm timing of device reads as a benchmark for {name}
- cpu/clocktest
Runs a test for clock jitter on SMP machines.
- cpu/frequency_governors
This test checks that CPU frequency governors are obeyed when set.
- cpu/maxfreq_test
Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.
- cpu/maxfreq_test-log-attach
Attaches the log generated by cpu/maxfreq_test to the results submission.
- cpu/topology
Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.
- disk/detect
Uses lsblk to gather information about each disk detected on the system under test.
- disk/disk_cpu_load_{name}
Check to ensure CPU load of {product_slug} is not too high
- disk/disk_stress_ng_{name}
Disk stress_ng test for {product_slug}
- disk/read_performance_{name}
Verify that disk storage performs at or above baseline performance
- disk/smart_{name}
This tests the SMART capabilities for {product_slug} (Note that this test may not work against hardware RAID)
- disk/stats_{name}
This test checks disk stats, generates some activity and rechecks stats to verify they’ve changed. It also verifies that disks appear in the various files they’re supposed to.
- info/hdparm_{name}.txt
Attaches the bootchart png file for bootchart runs
- info/kvm_output
Attaches the debug log from the virtualization/kvm_check_vm test to the results submission.
- mediacard/sd-preinserted-server
This is a fully automated version of mediacard/sd-automated and assumes that the system under test has a memory card device plugged in prior to checkbox execution.
- memory/info
This test checks the amount of memory which is reporting in meminfo against the size of the memory modules detected by DMI.
- memory/memory_stress_ng
Test to perform some basic stress and exercise of system memory via the stress_ng tool. This test also includes an over-commit function to force swapping to disk, thus SUTs should have suitably large swap files for the amount of RAM they have installed.
- miscellanea/klog
Run Firmware Test Suite (fwts) klog tests.
- miscellanea/klog_results.log
Attaches the FWTS klog results log to the submission
- miscellanea/olog
Run Firmware Test Suite (fwts) olog tests (ppc64el only).
- miscellanea/olog_results.log
Attaches the FWTS olog results log to the submission
- miscellanea/oops
Run Firmware Test Suite (fwts) oops tests.
- miscellanea/oops_results.log
Attaches the FWTS oops results log to the submission
- networking/predictable_names
Verify that all network interfaces have predictable names.
- optical/detect
Detects optical drives (CD/DVD) attached to the system.
- optical/read-automated_{name}
This is an automated version of optical/read. It assumes you have already inserted a data CD into your optical drive prior to running Checkbox.
- power-management/rtc
Verify that the Real-time clock (RTC) device functions properly, if present.
- stress/cpu_stress_ng_test
Impose a high system load using the ‘stress_ng’ tool to exercise the CPU for several hours. The test is considered passed if the system does not freeze and if the stress_ng tool does not report errors.
- usb/detect
Detects and shows USB devices attached to this system.
- usb/storage-preinserted
Tests USB 2.0 or 1.1 ports on a system by doing write/read/compare tests on randomly created data. It requires that a USB stick is plugged into an available USB port before running the certification suite.
- usb3/storage-preinserted
Tests USB 3.0 ports on a system by doing write/read/compare tests on randomly created data. It requires that a USB stick is plugged into an available USB port before running the certification suite. Additionally, it will only work with USB sticks and ports rated for USB 3.0 speeds or faster.
- virtualization/kvm_check_vm
Verifies that a KVM guest can be created and booted using an Ubuntu Server cloud image.
- virtualization/verify_lxd
Verifies that an LXD container can be created and launched