====================================================== Ubuntu Server Hardware Certification Test Case Guide ====================================================== .. include:: .. header:: |ubuntu_logo| .. |ubuntu_logo| image:: images/logo-ubuntu_su-white_orange-hex.png :scale: 20% .. footer:: |canonical_logo| .. |canonical_logo| image:: images/logo-canonical_no-tm-white-hex.png :scale: 10% Introduction ============ This guide describes the jobs performed in Ubuntu server certification testing. A job can be anything executed by Checkbox, typically these are either information gathering jobs or test cases. Test cases follow the format Category/TestName such as `ethernet/detect`. Some job names may simply be a TestName without the category designator. These are typically jobs that gather and/or attach hardware info for the submission file. Jobs are grouped into three categories: mandatory_include Items that Checkbox *will* run every time. These jobs can not be skipped. include Items that Checkbox *may* run. These jobs can be skipped depending on the presence or absence of certain components, or software. bootstrap_include Items that Checkbox will run before the final test list is created. These jobs accomplish tasks such as gathering initial system information that is used to determine which test cases are applicable to the SUT. This guide is based on the `server-full-20.04.pxu` list used for full Server Certification. Other lists in the `canonical-certification-server` UI are either subsets of this list, or not applicable to 20.04 certification, such as the 18.04 lists. Tests ===== bootstrap include: ------------------ device No description available fwts No description available include: -------- benchmarks/disk/hdparm-cache-read_{name} This test runs hdparm timing of cache reads as a benchmark for {name} benchmarks/disk/hdparm-read_{name} This test runs hdparm timing of device reads as a benchmark for {name} cpu/clocktest Runs a test for clock jitter on SMP machines. cpu/frequency_governors This test checks that CPU frequency governors are obeyed when set. cpu/maxfreq_test Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency. cpu/maxfreq_test-log-attach Attaches the log generated by cpu/maxfreq_test to the results submission. cpu/topology Parses information about CPU topology provided by proc and sysfs and checks that they are consistent. disk/detect Uses lsblk to gather information about each disk detected on the system under test. disk/disk_cpu_load_{name} Check to ensure CPU load of {product_slug} is not too high disk/disk_stress_ng_{name} Disk stress_ng test for {product_slug} disk/read_performance_{name} Verify that disk storage performs at or above baseline performance disk/smart_{name} This tests the SMART capabilities for {product_slug} (Note that this test may not work against hardware RAID) disk/stats_{name} This test checks disk stats, generates some activity and rechecks stats to verify they've changed. It also verifies that disks appear in the various files they're supposed to. info/hdparm_{name}.txt Attaches the bootchart png file for bootchart runs info/kvm_output Attaches the debug log from the virtualization/kvm_check_vm test to the results submission. mediacard/sd-preinserted-server This is a fully automated version of mediacard/sd-automated and assumes that the system under test has a memory card device plugged in prior to checkbox execution. memory/info This test checks the amount of memory which is reporting in meminfo against the size of the memory modules detected by DMI. memory/memory_stress_ng Test to perform some basic stress and exercise of system memory via the stress_ng tool. This test also includes an over-commit function to force swapping to disk, thus SUTs should have suitably large swap files for the amount of RAM they have installed. miscellanea/klog Run Firmware Test Suite (fwts) klog tests. miscellanea/klog_results.log Attaches the FWTS klog results log to the submission miscellanea/olog Run Firmware Test Suite (fwts) olog tests (ppc64el only). miscellanea/olog_results.log Attaches the FWTS olog results log to the submission miscellanea/oops Run Firmware Test Suite (fwts) oops tests. miscellanea/oops_results.log Attaches the FWTS oops results log to the submission networking/predictable_names Verify that all network interfaces have predictable names. optical/detect Detects optical drives (CD/DVD) attached to the system. optical/read-automated_{name} This is an automated version of optical/read. It assumes you have already inserted a data CD into your optical drive prior to running Checkbox. power-management/rtc Verify that the Real-time clock (RTC) device functions properly, if present. stress/cpu_stress_ng_test Impose a high system load using the 'stress_ng' tool to exercise the CPU for several hours. The test is considered passed if the system does not freeze and if the stress_ng tool does not report errors. usb/detect Detects and shows USB devices attached to this system. usb/storage-preinserted Tests USB 2.0 or 1.1 ports on a system by doing write/read/compare tests on randomly created data. It requires that a USB stick is plugged into an available USB port before running the certification suite. usb3/storage-preinserted Tests USB 3.0 ports on a system by doing write/read/compare tests on randomly created data. It requires that a USB stick is plugged into an available USB port before running the certification suite. Additionally, it will only work with USB sticks and ports rated for USB 3.0 speeds or faster. virtualization/kvm_check_vm Verifies that a KVM guest can be created and booted using an Ubuntu Server cloud image. virtualization/verify_lxd Verifies that an LXD container can be created and launched