Stable Release Update Certification Coverage for Real-time Ubuntu

Introduction

This document lists the coverage of regression tests performed for validating a Stable Release Update (SRU) on Real-time Ubuntu Certified systems. The test plan is not recommended for, nor will it be accepted for self-testing purposes.

For each test job, one of the following certification statuses is specified:

Blocking

Features that are required for certification. If any of the blocking tests fails, the certification will fail.

Non-blocking

Features that are tested but not mandatory for certification. Failure in non-blocking tests will not prevent certification. However, a note will be added to the certificate to inform potential customers or users.

Note

Only categories of hardware are tested and not specific types of hardware. For example, tests are run to verify USB controllers work, but the type of peripheral(s) used during those tests are not specified.

Coverage is flexible based on customer requirements (for example, if a device’s use cases don’t require LEDs, then LEDs can be non-blocking)

Full test descriptions can be found in Canonical certification site for partners:

https://certification.canonical.com

sru-rt

Note

The certification tests presented in this document are validated by Checkbox version 4.2.0.dev76.

Blocking

Bluetooth tests

The following test units are covered in this category:

bluetooth/detect-output

Store Bluetooth device information for reports.

Category ID:

bluetooth
Status:

Blocking
Purpose:

Automated test to store Bluetooth device information in the Checkbox report
After-suspend:

True
Plugin:

shell
bluetooth4/beacon_eddystone_url_interface

Test system can get beacon EddyStone URL advertisements on the {interface} adapter

Unit type:

template
Category ID:

bluetooth
Status:

Blocking
Description:

<missing description>
After-suspend:

True
From template:

bluetooth4/beacon_eddystone_url_interface
Template resource:

device
Template filter:

device.category == ‘BLUETOOTH’

Camera tests

The following test units are covered in this category:

camera/detect

This Automated test attempts to detect a camera.

Category ID:

camera
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

shell
camera/multiple-resolution-images_name

Webcam multiple resolution capture test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Blocking
After-suspend:

True
From template:

camera/multiple-resolution-images_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

CPU tests

The following test units are covered in this category:

cpu/cstates

Run C-States tests

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Uses the Firmware Test Suite (fwts) to test the power saving states of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
cpu/maxfreq_test

Test that the CPU can run at its max frequency

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.
Environment variable:

LD_LIBRARY_PATH, SNAP
User:

root
Plugin:

shell
cpu/offlining_test

Test offlining of each CPU core

Category ID:

cpu
Status:

Blocking
Purpose:

Attempts to offline each core in a multicore system.
After-suspend:

True
User:

root
Plugin:

shell
cpu/scaling_test

Test the CPU scaling capabilities

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use Firmware Test Suite (fwts cpufreq) to test the scaling capabilities of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
cpu/topology

Check CPU topology for accuracy between proc and sysfs

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.
After-suspend:

True
Plugin:

shell

Disk tests

The following test units are covered in this category:

disk/detect

Gathers information about each disk detected

Category ID:

disk
Status:

Blocking
Purpose:

Uses lsblk to gather information about each disk detected on the system under test.
After-suspend:

True
Plugin:

shell
disk/read_performance_name

Disk performance test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
After-suspend:

True
Environment variable:

DISK_MDADM_READ_PERF, DISK_NVME_READ_PERF, DISK_READ_PERF
From template:

disk/read_performance_name
Template resource:

device
Template filter:

device.category == ‘DISK’

Ethernet Device tests

The following test units are covered in this category:

ethernet/detect

Detect if at least one ethernet device is detected

Category ID:

ethernet
Status:

Blocking
Purpose:

Test to detect and return information about available network controllers on the system under test.
After-suspend:

True
Plugin:

shell

Graphics tests

The following test units are covered in this category:

graphics/VESA_drivers_not_in_use

Test that VESA drivers are not in use

Category ID:

graphics
Status:

Blocking
Purpose:

<missing purpose>
Description:

Check that VESA drivers are not in use
After-suspend:

True
Plugin:

shell

Memory tests

The following test units are covered in this category:

memory/info

Check the amount of memory reported by meminfo against DMI

Category ID:

memory
Status:

Blocking
Purpose:

This test checks the amount of memory which is reported in meminfo against the size of the memory modules detected by DMI.
User:

root
Plugin:

shell

Miscellaneous tests

The following test units are covered in this category:

miscellanea/grub_file_check

Check if the file core.efi exists to make sure shim and grub can be upgraded

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
miscellanea/oops

Run FWTS OOPS check

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (FWTS) oops tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell

Non-device specific networking tests

The following test units are covered in this category:

networking/gateway_ping

Verifies if the system can connect to the Internet.

Category ID:

networking
Status:

Blocking
Purpose:

Tests whether the system has a working Internet connection.
After-suspend:

True
User:

root
Plugin:

shell
networking/ntp

Test NTP server synchronization capability.

Category ID:

networking
Status:

Blocking
Purpose:

Test to see if we can sync local clock to an NTP server
After-suspend:

True
User:

root
Plugin:

shell

Optical Drive tests

The following test units are covered in this category:

optical/detect

Displays discovered optical drives

Category ID:

optical
Status:

Blocking
Purpose:

Detects optical drives (CD/DVD) attached to the system.
Plugin:

shell
optical/read-automated_name

Tests read functionality of optical drive {name}

Unit type:

template
Category ID:

optical
Status:

Blocking
After-suspend:

True
From template:

optical/read-automated_name
Template resource:

device
Template filter:

device.category == ‘CDROM’

Power Management tests

The following test units are covered in this category:

power-management/cold-reboot

Cold reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This test powers off the system and then powers it on using RTC
Environment variable:

COLD_REBOOT_DELAY, RTC_DEVICE_FILE
User:

root
Plugin:

shell
power-management/fwts_wakealarm

Executes ACPI Wakealarm test to validate functionality.

Category ID:

power-management
Status:

Blocking
Purpose:

Test ACPI Wakealarm (fwts wakealarm)
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
power-management/post-cold-reboot

Post cold reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the cold reboot
Plugin:

shell
power-management/post-warm-reboot

Post warm reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the warm reboot
Plugin:

shell
power-management/rtc

Test that RTC functions properly (if present)

Category ID:

power-management
Status:

Blocking
Purpose:

Verify that the Real-time clock (RTC) device functions properly, if present.
After-suspend:

True
Environment variable:

RTC_DEVICE_FILE
User:

root
Plugin:

shell
power-management/tickless_idle

Verify the tickless idle feature configuration in the kernel.

Category ID:

power-management
Status:

Blocking
Purpose:

Check to see if CONFIG_NO_HZ is set in the kernel (this is just a simple regression check)
After-suspend:

True
Plugin:

shell
power-management/warm-reboot

Warm reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This tests reboots the system using the reboot command
User:

root
Plugin:

shell

Suspend tests

The following test units are covered in this category:

suspend/audio_after_suspend_auto

Verify mixer settings consistency after system suspend.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that mixer settings after suspend are the same as before suspend.
Plugin:

shell
suspend/audio_before_suspend

Record mixer settings before suspending.

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/cpu_after_suspend_auto

Ensure all CPUs are online post-resumption.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that all CPUs are online after resuming.
Plugin:

shell
suspend/cpu_before_suspend

Verify that all the CPUs are online before suspending

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/memory_after_suspend_auto

Ensure all memory is accessible after waking from suspend mode.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that all memory is available after resuming from suspend.
Plugin:

shell
suspend/memory_before_suspend

Dumps memory info to a file for comparison after suspend

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/network_after_suspend_auto

Verify the network functionality after system resume.

Category ID:

suspend
Status:

Blocking
Purpose:

Test the network after resuming.
User:

root
Plugin:

shell
suspend/network_before_suspend

Record the current network before suspending.

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell

Touchpad tests

The following test units are covered in this category:

touchpad/detected-as-mouse

Check if the touchpad is recognized as a mouse by the system.

Category ID:

touchpad
Status:

Blocking
Purpose:

This test will check if your touchpad was detected as a mouse.
After-suspend:

True
Plugin:

shell

TPM 2.0 (Trusted Platform Module)

The following test units are covered in this category:

clevis-encrypt-tpm2/detect-ecc-capabilities

Ensure the TPM has required capabilities for clevis ECC test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/detect-rsa-capabilities

Ensure the TPM has required capabilities for clevis RSA test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/ecc

clevis encrypt/decrypt key ecc

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/precheck

clevis encrypt/decrypt precheck

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
clevis-encrypt-tpm2/rsa

clevis encrypt/decrypt key rsa

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell

USB tests

The following test units are covered in this category:

usb/detect

Display USB devices attached to SUT

Category ID:

usb
Status:

Blocking
Purpose:

Detects and shows USB devices attached to this system.
After-suspend:

True
Plugin:

shell

Wireless networking tests

The following test units are covered in this category:

wireless/wireless_connection_open_ac_nm_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_nm_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_nm_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_nm_interface

Connect to an unencrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_nm_interface

Connect to an unencrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_nm_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_nm_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_nm_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_nm_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_nm_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_nm_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_nm_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_scanning_interface

Test system can discover Wi-Fi networks on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_scanning_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Non-blocking

Audio tests

The following test units are covered in this category:

audio/alsa_info_attachment

Attach audio hardware data collection log to the results.

Category ID:

audio
Status:

Non-blocking
Purpose:

Attaches the audio hardware data collection log to the results.
Plugin:

attachment
audio/alsa_info_collect

Collect audio-related system information for simulation and detailed testing.

Category ID:

audio
Status:

Non-blocking
Purpose:

Collect audio-related system information. This data can be used to simulate this computer’s audio subsystem and perform more detailed tests under a controlled environment.
User:

root
Plugin:

shell
audio/alsa_record_playback_automated

Test playback and recording functionality on the default audio input and output.

Category ID:

audio
Status:

Non-blocking
Purpose:

Play back a sound on the default output and listen for it on the default input. This test is intended mostly for laptops, where the default output will be the internal speakers and the default input will be the internal microphone.
After-suspend:

True
Plugin:

shell
audio/detect_sinks

Ensure audio sinks are available for detection.

Category ID:

audio
Status:

Non-blocking
Purpose:

Test to detect if there are available sinks
After-suspend:

True
Plugin:

shell
audio/list_devices

Check if audio devices can be detected.

Category ID:

audio
Status:

Non-blocking
Purpose:

Test to detect audio devices
Plugin:

shell

Benchmarks tests

The following test units are covered in this category:

benchmarks/disk/hdparm-cache-read_name

Cached read timing benchmark of {name} using hdparm

Unit type:

template
Category ID:

benchmarks
Status:

Non-blocking
After-suspend:

True
From template:

benchmarks/disk/hdparm-cache-read_name
Template resource:

device
Template filter:

device.category == ‘DISK’
benchmarks/disk/hdparm-read_name

Raw read timing benchmark of {name} using hdparm

Unit type:

template
Category ID:

benchmarks
Status:

Non-blocking
After-suspend:

True
From template:

benchmarks/disk/hdparm-read_name
Template resource:

device
Template filter:

device.category == ‘DISK’

Bluetooth tests

The following test units are covered in this category:

bluetooth/zapper-a2dp

Check if the system can connect to a Bluetooth speaker using A2DP profile

Category ID:

bluetooth
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

ZAPPER_HOST
Plugin:

shell

Camera tests

The following test units are covered in this category:

camera/camera-quality-image_name

Attach the image used for the BRISQUE score for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Description:

This test will attach the image used for the BRISQUE score.
After-suspend:

True
From template:

camera/camera-quality-image_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’
camera/camera-quality_name

Webcam BRISQUE score for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
After-suspend:

True
From template:

camera/camera-quality_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’
camera/multiple-resolution-images-attachment_name

Attach an image from the multiple resolution images test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Description:

This test will attach one of the images used for the multiple resolution images test.
After-suspend:

True
From template:

camera/multiple-resolution-images-attachment_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

CPU tests

The following test units are covered in this category:

cpu/cstates_results.log

Attach C-States test log

Category ID:

cpu
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission.
Plugin:

attachment
cpu/maxfreq_test-log-attach

Attach CPU max frequency log

Category ID:

cpu
Status:

Non-blocking
Purpose:

Attaches the log generated by cpu/maxfreq_test to the results submission.
Plugin:

attachment
cpu/scaling_test-log-attach

Attach CPU scaling capabilities log

Category ID:

cpu
Status:

Non-blocking
Purpose:

Attaches the log generated by cpu/scaling_test to the results submission.
Plugin:

attachment

Disk tests

The following test units are covered in this category:

disk/apste_support_on_name

Check support for Autonomous Power State Transition on {name}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
Description:

<missing description>
After-suspend:

True
From template:

disk/apste_support_on_name
Template resource:

device
Template filter:

device.driver == ‘nvme’ and device.category == ‘OTHER’
disk/check-software-raid

Validate the configuration of software RAID devices are expected

Category ID:

disk
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Examine the system to detect Software RAID devices are created and the RAID mode are expected the SOFTWARE_RAID_LEVEL variable is needed for this tests. e.g. SOFTWARE_RAID_LEVEL=”raid0 raid1 raid5”
Environment variable:

SOFTWARE_RAID_LEVEL
User:

root
Plugin:

shell
thunderbolt3/storage-preinserted

Storage test on Thunderbolt 3 pre-inserted

Category ID:

disk
Status:

Non-blocking
Purpose:

This is an automated test which performs read/write operations on a preinserted Thunderbolt HDD
After-suspend:

True
User:

root
Plugin:

shell

Firmware tests

The following test units are covered in this category:

firmware/fwts_desktop_diagnosis

Run FWTS QA-concerned desktop-specific diagnosis tests.

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (fwts) QA-concerned desktop-specific diagnosis tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
firmware/fwts_desktop_diagnosis_results.log.gz

Attach FWTS desktop diagnosis log to submission

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission
Plugin:

attachment
firmware/fwts_dump_acpi_attachment.gz

Collect the ACPI tables dump from FWTS

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

attachment

Graphics tests

The following test units are covered in this category:

graphics/xorg-failsafe

Test that X is not in failsafe mode.

Category ID:

graphics
Status:

Non-blocking
Purpose:

Test that the X is not running in failsafe mode.
Plugin:

shell
graphics/xorg-process

Test that the X process is running.

Category ID:

graphics
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Test that the X process is running.
Plugin:

shell
graphics/xorg-version

Test Xorg version output.

Category ID:

graphics
Status:

Non-blocking
Purpose:

Test to output the Xorg version
Plugin:

shell

Informational tests

The following test units are covered in this category:

acpi_sleep_attachment

Attach the contents of /proc/acpi/sleep for further analysis.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the contents of /proc/acpi/sleep if it exists.
Plugin:

attachment
codecs_attachment

Attach a report of installed codecs for Intel HDA.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a report of installed codecs for Intel HDA
Plugin:

attachment
cpuinfo_attachment

Attach a copy of /proc/cpuinfo

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a report of CPU information
Plugin:

attachment
dkms_info_attachment

Attaches json dumps of installed dkms package information.

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches json dumps of installed dkms package information.
Plugin:

attachment
dmesg_attachment

Attach a copy of dmesg or the current dmesg buffer to the test results.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a copy of /var/log/dmesg or the current dmesg buffer to the test results
User:

root
Plugin:

attachment
dmi_attachment

Attach a copy of /sys/class/dmi/id/*

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches info on DMI
Plugin:

attachment
dmidecode_attachment

Attach output of dmidecode

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches dmidecode output
User:

root
Plugin:

attachment
efi_attachment

Attaches firmware version info

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the firmware version
User:

root
Plugin:

attachment
info/buildstamp

Attaches the buildstamp identifier for the OS

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the buildstamp identifier for the OS
Plugin:

attachment
info/disk_partitions

Attaches info about disk partitions

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches information about disk partitions
User:

root
Plugin:

attachment
info/hdparm_name.txt

Attaches info from hdparm about {name}

Unit type:

template
Category ID:

info
Status:

Non-blocking
Description:

<missing description>
From template:

info/hdparm_name.txt
Template resource:

device
Template filter:

device.category == ‘DISK’
info/secure-boot-check

Check secure boot state

Category ID:

info
Status:

Non-blocking
Purpose:

Output whether secure boot is enabled or disabled
Plugin:

attachment
info/systemd-analyze

System boot-up performance statistics

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
info/systemd-analyze-critical-chain

Print the tree of the time-critical chain of SystemD

Category ID:

info
Status:

Non-blocking
Purpose:

This job prints a tree of the time-critical chain of SystemD units.
Plugin:

shell
info/touchpad_driver

Gather touchpad name, driver name, and driver version information.

Category ID:

info
Status:

Non-blocking
Purpose:

Returns the name, driver name and driver version of any touchpad discovered on the system.
Plugin:

attachment
kernel_cmdline_attachment

Attach a copy of /proc/cmdline

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the kernel command line used to boot
Plugin:

attachment
lsmod_attachment

Attach a list of currently running kernel modules

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a list of the currently running kernel modules.
Plugin:

attachment
lspci_attachment

Attach a list of PCI devices

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches very verbose lspci output.
Plugin:

attachment
lspci_standard_config_attachment

Attach PCI configuration space hex dump

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a hex dump of the standard part of the PCI configuration space for all PCI devices.
Plugin:

attachment
lstopo_visual_attachment

Attach the output of lstopo command to present system topology.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the system topology as presented by the lstopo command
Plugin:

attachment
lsusb_attachment

Attach output of lsusb

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a list of detected USB devices.
After-suspend:

True
User:

root
Plugin:

attachment
manifest

Hardware Manifest

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

This job loads the hardware manifest and exposes it as a resource.
Plugin:

resource
meminfo_attachment

Attach copy of /proc/meminfo

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches info on system memory as seen in /proc/meminfo.
Plugin:

attachment
modinfo_attachment

Attach modinfo information

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches modinfo information for all currently loaded modules
Plugin:

attachment
modprobe_attachment

Attach the contents of /etc/modprobe.*

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of the various modprobe conf files.
User:

root
Plugin:

attachment
modules_attachment

Attach the contents of /etc/modules

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of the /etc/modules file.
Plugin:

attachment
net_if_management_attachment

Collect logging from the net_if_management job

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Allows logging of debug info and errors by the associated resource job
Plugin:

attachment
recovery_info_attachment

Attach the recovery partition versions

Category ID:

info
Status:

Non-blocking
Purpose:

Attach the recovery partition version image_version is the preinstalled OS image version bto_version is only for dell_recovery Example: image_version: somerville-trusty-amd64-20140620-0 bto_version: A00_dell-bto-trusty-houston-15-A11-iso-20141203-0.iso
User:

root
Plugin:

attachment
sysctl_attachment

Attach sysctl configuration files.

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of various sysctl config files.
User:

root
Plugin:

attachment
sysfs_attachment

Attach detailed sysfs property output from udev

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches a report of sysfs attributes.
Plugin:

attachment
udev_attachment

Attach dump of udev database

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches a dump of the udev database showing system hardware information.
Plugin:

attachment

Input Devices tests

The following test units are covered in this category:

input/fixed_screen_orientation_on_product___index__

Check whether screen orientation is fixed on {product}

Unit type:

template
Category ID:

input
Status:

Non-blocking
After-suspend:

True
From template:

input/fixed_screen_orientation_on_product___index__
Template resource:

dmi
Template filter:

dmi.category == ‘CHASSIS’ and dmi.product not in (‘Tablet’, ‘Convertible’, ‘Detachable’)
input/zapper-keyboard

Check if the system recognizes keyboard events from Zapper

Category ID:

input
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

ZAPPER_HOST
User:

root
Plugin:

shell

Media Card tests

The following test units are covered in this category:

Miscellaneous tests

The following test units are covered in this category:

install/apt-get-gets-updates

Ensure apt can access repositories and get updates without installing them, to aid in recovery from broken updates.

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Tests to see that apt can access repositories and get updates (does not install updates). This is done to confirm that you could recover from an incomplete or broken update.
User:

root
Plugin:

shell
miscellanea/dkms_build_validation

Validate the build status of DKMS modules, automatically

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Firstly, check the built number of DKMS modules. Secondly, scan APT system update log to see any build error messages. This test is mainly used for pre-installed images which use DKMS modules.
Plugin:

shell
miscellanea/oops_results.log

Attach the FWTS oops results for submission.

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Attaches the FWTS oops results log to the submission
Plugin:

attachment
miscellanea/submission-resources

Check that data for a complete result are present

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

A meta-job that verifies the data necessary for a complete result submission are present. Failure indicates that the results are incomplete and may be rejected.
Plugin:

shell

Mobile broadband tests

The following test units are covered in this category:

mobilebroadband/cdma_connection

Ensure a CDMA based modem establishes and maintains a mobile broadband connection.

Category ID:

mobilebroadband
Status:

Non-blocking
Purpose:

Creates a mobile broadband connection for a CDMA based modem and checks the connection to ensure it’s working.
After-suspend:

True
Environment variable:

CDMA_CONN_NAME, CDMA_PASSWORD, CDMA_USERNAME
User:

root
Plugin:

shell
mobilebroadband/gsm_connection

Establish and verify GSM mobile broadband connection.

Category ID:

mobilebroadband
Status:

Non-blocking
Purpose:

Creates a mobile broadband connection for a GSM based modem and checks the connection to ensure it’s working.
After-suspend:

True
Environment variable:

GSM_APN, GSM_CONN_NAME, GSM_PASSWORD, GSM_USERNAME
User:

root
Plugin:

shell

Monitor tests

The following test units are covered in this category:

monitor/zapper-edid

Check if the system automatically changes the resolution based on EDID

Category ID:

monitor
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

ZAPPER_HOST
Plugin:

shell

Non-device specific networking tests

The following test units are covered in this category:

networking/http

Ensure downloading files through HTTP works correctly.

Category ID:

networking
Status:

Non-blocking
Purpose:

Automated test case to make sure that it’s possible to download files through HTTP
Environment variable:

TRANSFER_SERVER
Plugin:

shell

Power Management tests

The following test units are covered in this category:

power-management/fwts_wakealarm-log-attach

Attach and display fwts wakealarm test log.

Category ID:

power-management
Status:

Non-blocking
Purpose:

Attach log from fwts wakealarm test
Plugin:

attachment

Real Time Performance Tests

The following test units are covered in this category:

rt-performance-tests/cyclictest_rt

A short-duration cyclictest to validate the system’s real-time capabilities.

Category ID:

rt-performance-tests
Status:

Non-blocking
Purpose:

This test runs for 60 seconds to provide a rapid assessment of the system’s real-time performance. It uses the cyclictest tool to measure the scheduling latency and overflow occurrences. The goal is to ensure that basic real-time functions can be executed without latency spikes.
User:

root
Plugin:

shell

Touchpad tests

The following test units are covered in this category:

touchpad/palm-rejection-firmware-labeling_product_slug

Check palm rejection firmware/labeling for touchpads

Unit type:

template
Category ID:

touchpad
Status:

Non-blocking
After-suspend:

True
From template:

touchpad/palm-rejection-firmware-labeling_product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHPAD’ and device.driver == ‘hid-multitouch’

TPM 2.0 (Trusted Platform Module)

The following test units are covered in this category:

tpm2/fwts-event-log-dump

Dump the contents of the TPM Event Log

Category ID:

tpm2
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

The information in the TPM Event Log can be useful in debugging problems with TPM command support and adherance to standards. This can be of particular help when checking whether a device can support Ubuntu Core Full Disk Encryption.
User:

root
Plugin:

attachment

USB tests

The following test units are covered in this category:

usb/storage-detect

Detect storage partitions on a device on the USB bus

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
usb/zapper-usb-insert-port_alias

Zapper-driven USB insertion test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Non-blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb/zapper-usb-insert-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’
usb/zapper-usb-remove-port_alias

Zapper-driven USB removal test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Non-blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb/zapper-usb-remove-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’
usb3/zapper-usb-insert-port_alias

Zapper-driven USB 3.0 insertion test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Non-blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb3/zapper-usb-insert-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’
usb3/zapper-usb-remove-port_alias

Zapper-driven USB 3.0 removal test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Non-blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb3/zapper-usb-remove-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’

Wireless networking tests

The following test units are covered in this category:

wireless/check_iwlwifi_microcode_crash_interface

Check there have been no iwlwifi crashes

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
After-suspend:

True
From template:

wireless/check_iwlwifi_microcode_crash_interface
Template resource:

device
Template filter:

device.driver == ‘iwlwifi’
wireless/detect

Detect if at least one Wireless LAN device is detected

Category ID:

wireless
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
wireless/nm_connection_restore_interface

Restore any NetworkManager 802.11 configurations after testing

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
Description:

<missing description>
After-suspend:

True
From template:

wireless/nm_connection_restore_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/nm_connection_save_interface

Save any NetworkManager 802.11 configurations prior to testing

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
Description:

<missing description>
After-suspend:

True
From template:

wireless/nm_connection_save_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Appendix A. FWTS tests

As part of the certification process, we run a series of firmware tests that are part of the Canonical Firmware Test Suite. In general, any HIGH or CRITICAL error found in the fwts log can cause potential errors in the system and should be looked at by OEMs/ODMs.

Category

Test Item

Description

Information

acpidump

Check ACPI table acpidump.

Information

version

Gather kernel system information.

ACPI

acpitables

ACPI table settings confidence checks.

ACPI

apicinstance

Check for single instance of APIC/MADT table.

ACPI

hpet_check

High Precision Event Timer configuration test.

ACPI

mcfg

MCFG PCI Express* memory mapped config space.

ACPI

method

ACPI DSDT Method Semantic Tests.

CPU

mpcheck

Check Multi Processor tables.

CPU

msr

CPU MSR consistency check.

CPU

mtrr

MTRR validation.

System

apicedge

APIC Edge/Level Check.

System

klog

Scan kernel log for errors and warnings.