Stable Release Update Certification Coverage for Real-time Ubuntu

Introduction

This document lists the coverage of regression tests performed for validating a Stable Release Update (SRU) on Real-time Ubuntu Certified systems. The test plan is not recommended for, nor will it be accepted for self-testing purposes.

For each test job, one of the following certification statuses is specified:

Blocking

Features that are required for certification. If any of the blocking tests fails, the certification will fail.

Non-blocking

Features that are tested but not mandatory for certification. Failure in non-blocking tests will not prevent certification. However, a note will be added to the certificate to inform potential customers or users.

Note

Only categories of hardware are tested and not specific types of hardware. For example, tests are run to verify USB controllers work, but the type of peripheral(s) used during those tests are not specified.

Coverage is flexible based on customer requirements (for example, if a device’s use cases don’t require LEDs, then LEDs can be non-blocking)

Full test descriptions can be found in Canonical certification site for partners:

https://certification.canonical.com

sru-rt

Note

The certification tests presented in this document are validated by Checkbox version 4.2.0.dev76.

Blocking

Bluetooth tests

The following test units are covered in this category:

bluetooth/detect-output

Store Bluetooth device information for reports.

Category ID:

bluetooth

Status:

Blocking

Purpose:

Automated test to store Bluetooth device information in the Checkbox report

After-suspend:

True

Plugin:

shell

bluetooth4/beacon_eddystone_url_interface

Test system can get beacon EddyStone URL advertisements on the {interface} adapter

Unit type:

template

Category ID:

bluetooth

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

bluetooth4/beacon_eddystone_url_interface

Template resource:

device

Template filter:

device.category == ‘BLUETOOTH’

Camera tests

The following test units are covered in this category:

camera/detect

This Automated test attempts to detect a camera.

Category ID:

camera

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

User:

root

Plugin:

shell

camera/multiple-resolution-images_name

Webcam multiple resolution capture test for {product_slug}

Unit type:

template

Category ID:

camera

Status:

Blocking

After-suspend:

True

From template:

camera/multiple-resolution-images_name

Template resource:

device

Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

CPU tests

The following test units are covered in this category:

cpu/cstates

Run C-States tests

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Uses the Firmware Test Suite (fwts) to test the power saving states of the CPU.

Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP

User:

root

Plugin:

shell

cpu/maxfreq_test

Test that the CPU can run at its max frequency

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.

Environment variable:

LD_LIBRARY_PATH, SNAP

User:

root

Plugin:

shell

cpu/offlining_test

Test offlining of each CPU core

Category ID:

cpu

Status:

Blocking

Purpose:

Attempts to offline each core in a multicore system.

After-suspend:

True

User:

root

Plugin:

shell

cpu/scaling_test

Test the CPU scaling capabilities

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Use Firmware Test Suite (fwts cpufreq) to test the scaling capabilities of the CPU.

Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP

User:

root

Plugin:

shell

cpu/topology

Check CPU topology for accuracy between proc and sysfs

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.

After-suspend:

True

Plugin:

shell

Disk tests

The following test units are covered in this category:

disk/detect

Gathers information about each disk detected

Category ID:

disk

Status:

Blocking

Purpose:

Uses lsblk to gather information about each disk detected on the system under test.

After-suspend:

True

Plugin:

shell

disk/read_performance_name

Disk performance test for {product_slug}

Unit type:

template

Category ID:

disk

Status:

Blocking

After-suspend:

True

Environment variable:

DISK_MDADM_READ_PERF, DISK_NVME_READ_PERF, DISK_READ_PERF

From template:

disk/read_performance_name

Template resource:

device

Template filter:

device.category == ‘DISK’

Ethernet Device tests

The following test units are covered in this category:

ethernet/detect

Detect if at least one ethernet device is detected

Category ID:

ethernet

Status:

Blocking

Purpose:

Test to detect and return information about available network controllers on the system under test.

After-suspend:

True

Plugin:

shell

Graphics tests

The following test units are covered in this category:

graphics/VESA_drivers_not_in_use

Test that VESA drivers are not in use

Category ID:

graphics

Status:

Blocking

Purpose:

<missing purpose>

Description:

Check that VESA drivers are not in use

After-suspend:

True

Plugin:

shell

Memory tests

The following test units are covered in this category:

memory/info

Check the amount of memory reported by meminfo against DMI

Category ID:

memory

Status:

Blocking

Purpose:

This test checks the amount of memory which is reported in meminfo against the size of the memory modules detected by DMI.

User:

root

Plugin:

shell

Miscellaneous tests

The following test units are covered in this category:

miscellanea/grub_file_check

Check if the file core.efi exists to make sure shim and grub can be upgraded

Category ID:

miscellanea

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

miscellanea/oops

Run FWTS OOPS check

Category ID:

miscellanea

Status:

Blocking

Purpose:

<missing purpose>

Description:

Run Firmware Test Suite (FWTS) oops tests.

Environment variable:

PLAINBOX_SESSION_SHARE

User:

root

Plugin:

shell

Non-device specific networking tests

The following test units are covered in this category:

networking/gateway_ping

Verifies if the system can connect to the Internet.

Category ID:

networking

Status:

Blocking

Purpose:

Tests whether the system has a working Internet connection.

After-suspend:

True

User:

root

Plugin:

shell

networking/ntp

Test NTP server synchronization capability.

Category ID:

networking

Status:

Blocking

Purpose:

Test to see if we can sync local clock to an NTP server

After-suspend:

True

User:

root

Plugin:

shell

Optical Drive tests

The following test units are covered in this category:

optical/detect

Displays discovered optical drives

Category ID:

optical

Status:

Blocking

Purpose:

Detects optical drives (CD/DVD) attached to the system.

Plugin:

shell

optical/read-automated_name

Tests read functionality of optical drive {name}

Unit type:

template

Category ID:

optical

Status:

Blocking

After-suspend:

True

From template:

optical/read-automated_name

Template resource:

device

Template filter:

device.category == ‘CDROM’

Power Management tests

The following test units are covered in this category:

power-management/cold-reboot

Cold reboot

Category ID:

power-management

Status:

Blocking

Purpose:

This test powers off the system and then powers it on using RTC

Environment variable:

COLD_REBOOT_DELAY, RTC_DEVICE_FILE

User:

root

Plugin:

shell

power-management/fwts_wakealarm

Executes ACPI Wakealarm test to validate functionality.

Category ID:

power-management

Status:

Blocking

Purpose:

Test ACPI Wakealarm (fwts wakealarm)

Environment variable:

PLAINBOX_SESSION_SHARE

User:

root

Plugin:

shell

power-management/post-cold-reboot

Post cold reboot service check

Category ID:

power-management

Status:

Blocking

Purpose:

Check there are no failed services after the cold reboot

Plugin:

shell

power-management/post-warm-reboot

Post warm reboot service check

Category ID:

power-management

Status:

Blocking

Purpose:

Check there are no failed services after the warm reboot

Plugin:

shell

power-management/rtc

Test that RTC functions properly (if present)

Category ID:

power-management

Status:

Blocking

Purpose:

Verify that the Real-time clock (RTC) device functions properly, if present.

After-suspend:

True

Environment variable:

RTC_DEVICE_FILE

User:

root

Plugin:

shell

power-management/tickless_idle

Verify the tickless idle feature configuration in the kernel.

Category ID:

power-management

Status:

Blocking

Purpose:

Check to see if CONFIG_NO_HZ is set in the kernel (this is just a simple regression check)

After-suspend:

True

Plugin:

shell

power-management/warm-reboot

Warm reboot

Category ID:

power-management

Status:

Blocking

Purpose:

This tests reboots the system using the reboot command

User:

root

Plugin:

shell

Suspend tests

The following test units are covered in this category:

suspend/audio_after_suspend_auto

Verify mixer settings consistency after system suspend.

Category ID:

suspend

Status:

Blocking

Purpose:

Verify that mixer settings after suspend are the same as before suspend.

Plugin:

shell

suspend/audio_before_suspend

Record mixer settings before suspending.

Category ID:

suspend

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

suspend/cpu_after_suspend_auto

Ensure all CPUs are online post-resumption.

Category ID:

suspend

Status:

Blocking

Purpose:

Verify that all CPUs are online after resuming.

Plugin:

shell

suspend/cpu_before_suspend

Verify that all the CPUs are online before suspending

Category ID:

suspend

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

suspend/memory_after_suspend_auto

Ensure all memory is accessible after waking from suspend mode.

Category ID:

suspend

Status:

Blocking

Purpose:

Verify that all memory is available after resuming from suspend.

Plugin:

shell

suspend/memory_before_suspend

Dumps memory info to a file for comparison after suspend

Category ID:

suspend

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

suspend/network_after_suspend_auto

Verify the network functionality after system resume.

Category ID:

suspend

Status:

Blocking

Purpose:

Test the network after resuming.

User:

root

Plugin:

shell

suspend/network_before_suspend

Record the current network before suspending.

Category ID:

suspend

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

Touchpad tests

The following test units are covered in this category:

touchpad/detected-as-mouse

Check if the touchpad is recognized as a mouse by the system.

Category ID:

touchpad

Status:

Blocking

Purpose:

This test will check if your touchpad was detected as a mouse.

After-suspend:

True

Plugin:

shell

TPM 2.0 (Trusted Platform Module)

The following test units are covered in this category:

clevis-encrypt-tpm2/detect-ecc-capabilities

Ensure the TPM has required capabilities for clevis ECC test

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

clevis-encrypt-tpm2/detect-rsa-capabilities

Ensure the TPM has required capabilities for clevis RSA test

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

clevis-encrypt-tpm2/ecc

clevis encrypt/decrypt key ecc

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

clevis-encrypt-tpm2/precheck

clevis encrypt/decrypt precheck

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

clevis-encrypt-tpm2/rsa

clevis encrypt/decrypt key rsa

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

USB tests

The following test units are covered in this category:

usb/detect

Display USB devices attached to SUT

Category ID:

usb

Status:

Blocking

Purpose:

Detects and shows USB devices attached to this system.

After-suspend:

True

Plugin:

shell

Wireless networking tests

The following test units are covered in this category:

wireless/wireless_connection_open_ac_nm_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_ac_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_ax_nm_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_ax_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_be_nm_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_be_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_bg_nm_interface

Connect to an unencrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_bg_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_n_nm_interface

Connect to an unencrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_n_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa3_ax_nm_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa3_ax_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa3_be_nm_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa3_be_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_ac_nm_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_ac_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_ax_nm_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_ax_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_be_nm_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_be_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_bg_nm_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_bg_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_n_nm_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_n_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_scanning_interface

Test system can discover Wi-Fi networks on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_scanning_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Non-blocking

Audio tests

The following test units are covered in this category:

audio/alsa_info_attachment

Attach audio hardware data collection log to the results.

Category ID:

audio

Status:

Non-blocking

Purpose:

Attaches the audio hardware data collection log to the results.

Plugin:

attachment

audio/alsa_info_collect

Collect audio-related system information for simulation and detailed testing.

Category ID:

audio

Status:

Non-blocking

Purpose:

Collect audio-related system information. This data can be used to simulate this computer’s audio subsystem and perform more detailed tests under a controlled environment.

User:

root

Plugin:

shell

audio/alsa_record_playback_automated

Test playback and recording functionality on the default audio input and output.

Category ID:

audio

Status:

Non-blocking

Purpose:

Play back a sound on the default output and listen for it on the default input. This test is intended mostly for laptops, where the default output will be the internal speakers and the default input will be the internal microphone.

After-suspend:

True

Plugin:

shell

audio/detect_sinks

Ensure audio sinks are available for detection.

Category ID:

audio

Status:

Non-blocking

Purpose:

Test to detect if there are available sinks

After-suspend:

True

Plugin:

shell

audio/list_devices

Check if audio devices can be detected.

Category ID:

audio

Status:

Non-blocking

Purpose:

Test to detect audio devices

Plugin:

shell

Benchmarks tests

The following test units are covered in this category:

benchmarks/disk/hdparm-cache-read_name

Cached read timing benchmark of {name} using hdparm

Unit type:

template

Category ID:

benchmarks

Status:

Non-blocking

After-suspend:

True

From template:

benchmarks/disk/hdparm-cache-read_name

Template resource:

device

Template filter:

device.category == ‘DISK’

benchmarks/disk/hdparm-read_name

Raw read timing benchmark of {name} using hdparm

Unit type:

template

Category ID:

benchmarks

Status:

Non-blocking

After-suspend:

True

From template:

benchmarks/disk/hdparm-read_name

Template resource:

device

Template filter:

device.category == ‘DISK’

Bluetooth tests

The following test units are covered in this category:

bluetooth/zapper-a2dp

Check if the system can connect to a Bluetooth speaker using A2DP profile

Category ID:

bluetooth

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

ZAPPER_HOST

Plugin:

shell

Camera tests

The following test units are covered in this category:

camera/camera-quality-image_name

Attach the image used for the BRISQUE score for {product_slug}

Unit type:

template

Category ID:

camera

Status:

Non-blocking

Description:

This test will attach the image used for the BRISQUE score.

After-suspend:

True

From template:

camera/camera-quality-image_name

Template resource:

device

Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

camera/camera-quality_name

Webcam BRISQUE score for {product_slug}

Unit type:

template

Category ID:

camera

Status:

Non-blocking

After-suspend:

True

From template:

camera/camera-quality_name

Template resource:

device

Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

camera/multiple-resolution-images-attachment_name

Attach an image from the multiple resolution images test for {product_slug}

Unit type:

template

Category ID:

camera

Status:

Non-blocking

Description:

This test will attach one of the images used for the multiple resolution images test.

After-suspend:

True

From template:

camera/multiple-resolution-images-attachment_name

Template resource:

device

Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

CPU tests

The following test units are covered in this category:

cpu/cstates_results.log

Attach C-States test log

Category ID:

cpu

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Attaches the FWTS desktop diagnosis results log to the submission.

Plugin:

attachment

cpu/maxfreq_test-log-attach

Attach CPU max frequency log

Category ID:

cpu

Status:

Non-blocking

Purpose:

Attaches the log generated by cpu/maxfreq_test to the results submission.

Plugin:

attachment

cpu/scaling_test-log-attach

Attach CPU scaling capabilities log

Category ID:

cpu

Status:

Non-blocking

Purpose:

Attaches the log generated by cpu/scaling_test to the results submission.

Plugin:

attachment

Disk tests

The following test units are covered in this category:

disk/apste_support_on_name

Check support for Autonomous Power State Transition on {name}

Unit type:

template

Category ID:

disk

Status:

Non-blocking

Description:

<missing description>

After-suspend:

True

From template:

disk/apste_support_on_name

Template resource:

device

Template filter:

device.driver == ‘nvme’ and device.category == ‘OTHER’

disk/check-software-raid

Validate the configuration of software RAID devices are expected

Category ID:

disk

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Examine the system to detect Software RAID devices are created and the RAID mode are expected the SOFTWARE_RAID_LEVEL variable is needed for this tests. e.g. SOFTWARE_RAID_LEVEL=”raid0 raid1 raid5”

Environment variable:

SOFTWARE_RAID_LEVEL

User:

root

Plugin:

shell

thunderbolt3/storage-preinserted

Storage test on Thunderbolt 3 pre-inserted

Category ID:

disk

Status:

Non-blocking

Purpose:

This is an automated test which performs read/write operations on a preinserted Thunderbolt HDD

After-suspend:

True

User:

root

Plugin:

shell

Firmware tests

The following test units are covered in this category:

firmware/fwts_desktop_diagnosis

Run FWTS QA-concerned desktop-specific diagnosis tests.

Category ID:

firmware

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Run Firmware Test Suite (fwts) QA-concerned desktop-specific diagnosis tests.

Environment variable:

PLAINBOX_SESSION_SHARE

User:

root

Plugin:

shell

firmware/fwts_desktop_diagnosis_results.log.gz

Attach FWTS desktop diagnosis log to submission

Category ID:

firmware

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Attaches the FWTS desktop diagnosis results log to the submission

Plugin:

attachment

firmware/fwts_dump_acpi_attachment.gz

Collect the ACPI tables dump from FWTS

Category ID:

firmware

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

attachment

Graphics tests

The following test units are covered in this category:

graphics/xorg-failsafe

Test that X is not in failsafe mode.

Category ID:

graphics

Status:

Non-blocking

Purpose:

Test that the X is not running in failsafe mode.

Plugin:

shell

graphics/xorg-process

Test that the X process is running.

Category ID:

graphics

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Test that the X process is running.

Plugin:

shell

graphics/xorg-version

Test Xorg version output.

Category ID:

graphics

Status:

Non-blocking

Purpose:

Test to output the Xorg version

Plugin:

shell

Informational tests

The following test units are covered in this category:

acpi_sleep_attachment

Attach the contents of /proc/acpi/sleep for further analysis.

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches the contents of /proc/acpi/sleep if it exists.

Plugin:

attachment

codecs_attachment

Attach a report of installed codecs for Intel HDA.

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches a report of installed codecs for Intel HDA

Plugin:

attachment

cpuinfo_attachment

Attach a copy of /proc/cpuinfo

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches a report of CPU information

Plugin:

attachment

dkms_info_attachment

Attaches json dumps of installed dkms package information.

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Attaches json dumps of installed dkms package information.

Plugin:

attachment

dmesg_attachment

Attach a copy of dmesg or the current dmesg buffer to the test results.

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches a copy of /var/log/dmesg or the current dmesg buffer to the test results

User:

root

Plugin:

attachment

dmi_attachment

Attach a copy of /sys/class/dmi/id/*

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches info on DMI

Plugin:

attachment

dmidecode_attachment

Attach output of dmidecode

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Attaches dmidecode output

User:

root

Plugin:

attachment

efi_attachment

Attaches firmware version info

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches the firmware version

User:

root

Plugin:

attachment

info/buildstamp

Attaches the buildstamp identifier for the OS

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Attaches the buildstamp identifier for the OS

Plugin:

attachment

info/disk_partitions

Attaches info about disk partitions

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches information about disk partitions

User:

root

Plugin:

attachment

info/hdparm_name.txt

Attaches info from hdparm about {name}

Unit type:

template

Category ID:

info

Status:

Non-blocking

Description:

<missing description>

From template:

info/hdparm_name.txt

Template resource:

device

Template filter:

device.category == ‘DISK’

info/secure-boot-check

Check secure boot state

Category ID:

info

Status:

Non-blocking

Purpose:

Output whether secure boot is enabled or disabled

Plugin:

attachment

info/systemd-analyze

System boot-up performance statistics

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

info/systemd-analyze-critical-chain

Print the tree of the time-critical chain of SystemD

Category ID:

info

Status:

Non-blocking

Purpose:

This job prints a tree of the time-critical chain of SystemD units.

Plugin:

shell

info/touchpad_driver

Gather touchpad name, driver name, and driver version information.

Category ID:

info

Status:

Non-blocking

Purpose:

Returns the name, driver name and driver version of any touchpad discovered on the system.

Plugin:

attachment

kernel_cmdline_attachment

Attach a copy of /proc/cmdline

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches the kernel command line used to boot

Plugin:

attachment

lsmod_attachment

Attach a list of currently running kernel modules

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches a list of the currently running kernel modules.

Plugin:

attachment

lspci_attachment

Attach a list of PCI devices

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches very verbose lspci output.

Plugin:

attachment

lspci_standard_config_attachment

Attach PCI configuration space hex dump

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches a hex dump of the standard part of the PCI configuration space for all PCI devices.

Plugin:

attachment

lstopo_visual_attachment

Attach the output of lstopo command to present system topology.

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches the system topology as presented by the lstopo command

Plugin:

attachment

lsusb_attachment

Attach output of lsusb

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches a list of detected USB devices.

After-suspend:

True

User:

root

Plugin:

attachment

manifest

Hardware Manifest

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

This job loads the hardware manifest and exposes it as a resource.

Plugin:

resource

meminfo_attachment

Attach copy of /proc/meminfo

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches info on system memory as seen in /proc/meminfo.

Plugin:

attachment

modinfo_attachment

Attach modinfo information

Category ID:

info

Status:

Non-blocking

Purpose:

Attaches modinfo information for all currently loaded modules

Plugin:

attachment

modprobe_attachment

Attach the contents of /etc/modprobe.*

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Attaches the contents of the various modprobe conf files.

User:

root

Plugin:

attachment

modules_attachment

Attach the contents of /etc/modules

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Attaches the contents of the /etc/modules file.

Plugin:

attachment

net_if_management_attachment

Collect logging from the net_if_management job

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Allows logging of debug info and errors by the associated resource job

Plugin:

attachment

recovery_info_attachment

Attach the recovery partition versions

Category ID:

info

Status:

Non-blocking

Purpose:

Attach the recovery partition version image_version is the preinstalled OS image version bto_version is only for dell_recovery Example: image_version: somerville-trusty-amd64-20140620-0 bto_version: A00_dell-bto-trusty-houston-15-A11-iso-20141203-0.iso

User:

root

Plugin:

attachment

sysctl_attachment

Attach sysctl configuration files.

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Attaches the contents of various sysctl config files.

User:

root

Plugin:

attachment

sysfs_attachment

Attach detailed sysfs property output from udev

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Attaches a report of sysfs attributes.

Plugin:

attachment

udev_attachment

Attach dump of udev database

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Attaches a dump of the udev database showing system hardware information.

Plugin:

attachment

Input Devices tests

The following test units are covered in this category:

input/fixed_screen_orientation_on_product___index__

Check whether screen orientation is fixed on {product}

Unit type:

template

Category ID:

input

Status:

Non-blocking

After-suspend:

True

From template:

input/fixed_screen_orientation_on_product___index__

Template resource:

dmi

Template filter:

dmi.category == ‘CHASSIS’ and dmi.product not in (‘Tablet’, ‘Convertible’, ‘Detachable’)

input/zapper-keyboard

Check if the system recognizes keyboard events from Zapper

Category ID:

input

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

ZAPPER_HOST

User:

root

Plugin:

shell

Media Card tests

The following test units are covered in this category:

Miscellaneous tests

The following test units are covered in this category:

install/apt-get-gets-updates

Ensure apt can access repositories and get updates without installing them, to aid in recovery from broken updates.

Category ID:

miscellanea

Status:

Non-blocking

Purpose:

Tests to see that apt can access repositories and get updates (does not install updates). This is done to confirm that you could recover from an incomplete or broken update.

User:

root

Plugin:

shell

miscellanea/dkms_build_validation

Validate the build status of DKMS modules, automatically

Category ID:

miscellanea

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

Firstly, check the built number of DKMS modules. Secondly, scan APT system update log to see any build error messages. This test is mainly used for pre-installed images which use DKMS modules.

Plugin:

shell

miscellanea/oops_results.log

Attach the FWTS oops results for submission.

Category ID:

miscellanea

Status:

Non-blocking

Purpose:

Attaches the FWTS oops results log to the submission

Plugin:

attachment

miscellanea/submission-resources

Check that data for a complete result are present

Category ID:

miscellanea

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

A meta-job that verifies the data necessary for a complete result submission are present. Failure indicates that the results are incomplete and may be rejected.

Plugin:

shell

Mobile broadband tests

The following test units are covered in this category:

mobilebroadband/cdma_connection

Ensure a CDMA based modem establishes and maintains a mobile broadband connection.

Category ID:

mobilebroadband

Status:

Non-blocking

Purpose:

Creates a mobile broadband connection for a CDMA based modem and checks the connection to ensure it’s working.

After-suspend:

True

Environment variable:

CDMA_CONN_NAME, CDMA_PASSWORD, CDMA_USERNAME

User:

root

Plugin:

shell

mobilebroadband/gsm_connection

Establish and verify GSM mobile broadband connection.

Category ID:

mobilebroadband

Status:

Non-blocking

Purpose:

Creates a mobile broadband connection for a GSM based modem and checks the connection to ensure it’s working.

After-suspend:

True

Environment variable:

GSM_APN, GSM_CONN_NAME, GSM_PASSWORD, GSM_USERNAME

User:

root

Plugin:

shell

Monitor tests

The following test units are covered in this category:

monitor/zapper-edid

Check if the system automatically changes the resolution based on EDID

Category ID:

monitor

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

ZAPPER_HOST

Plugin:

shell

Non-device specific networking tests

The following test units are covered in this category:

networking/http

Ensure downloading files through HTTP works correctly.

Category ID:

networking

Status:

Non-blocking

Purpose:

Automated test case to make sure that it’s possible to download files through HTTP

Environment variable:

TRANSFER_SERVER

Plugin:

shell

Power Management tests

The following test units are covered in this category:

power-management/fwts_wakealarm-log-attach

Attach and display fwts wakealarm test log.

Category ID:

power-management

Status:

Non-blocking

Purpose:

Attach log from fwts wakealarm test

Plugin:

attachment

Real Time Performance Tests

The following test units are covered in this category:

rt-performance-tests/cyclictest_rt

A short-duration cyclictest to validate the system’s real-time capabilities.

Category ID:

rt-performance-tests

Status:

Non-blocking

Purpose:

This test runs for 60 seconds to provide a rapid assessment of the system’s real-time performance. It uses the cyclictest tool to measure the scheduling latency and overflow occurrences. The goal is to ensure that basic real-time functions can be executed without latency spikes.

User:

root

Plugin:

shell

Touchpad tests

The following test units are covered in this category:

touchpad/palm-rejection-firmware-labeling_product_slug

Check palm rejection firmware/labeling for touchpads

Unit type:

template

Category ID:

touchpad

Status:

Non-blocking

After-suspend:

True

From template:

touchpad/palm-rejection-firmware-labeling_product_slug

Template resource:

device

Template filter:

device.category == ‘TOUCHPAD’ and device.driver == ‘hid-multitouch’

TPM 2.0 (Trusted Platform Module)

The following test units are covered in this category:

tpm2/fwts-event-log-dump

Dump the contents of the TPM Event Log

Category ID:

tpm2

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

The information in the TPM Event Log can be useful in debugging problems with TPM command support and adherance to standards. This can be of particular help when checking whether a device can support Ubuntu Core Full Disk Encryption.

User:

root

Plugin:

attachment

USB tests

The following test units are covered in this category:

usb/storage-detect

Detect storage partitions on a device on the USB bus

Category ID:

usb

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

usb/zapper-usb-insert-port_alias

Zapper-driven USB insertion test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Non-blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb/zapper-usb-insert-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’

usb/zapper-usb-remove-port_alias

Zapper-driven USB removal test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Non-blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb/zapper-usb-remove-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’

usb3/zapper-usb-insert-port_alias

Zapper-driven USB 3.0 insertion test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Non-blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb3/zapper-usb-insert-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’

usb3/zapper-usb-remove-port_alias

Zapper-driven USB 3.0 removal test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Non-blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb3/zapper-usb-remove-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’

Wireless networking tests

The following test units are covered in this category:

wireless/check_iwlwifi_microcode_crash_interface

Check there have been no iwlwifi crashes

Unit type:

template

Category ID:

wireless

Status:

Non-blocking

After-suspend:

True

From template:

wireless/check_iwlwifi_microcode_crash_interface

Template resource:

device

Template filter:

device.driver == ‘iwlwifi’

wireless/detect

Detect if at least one Wireless LAN device is detected

Category ID:

wireless

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

wireless/nm_connection_restore_interface

Restore any NetworkManager 802.11 configurations after testing

Unit type:

template

Category ID:

wireless

Status:

Non-blocking

Description:

<missing description>

After-suspend:

True

From template:

wireless/nm_connection_restore_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/nm_connection_save_interface

Save any NetworkManager 802.11 configurations prior to testing

Unit type:

template

Category ID:

wireless

Status:

Non-blocking

Description:

<missing description>

After-suspend:

True

From template:

wireless/nm_connection_save_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Appendix A. FWTS tests

As part of the certification process, we run a series of firmware tests that are part of the Canonical Firmware Test Suite. In general, any HIGH or CRITICAL error found in the fwts log can cause potential errors in the system and should be looked at by OEMs/ODMs.

Category

Test Item

Description

Information

acpidump

Check ACPI table acpidump.

Information

version

Gather kernel system information.

ACPI

acpitables

ACPI table settings confidence checks.

ACPI

apicinstance

Check for single instance of APIC/MADT table.

ACPI

hpet_check

High Precision Event Timer configuration test.

ACPI

mcfg

MCFG PCI Express* memory mapped config space.

ACPI

method

ACPI DSDT Method Semantic Tests.

CPU

mpcheck

Check Multi Processor tables.

CPU

msr

CPU MSR consistency check.

CPU

mtrr

MTRR validation.

System

apicedge

APIC Edge/Level Check.

System

klog

Scan kernel log for errors and warnings.