Internet of Things Certified Hardware Coverage for Ubuntu Core 26 / Ubuntu 26.04

Introduction

This document lists the coverage for certification of Internet of Things (IoT) devices with Ubuntu images. IoT devices can be certified with the following image types:

  • Ubuntu Core 26

  • Ubuntu Server 26.04 LTS

  • Ubuntu Desktop 26.04 LTS

The guide applies to devices submitted to Canonical through one of the following programmes:

  • IoT Devices Enablement Programme with Certification

  • IoT ODM Partner Programme

For each test job, one of the following certification statuses is specified:

Blocking

Features that are required for certification. If any of the blocking tests fails, the certification will fail.

Non-blocking

Features that are tested but not mandatory for certification. Failure in non-blocking tests will not prevent certification. However, a note will be added to the certificate to inform potential customers or users.

Note

Only categories of hardware are tested and not specific types of hardware. For example, tests are run to verify USB controllers work, but the type of peripheral(s) used during those tests are not specified.

Coverage is flexible based on customer requirements (for example, if a device’s use cases don’t require LEDs, then LEDs can be non-blocking)

Certain test jobs are designed to validate specific hardware capabilities, such as camera and audio playback functionality. To ensure that the required hardware capabilities are present and properly recognised on the machine under test, these features are explicitly defined in manifest entries and linked to the relevant test jobs. This prevents test jobs from being skipped due to system deficiencies in automated detection.

Full test descriptions can be found in Canonical certification site for partners:

https://certification.canonical.com

client-cert-iot-ubuntucore-26

Note

The certification tests presented in this document are validated by Checkbox version 7.3.0.dev42.

Blocking

Audio tests

The following test units are covered in this category:

audio/alsa-loopback-automated

Captured sound matches played one (automated)

Category ID:

audio
Status:

Blocking
Purpose:

Check if sound that is ‘hearable’ by capture device. This test is intended mostly for devices without internal speakers/microphones, but with a line-in and a line-out ports to which a loopback cable can be connected.
After-suspend:

True
Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH, LD_LIBRARY_PATH
Plugin:

shell
Requires:

manifest.has_audio_loopback_connector == ‘True’
For details about the required manifest entry, see has_audio_loopback_connector.
audio/alsa-playback

Playback works

Category ID:

audio
Status:

Blocking
Purpose:

Check if sound is played through ALSA on the default output
Steps:

  1. Make sure speakers or headphones are connected to the device 2. Commence the test
Verification:

Did you hear the sound?
After-suspend:

True
Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH
Plugin:

user-interact-verify

Bluetooth tests

The following test units are covered in this category:

bluetooth/audio-a2dp

Verify Bluetooth device’s High Fidelity Playback (A2DP) capability.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check the High Fidelity Playback (A2DP) capability of your Bluetooth device, to see if you can hear audio from it.
Steps:

  1. Enable and pair the Bluetooth headset 2. Click “Test” to play a brief tone on your Bluetooth device, if it failed to set the Mode to A2DP, please select the device and change it manually in the “Sound Settings”
Verification:

Did you hear the tone?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_bt_smart == ‘True’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_bt_smart.
bluetooth/audio_record_playback

Verify Bluetooth HSP/HFP profile capability for recording and playback.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check the Headset Head Unit (HSP/HFP) capability of your Bluetooth device, to check if you can record sounds.
Steps:

  1. Enable and pair the bluetooth headset. 2. Click “Test”, then speak into your Bluetooth microphone. 3. After a few seconds, your speech will be played back to you.
Verification:

Did you hear your speech played back?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_bt_smart == ‘True’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_bt_smart.
bluetooth/bluetooth_obex_send

Bluetooth OBEX send

Category ID:

bluetooth
Status:

Blocking
Purpose:

This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable
After-suspend:

True
Environment variable:

BTDEVADDR, PLAINBOX_PROVIDER_DATA
Plugin:

shell
Requires:

package.name == ‘bluez’ or snap.name == ‘bluez’ executable.name == ‘obexftp’ and executable.name == ‘hcitool’ device.category == ‘BLUETOOTH’ manifest.has_bt_obex_support == ‘True’
For details about the required manifest entry, see has_bt_obex_support.
bluetooth/detect

Make sure at least one bluetooth device is detected

Category ID:

bluetooth
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_bt_adapter == ‘True’
For details about the required manifest entry, see has_bt_adapter.
bluetooth/detect-output

Store Bluetooth device information for reports.

Category ID:

bluetooth
Status:

Blocking
Purpose:

Automated test to store Bluetooth device information in the Checkbox report
After-suspend:

True
Plugin:

shell
Requires:

package.name == ‘bluez’ or snap.name == ‘bluez’ device.category == ‘BLUETOOTH’
bluetooth4/HOGP-keyboard

Verify HOGP keyboard functionality with Bluetooth Smart.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart keyboard.
Steps:

  1. Enable a Bluetooth Smart keyboard, and put it into pairing mode. 2. Commence the test to do the auto-pairing, you will be asked to select the targeting keyboard from the list. 3. After it’s paired and connected, enter some text with your keyboard.
Verification:

Did the Bluetooth Smart keyboard work as expected?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_bt_smart == ‘True’ package.name == ‘bluez’
For details about the required manifest entry, see has_bt_smart.

CPU tests

The following test units are covered in this category:

cpu/cstates

Run C-States tests

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Uses the Firmware Test Suite (fwts) to test the power saving states of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’ cpuinfo.platform not in (“aarch64”, “armv7l”, “ppc64el”, “ppc64le”, “s390x”)
cpu/maxfreq_test

Test that the CPU can run at its max frequency

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.
Environment variable:

LD_LIBRARY_PATH, SNAP
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’ cpuinfo.platform in (“i386”, “x86_64”)
cpu/offlining_test

Test offlining of each CPU core

Category ID:

cpu
Status:

Blocking
Purpose:

Attempts to offline each core in a multicore system.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

cpu_offlining.state == ‘supported’
cpu/scaling_test

Test the CPU scaling capabilities

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use Firmware Test Suite (fwts cpufreq) to test the scaling capabilities of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’ ‘userspace’ in cpuinfo.governors cpuinfo.platform not in (“ppc64el”, “ppc64le”, “s390x”)
cpu/topology

Check CPU topology for accuracy between proc and sysfs

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.
After-suspend:

True
Plugin:

shell
Requires:

int(cpuinfo.count) > 1 and (cpuinfo.platform == ‘i386’ or cpuinfo.platform == ‘x86_64’)

Disk tests

The following test units are covered in this category:

disk/read_performance_name

Disk performance test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
Purpose:

Verify that disk storage performs at or above baseline performance
After-suspend:

True
From template:

disk/read_performance_name
Template resource:

device
Template filter:

device.category == ‘DISK’
disk/storage_device_name

Disk I/O stress test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
Purpose:

Take the path of the storage device and test if it is a block device.
After-suspend:

True
From template:

disk/storage_device_name
Template resource:

device
Template filter:

device.category == ‘DISK’
thunderbolt3/storage-manual

Thunderbolt 3 HDD storage insertion + read/write + removal

Category ID:

disk
Status:

Blocking
Purpose:

This test will check if the connection of a Thunderbolt 3 HDD could be detected, then performs read/write operations on the attached Thunderbolt 3 HDD storage and checks if the removal of the Thunderbolt 3 HDD can be detected.
Steps:

  1. Click ‘Test’ to begin the test. This test will timeout and fail if the insertion has not been detected within 30 seconds. 2. Plug a Thunderbolt 3 HDD into an available Thunderbolt 3 port, if it’s not mounted automatically, please click the HDD icon to mount it. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the previously attached Thunderbolt 3 HDD.
Verification:

The verification of this test is automated. Do not change the automatically selected result
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

manifest.has_thunderbolt3 == ‘True’
For details about the required manifest entry, see has_thunderbolt3.

Ethernet Device tests

The following test units are covered in this category:

ethernet/detect

Detect if at least one ethernet device is detected

Category ID:

ethernet
Status:

Blocking
Purpose:

Test to detect and return information about available network controllers on the system under test.
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_ethernet_adapter == ‘True’
For details about the required manifest entry, see has_ethernet_adapter.
ethernet/hotplug-interface

Ensure hotplugging works on port {{ interface }}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

Check that hotplugging works on port {{ interface }}
After-suspend:

True
From template:

ethernet/hotplug-interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/iperf3_reverse_interface

Iperf3 stress testing for {interface} (reverse)

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

This test uses iperf3 to ensure network devices pass data at an acceptable minimum percentage of advertised speed (Reverse).
From template:

ethernet/iperf3_reverse_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/iperf3_interface

Iperf3 stress testing for {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

This test uses iperf3 to ensure network devices pass data at an acceptable minimum percentage of advertised speed.
From template:

ethernet/iperf3_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/ping-with-any-cable-interface

Can ping the gateway with any cable Ethernet interface

Category ID:

ethernet
Status:

Blocking
Purpose:

Check any of the cable Ethernet ports available on the system can ping its default gateway.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

device.category == ‘NETWORK’
ethernet/ping_interface

Can ping another machine over Ethernet port {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

Check Ethernet works by pinging another machine
After-suspend:

True
From template:

ethernet/ping_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’

I²C (Inter-Integrated Circuit)

The following test units are covered in this category:

i2c/i2c-bus-detect

Check presence of an I²C bus

Category ID:

i2c
Status:

Blocking
Purpose:

If an expected number of I²C buses is provided, the job will verify the detected number is correct. If the expected number of buses is not provided, the job will pass if at least one I²C bus is detected.
Environment variable:

I2C_BUS_NUMBER
User:

root
Plugin:

shell
Requires:

manifest.has_i2c == ‘True’
For details about the required manifest entry, see has_i2c.
i2c/i2c-device-detect

Check if any I²C devices can be detected

Category ID:

i2c
Status:

Blocking
Purpose:

The test will pass if there’s at least one I²C device detected on any I²C bus.
Environment variable:

IGNORED_I2C_BUSES
User:

root
Plugin:

shell

Media Card tests

The following test units are covered in this category:

mediacard/sdhc-storage-manual

Test SDHC card insertion + read/write + removal.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a UNLOCKED Secure Digital High-Capacity (SDHC) media card Then it performs a read and write test on the SDHC card. Finally, it checks that the system detects the removal of the SDHC card.
Steps:

  1. Commence the test and then insert an UNLOCKED SDHC card into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the SDHC card from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

manifest.has_card_reader == ‘True’
For details about the required manifest entry, see has_card_reader.

Memory tests

The following test units are covered in this category:

memory/info

Check the amount of memory reported by meminfo against DMI

Category ID:

memory
Status:

Blocking
Purpose:

This test checks the amount of memory which is reported in meminfo against the size of the memory modules detected by DMI.
User:

root
Plugin:

shell

Miscellaneous tests

The following test units are covered in this category:

miscellanea/oops

Run FWTS OOPS check

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (FWTS) oops tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’

Monitor tests

The following test units are covered in this category:

monitor/displayport_hotplug

Can hotplug monitor (DisplayPort)

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the DisplayPort port and the ability to do hotplugging.
Steps:

Skip this test if your system does not have a DisplayPort port. 1. If a display is already connected, unplug it. 2. (Re-)Connect a display to the DisplayPort port on your system
Verification:

Was the interface displayed correctly on the screen?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_dp == ‘True’
For details about the required manifest entry, see has_dp.
monitor/dvi

Check DVI port functionality by connecting a display and verifying different display modes.

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your DVI port as a monitor interconnect.
Steps:

  1. Connect a display (if not already connected) to the DVI port on your system. 2. Switch display modes within your Display Settings, check if it can be set to mirrored, extended, display-only on external or onboard only.
Verification:

Was the desktop displayed correctly on the DVI-connected screen in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_dvi == ‘True’
For details about the required manifest entry, see has_dvi.
monitor/hdmi

Check HDMI port functionality as a monitor interconnect.

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your HDMI port as a monitor interconnect.
Steps:

  1. Connect a display (if not already connected) to the HDMI port on your system. 2. Switch display modes within your Display Settings, check if it can be set to mirrored, extended, displayed on external or onboard only.
Verification:

Was the desktop displayed correctly on the HDMI-connected screen in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_hdmi == ‘True’
For details about the required manifest entry, see has_hdmi.
monitor/hdmi_hotplug

Can hotplug monitor (HDMI)

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the HDMI port and the ability to do hotplugging.
Steps:

Skip this test if your system does not have a HDMI port. 1. If a display is already connected, unplug it. 2. (Re-)Connect a display to the HDMI port on your system
Verification:

Was the interface displayed correctly on the screen?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_hdmi == ‘True’
For details about the required manifest entry, see has_hdmi.

Non-device specific networking tests

The following test units are covered in this category:

networking/gateway_ping

Verifies if the system can connect to the Internet.

Category ID:

networking
Status:

Blocking
Purpose:

Tests whether the system has a working Internet connection.
After-suspend:

True
User:

root
Plugin:

shell
networking/info_device__index___interface

Network Information of device {__index__} ({interface})

Unit type:

template
Category ID:

networking
Status:

Blocking
Purpose:

This test will check the network device {__index__} ({interface})
After-suspend:

True
From template:

networking/info_device__index___interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’

Power Management tests

The following test units are covered in this category:

power-management/cold-reboot

Cold reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This test powers off the system and then powers it on using RTC
Environment variable:

COLD_REBOOT_DELAY, RTC_DEVICE_FILE
User:

root
Plugin:

shell
Requires:

rtc.state == ‘supported’ rtc.wakealarm == ‘supported’
power-management/fwts_wakealarm

Executes ACPI Wakealarm test to validate functionality.

Category ID:

power-management
Status:

Blocking
Purpose:

Test ACPI Wakealarm (fwts wakealarm)
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’
power-management/post-cold-reboot

Post cold reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the cold reboot
User:

root
Plugin:

shell
power-management/post-warm-reboot

Post warm reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the warm reboot
User:

root
Plugin:

shell
power-management/rtc

Test that RTC functions properly (if present)

Category ID:

power-management
Status:

Blocking
Purpose:

Verify that the Real-time clock (RTC) device functions properly, if present.
After-suspend:

True
Environment variable:

RTC_DEVICE_FILE
User:

root
Plugin:

shell
Requires:

rtc.state == ‘supported’ cpuinfo.other != ‘emulated by qemu’
power-management/warm-reboot

Warm reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This tests reboots the system using the reboot command
User:

root
Plugin:

shell

Serial Port

The following test units are covered in this category:

serial/loopback-dev

Serial loopback test of {dev}

Unit type:

template
Category ID:

serial
Status:

Blocking
Purpose:

Check if serial port is working hardwired
After-suspend:

True
From template:

serial/loopback-dev
Template resource:

serial_ports_static

Suspend tests

The following test units are covered in this category:

suspend/cpu_after_suspend_auto

Ensure all CPUs are online post-resumption.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that all CPUs are online after resuming.
Plugin:

shell
suspend/cpu_before_suspend

Verify that all the CPUs are online before suspending

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/memory_after_suspend_auto

Ensure all memory is accessible after waking from suspend mode.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that all memory is available after resuming from suspend.
Plugin:

shell
suspend/memory_before_suspend

Dumps memory info to a file for comparison after suspend

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/network_after_suspend_auto

Verify the network functionality after system resume.

Category ID:

suspend
Status:

Blocking
Purpose:

Test the network after resuming.
User:

root
Plugin:

shell
suspend/network_before_suspend

Record the current network before suspending.

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
suspend/oops_after_suspend

Run fwts oops tests post-suspension to ensure system stability.

Category ID:

suspend
Status:

Blocking
Purpose:

Run Firmware Test Suite (fwts) oops tests after suspend.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’
suspend/validate_suspend_status

Tests that the machine suspended correctly

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

Query sysfs and debugfs for the amount of times the system was suspended. We expect the count in /sys/power/suspend_stats/success and suscess in /sys/kernel/debugfs/suspend_stats to be non zero
User:

root
Plugin:

shell

Touchscreen tests

The following test units are covered in this category:

touchscreen/evdev/2-touch-tap-product_slug

Validate proper detection of a 2-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that 2-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/2-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/3-touch-tap-product_slug

Validate proper detection of a 3-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that 3-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/3-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/4-touch-tap-product_slug

Validate the detection of a 4-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that 4-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/4-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/single-touch-tap-product_slug

Validate the detection of a single-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that single-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/single-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’

TPM 2.0 (Trusted Platform Module)

The following test units are covered in this category:

clevis-encrypt-tpm2/detect-ecc-capabilities

Ensure the TPM has required capabilities for clevis ECC test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/detect-rsa-capabilities

Ensure the TPM has required capabilities for clevis RSA test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/ecc

clevis encrypt/decrypt key ecc

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/precheck

clevis encrypt/decrypt precheck

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_tpm2_chip == ‘True’ executable.name == ‘clevis-encrypt-tpm2’
For details about the required manifest entry, see has_tpm2_chip.
clevis-encrypt-tpm2/rsa

clevis encrypt/decrypt key rsa

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell

USB tests

The following test units are covered in this category:

usb-c/c-to-ethernet-adapter-insert

Check if USB Type-C to Ethernet adapter works

Category ID:

usb
Status:

Blocking
Purpose:

This test will check if system detects network interface of the Type-C to ethernet adapter.
Steps:

  1. Prepare USB Type-C to Ethernet adapter 2. Start the test 3. When the message “INSERT NOW” is shown, plug in the adapter to Type-C port
Verification:

<missing verification>
After-suspend:

True
Plugin:

user-interact
Requires:

manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.
usb-c/storage-manual

USB 3.0 storage device insertion + read/write + removal on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3.0 storage device in a USB Type-C connector. Then it performs a read and write test on the device. Finally, it checks that the system correctly detects the removal of the USB 3 storage. NOTE: Make sure the USB 3 storage device has a partition before starting the test.
Steps:

  1. Commence the test and then insert a USB 3.0 storage device to a USB Type-C port. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 3 storage from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

usb.usb3 == ‘supported’ manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.
usb/detect

Display USB devices attached to SUT

Category ID:

usb
Status:

Blocking
Purpose:

Detects and shows USB devices attached to this system.
After-suspend:

True
Plugin:

shell
Requires:

cpuinfo.platform != ‘s390x’
usb/hid

USB keyboard works

Category ID:

usb
Status:

Blocking
Purpose:

Check USB input device works
Steps:

  1. Connect USB keyboard 2. Input something with USB keyboard
Verification:

What was just input is displayed correctly
After-suspend:

True
Plugin:

manual
usb/storage-detect

Detect storage partitions on a device on the USB bus

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_usb_storage == ‘True’
For details about the required manifest entry, see has_usb_storage.
usb/storage-manual

Test USB 2.0 storage device insertion + read/write + removal.

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect insertion of a USB 2.0 storage device. Then it performs a read and write test on the USB 2.0. Finally, it checks that the system correctly detects the removal of the USB 2.0. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test and then insert an USB 2.0 into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 2.0 from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb3/storage-manual

Test USB 3.0 storage device insertion + read/write + removal.

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect insertion of a USB 3.0 storage device. Then it performs a read and write test on the USB 3.0. Finally, it checks that the system correctly detects the removal of the USB 3.0. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test and then insert an USB 3.0 into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 3.0 from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

usb.usb3 == ‘supported’

Wireless networking tests

The following test units are covered in this category:

wireless/detect

Detect if at least one Wireless LAN device is detected

Category ID:

wireless
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_wlan_adapter == ‘True’
For details about the required manifest entry, see has_wlan_adapter.
wireless/wireless_connection_open_ac_nm_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ac AP
After-suspend:

True
From template:

wireless/wireless_connection_open_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ac_np_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ac AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_ac_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_nm_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ax AP
After-suspend:

True
From template:

wireless/wireless_connection_open_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_np_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ax AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_nm_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11be AP
After-suspend:

True
From template:

wireless/wireless_connection_open_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_np_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11be AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_nm_interface

Connect to an unencrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check the system can connect to an insecure 802.11b/g AP
After-suspend:

True
From template:

wireless/wireless_connection_open_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_np_interface

Connect to unencrypted 802.11b/g Wi-Fi network on {{ interface }} using netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check the system can connect to an unsecured 802.11b/g access point using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_bg_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_nm_interface

Connect to an unencrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check if the system can connect to an unsecured 802.11n access point.
After-suspend:

True
From template:

wireless/wireless_connection_open_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_np_interface

Connect to unencrypted 802.11n Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11n AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_n_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_nm_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa3 security
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_np_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa3 security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_nm_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa3 security
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_np_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa3 security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_nm_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ac AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_np_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ac AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ac_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_nm_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_np_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_nm_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_np_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_nm_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11b/g AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_np_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11b/g AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_bg_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_nm_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check if the system can connect to an 802.11n AP with WPA security.
After-suspend:

True
From template:

wireless/wireless_connection_wpa_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_np_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }} using netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check if the system can connect to an 802.11n AP with WPA security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_n_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_scanning_interface

Test system can discover Wi-Fi networks on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can find a wireless network AP nearby
After-suspend:

True
From template:

wireless/wireless_scanning_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless Wide Area Network

The following test units are covered in this category:

wwan/check-sim-present-manufacturer-model-hw_id-auto

Check if a SIM card is present in a slot connected to the modem

Unit type:

template
Category ID:

wwan
Status:

Blocking
Purpose:

<missing purpose>
Description:

Check if a SIM card is present in a slot connected to the modem
After-suspend:

True
From template:

wwan/check-sim-present-manufacturer-model-hw_id-auto
Template resource:

wwan_resource
wwan/detect

Identify if WWAN module is missing

Category ID:

wwan
Status:

Blocking
Purpose:

Tests that there is a WWAN module present and indicates that testing of it should follow.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

manifest.has_wwan_module == ‘True’ snap.name == ‘modem-manager’ or package.name == ‘modemmanager’
For details about the required manifest entry, see has_wwan_module.
wwan/gsm-connection-manufacturer-model-hw_id-auto

Verify a GSM broadband modem can create a data connection multiple times

Unit type:

template
Category ID:

wwan
Status:

Blocking
Purpose:

Any modems discovered by the resource job that list GSM support will be tested to ensure a data connection can be made.
After-suspend:

True
From template:

wwan/gsm-connection-manufacturer-model-hw_id-auto
Template resource:

wwan_resource

Non-blocking

Advanced Configuration and Power Interface

The following test units are covered in this category:

acpi/oem_osi

Test ACPI OEM _OSI strings

Category ID:

acpi
Status:

Non-blocking
Purpose:

This checks if the deprecated OEM _OSI strings are still used by checking the ACPI DSDT and SSDT tables
User:

root
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”)

Audio tests

The following test units are covered in this category:

audio/detect-capture-devices

Check that at least one audio capture device exists

Category ID:

audio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_audio_capture == ‘True’
For details about the required manifest entry, see has_audio_capture.
audio/detect-playback-devices

Check that at least one audio playback device exists

Category ID:

audio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_audio_playback == ‘True’
For details about the required manifest entry, see has_audio_playback.

Bluetooth - BlueZ Self Tests

The following test units are covered in this category:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test

BlueZ-{bluez-internal-bnep-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

Runs a specific test from the bnep test suite.
After-suspend:

True
From template:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test
Template resource:

bluez-internal-bnep-tests
bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test

BlueZ-{bluez-internal-hci-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Runs a specific test from the hci test suite
After-suspend:

True
From template:

bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test
Template resource:

bluez-internal-hci-tests
bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test

BlueZ-{bluez-internal-rfcomm-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

Runs a specific test from the rfcomm test suite
After-suspend:

True
From template:

bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test
Template resource:

bluez-internal-rfcomm-tests
bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test

BlueZ-{bluez-internal-uc-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

Runs a specific test from the user channel test suite
After-suspend:

True
From template:

bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test
Template resource:

bluez-internal-uc-tests

Bluetooth tests

The following test units are covered in this category:

bluetooth/bluez-controller-detect

Check bluez lists a controller if rfkill detects one

Category ID:

bluetooth
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

package.name == ‘bluez’ or snap.name == ‘bluez’ environment.CHECKBOX_RUNNING_STRICT_SNAP != “1” or (connections.slot == ‘bluez:service’ and connections.plug == ‘{{ __system_env__[“SNAP_NAME”] }}:bluez’)
bluetooth4/beacon_eddystone_url_interface

Test system can get beacon EddyStone URL advertisements on the {{ interface }} adapter

Unit type:

template
Category ID:

bluetooth
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

bluetooth4/beacon_eddystone_url_interface
Template resource:

device
Template filter:

device.category == ‘BLUETOOTH’

Camera tests

The following test units are covered in this category:

camera/camera-quality-rpi_name

Webcam BRISQUE score for rpi devices on {name}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

Uses an image of the camera to get the quality based on a No-Reference image quality assessment algorithm called BRISQUE. This test will timeout and fail if the quality has not been computed within 120 seconds.
From template:

camera/camera-quality-rpi_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images-rpi-attachment_name

Attach an image from the multiple resolution images test on rpi

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

This test will attach one of the images used for the multiple resolution images test on rpi.
From template:

camera/multiple-resolution-images-rpi-attachment_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images-rpi_name

Webcam multiple resolution capture test for Pi Camera

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

Takes multiple pictures based on the resolutions supported by the camera and validates their size and that they are of a valid format.
From template:

camera/multiple-resolution-images-rpi_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images_name

Webcam multiple resolution capture test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

Takes multiple pictures based on the resolutions supported by the camera and validates their size and that they are of a valid format.
After-suspend:

True
From template:

camera/multiple-resolution-images_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

CPU tests

The following test units are covered in this category:

cpu/arm64_vfp_support_platform

Validate that the Floating Point Unit is running on {platform} device

Unit type:

template
Category ID:

cpu
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Validate that the Floating Point Unit is running on {platform} device.
From template:

cpu/arm64_vfp_support_platform
Template resource:

cpuinfo
Template filter:

cpuinfo.platform == ‘aarch64’
cpu/armhf_vfp_support_platform

Validate that the Vector Floating Point Unit is running on {platform} device

Unit type:

template
Category ID:

cpu
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Validate that the Vector Floating Point Unit is running on {platform} device.
From template:

cpu/armhf_vfp_support_platform
Template resource:

cpuinfo
Template filter:

cpuinfo.platform == ‘armv7l’
cpu/clocktest

Tests the CPU for clock jitter

Category ID:

cpu
Status:

Non-blocking
Purpose:

Runs a test for clock jitter on SMP machines.
After-suspend:

True
Plugin:

shell
Requires:

cpuinfo.platform not in (“s390x”)
cpu/cstates_results.log

Attach C-States test log

Category ID:

cpu
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission.
Plugin:

attachment
Requires:

cpuinfo.platform not in (“aarch64”, “armv7l”, “s390x”)
cpu/maxfreq_test-log-attach

Attach CPU max frequency log

Category ID:

cpu
Status:

Non-blocking
Purpose:

Attaches the log generated by cpu/maxfreq_test to the results submission.
Plugin:

attachment
cpu/scaling_test-log-attach

Attach CPU scaling capabilities log

Category ID:

cpu
Status:

Non-blocking
Purpose:

Attaches the log generated by cpu/scaling_test to the results submission.
Plugin:

attachment

DesignWare Core SuperSpeed USB 3.0 Controller

The following test units are covered in this category:

usb-dwc3/driver-detect

Detect if the USB DWC3 drivers are loaded

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_usb_dwc3_controller == ‘True’
For details about the required manifest entry, see has_usb_dwc3_controller.
usb-dwc3/mass-storage

Check DUT can be detected as mass storage device

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

Check that after connecting the device under test (DUT) to another device (host), DUT can be detected as mass storage device by the host.
Steps:

  1. In BIOS, enable USB xDCI support. 2. Press Enter to setup a 256 MB FAT32 image on the device. 3. From an Ubuntu (or Windows) host, connect a cable to the USB OTG port of the DUT.
Verification:

The host detects and mounts a mass storage device. It has read and write permissions on it.
User:

root
Plugin:

user-interact-verify
usb-dwc3/mass-storage-cleanup

Cleanup mass storage setup after mass storage device test

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
usb-dwc3/module-detect

Detect if the USB DWC3 module is loaded

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_usb_dwc3_controller == ‘True’
For details about the required manifest entry, see has_usb_dwc3_controller.

Disk tests

The following test units are covered in this category:

disk/apste_support_on_name

Check support for Autonomous Power State Transition on {name}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

disk/apste_support_on_name
Template resource:

device
Template filter:

device.driver == ‘nvme’ and device.category == ‘OTHER’
disk/check-software-raid

Validate the configuration of software RAID devices are expected

Category ID:

disk
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Examine the system to detect Software RAID devices are created and the RAID mode are expected the SOFTWARE_RAID_LEVEL variable is needed for this tests. e.g. SOFTWARE_RAID_LEVEL=”raid0 raid1 raid5”
Environment variable:

SOFTWARE_RAID_LEVEL
User:

root
Plugin:

shell
Requires:

executable.name == ‘mdadm’ manifest.has_md_raid == ‘True’
For details about the required manifest entry, see has_md_raid.
disk/disk_stress_ng_name

Disk stress-ng test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
Purpose:

Disk stress-ng test for {product_slug}
From template:

disk/disk_stress_ng_name
Template resource:

device
Template filter:

device.category == ‘DISK’
disk/encryption/check-fde-tpm

Disk decryption after TPM change

Category ID:

disk
Status:

Non-blocking
Purpose:

The device partition is encrypted using TPM master key. To unseal the master key from TPM, PCR7 (Platform Configuration Register 7) needs to be identical to the value it had when the master key was sealed into TPM. Every time the device boots, it checks PCR7 to unseal TPM and retrieves master key from TPM to decrypt its data partition. If TPM PCR7 is modified (e.g. by flashing the BIOS), the device won’t be able to get the master key and decrypt its data partition. This test verifies the system’s resilience against unauthorized modifications by ensuring it cannot boot if the PCR7 value is altered.
Steps:

NOTE: YOU’LL HAVE TO RE-INSTALL THE IMAGE AFTER THIS TEST. 1. Install the image and make sure it boots and you can log in. 2. Ensure the BIOS is set up correctly (e.g., TPM enabled, UEFI boot mode). 3. Depending on the DUT, choose one of these methods to clean TPM: a. Turn the device off and upgrade/downgrade the BIOS or modify Secure Boot state. b. Clean TPM via BIOS menu. c. Install checkbox, execute “checkbox-[project name].checkbox-cli run com.canonical.certification::tpm2.0_3.0.4/tpm2_takeownership”. 4. Start or reboot the device.
Verification:

Mark this test as “Passed” if the device cannot boot anymore.
Plugin:

manual
Requires:

manifest.has_fde == ‘True’
For details about the required manifest entry, see has_fde.
disk/encryption/detect

Test that Full Disk Encryption is in use

Category ID:

disk
Status:

Non-blocking
Purpose:

Examine the system to detect if one of the standard full disk encryption implementations is in use
User:

root
Plugin:

shell
Requires:

executable.name == ‘lsblk’ executable.name == ‘dmsetup’ executable.name == ‘cryptsetup’ manifest.has_fde == ‘True’
For details about the required manifest entry, see has_fde.
disk/stats_name

Disk statistics for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
Purpose:

This test checks disk stats, generates some activity and rechecks stats to verify they’ve changed. It also verifies that disks appear in the various files they’re supposed to. . This test will inspect the following disk: . product name: {product_slug} sysfs path: {path} device node path: /dev/{name}
After-suspend:

True
From template:

disk/stats_name
Template resource:

device
Template filter:

device.category == ‘DISK’ and device.name != ‘’

Docker containers

The following test units are covered in this category:

docker/build-single_arch

Test docker build with a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/build-single_arch
Template resource:

docker_resource
docker/commit_arch

Test docker commit a change to a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/commit_arch
Template resource:

docker_resource
docker/compose-and-basic_arch

Test docker compose and basic command

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/compose-and-basic_arch
Template resource:

docker_resource
docker/compose-restart_arch

Test compose a container with restart policy applied

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/compose-restart_arch
Template resource:

docker_resource
docker/compose-single_arch

Test docker compose with a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/compose-single_arch
Template resource:

docker_resource
docker/copy_arch

Test copy a file bwtween a container and local filesystem

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/copy_arch
Template resource:

docker_resource
docker/deploy-registry_arch

Deploy a registry server and run it on localhost

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/deploy-registry_arch
Template resource:

docker_resource
docker/diff_arch

Test changes to files in Ubuntu container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/diff_arch
Template resource:

docker_resource
docker/export-and-import_arch

Test docker import and export a docker container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/export-and-import_arch
Template resource:

docker_resource
docker/info

Display system-wide information about docker

Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
User:

root
Requires:

executable.name == ‘docker’
docker/inspect_arch

Test query low-level information on a docker object

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/inspect_arch
Template resource:

docker_resource
docker/interative_arch

Test an interactive shell in Ubuntu container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/interative_arch
Template resource:

docker_resource
docker/kill_arch

Test killing containers

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/kill_arch
Template resource:

docker_resource
docker/restart-always_arch

Test container restart policy with always applied

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/restart-always_arch
Template resource:

docker_resource
docker/restart-on-failure_arch

Test container restart policy with on_failure applied

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/restart-on-failure_arch
Template resource:

docker_resource
docker/run_arch

Download and run ubuntu container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/run_arch
Template resource:

docker_resource
docker/save-and-load_arch

Test docker save and load a docker image

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/save-and-load_arch
Template resource:

docker_resource
docker/start-stop_arch

Start and stop a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/start-stop_arch
Template resource:

docker_resource
docker/update_arch

Test update configuration of one container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/update_arch
Template resource:

docker_resource
docker/version

Display docker version information

Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
User:

root
Requires:

executable.name == ‘docker’

Eeprom

The following test units are covered in this category:

eeprom/read-write

Test EEPROM read and write functions for the system.

Category ID:

eeprom
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

shell
Requires:

manifest.has_eeprom == ‘True’
For details about the required manifest entry, see has_eeprom.

Error Detection And Correction (EDAC) Memory Controllers

The following test units are covered in this category:

edac/default-report

Attach the default report from EDAC util

Category ID:

edac
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

attachment
Requires:

executable.name == ‘edac-util’ manifest.has_edac_module == ‘True’
For details about the required manifest entry, see has_edac_module.
edac/detect

Detect if the EDAC drivers are loaded and Memory Controllers are found

Category ID:

edac
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
Requires:

executable.name == ‘edac-util’ manifest.has_edac_module == ‘True’
For details about the required manifest entry, see has_edac_module.

Ethernet Device tests

The following test units are covered in this category:

ethernet/wol_S3_interface

Wake on LAN (WOL) test from S3 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Non-blocking
Purpose:

Check that another system can wake the System Under Test (SUT) up from S3 using the Ethernet port {interface}’s WOL function.
From template:

ethernet/wol_S3_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’ and device.interface != ‘UNKNOWN’
ethernet/wol_S4_interface

Wake on LAN (WOL) test from S4 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Non-blocking
Purpose:

Check that another system can wake up from S4 the System Under Test (SUT) using ethernet port {interface} Wake-on-LAN (WOL) function.
From template:

ethernet/wol_S4_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’
ethernet/wol_S5_interface

Wake on LAN (WOL) test from S5 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Non-blocking
Purpose:

Check that another system can wake the System Under Test (SUT) from S5 using the Ethernet port {interface} WOL function.
From template:

ethernet/wol_S5_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’

Firmware tests

The following test units are covered in this category:

firmware/fwts_desktop_diagnosis

Run FWTS QA-concerned desktop-specific diagnosis tests.

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (fwts) QA-concerned desktop-specific diagnosis tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’
firmware/fwts_desktop_diagnosis_results.log.gz

Attach FWTS desktop diagnosis log to submission

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission
Plugin:

attachment
firmware/fwts_dump_acpi_attachment.gz

Collect the ACPI tables dump from FWTS

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

attachment

Gathers information about the DUT

The following test units are covered in this category:

connections

Collect information about connections

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of plug and slot connections on the device
Plugin:

resource
dpkg

Collect information about dpkg version

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Gets info on the version of dpkg installed
Plugin:

resource
meminfo

Collect information about system memory (/proc/meminfo)

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates resource info based on /proc/meminfo
Plugin:

resource
package

Collect information about installed software packages

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of packages
Plugin:

resource
rtc

Creates resource info for RTC

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

resource
serial_assertion

Collect serial assertions on the device

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Queries the snapd REST API for serial assertions present on the device.
Plugin:

resource
sleep

Create resource info for supported sleep states

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

resource
snap

Collect information about installed snap packages

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of snap packages
Plugin:

resource
uname

Collect information about the running kernel

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Creates resource info from uname output
Plugin:

resource

General Purpose I/O

The following test units are covered in this category:

gpio/gpiomem_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via /dev/gpiomem

Unit type:

template
Category ID:

gpio
Status:

Non-blocking
Purpose:

Ensure GPIO lines exposed on headers are controllable via /dev/gpiomem using the RPi python module for specific models.
After-suspend:

True
From template:

gpio/gpiomem_loopback_pairs_model
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)
gpio/sysfs_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template
Category ID:

gpio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

gpio/sysfs_loopback_pairs_model
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)
gpio/sysfs_loopback_pairs_vendor_product

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template
Category ID:

gpio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

gpio/sysfs_loopback_pairs_vendor_product
Template resource:

dmi
Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’, ‘UPN-EHL01’)

Hardware video acceleration

The following test units are covered in this category:

va-api/va-initialize

Detect if the VA API could be loaded

Category ID:

va-api
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

package.name == ‘vainfo’ cpuinfo.platform == ‘x86_64’ manifest.has_va_api == ‘True’
For details about the required manifest entry, see has_va_api.

Image verification tests

The following test units are covered in this category:

image/model-grade

Check that the model grade is correctly set

Category ID:

image
Status:

Non-blocking
Purpose:

Images with the ‘dangerous’ grade (the lowest of all available grades) results in certain security measures being relaxed. Images that require strict security-related implementations must have the model grade set to non-dangerous grades - either the highest grade of ‘secured’, or a grade passed to Checkbox for checking via the MODEL_GRADE configuration variable.
Plugin:

shell
Requires:

lsb.distributor_id == “Ubuntu Core” and int(lsb.release) >= 20 manifest._dangerous_grade_core_image == ‘False’
For details about the required manifest entry, see _dangerous_grade_core_image.

Informational tests

The following test units are covered in this category:

acpi_sleep_attachment

Attach the contents of /proc/acpi/sleep for further analysis.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the contents of /proc/acpi/sleep if it exists.
Plugin:

attachment
codecs_attachment

Attach a report of installed codecs for Intel HDA.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a report of installed codecs for Intel HDA
Plugin:

attachment
cpuinfo_attachment

Attach a copy of /proc/cpuinfo

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a report of CPU information
Plugin:

attachment
dkms_info_attachment

Attaches json dumps of installed dkms package information.

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches json dumps of installed dkms package information.
Plugin:

attachment
Requires:

package.name == ‘dkms’
dmesg_attachment

Attach a copy of dmesg or the current dmesg buffer to the test results.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a copy of /var/log/dmesg or the current dmesg buffer to the test results
User:

root
Plugin:

attachment
dmi_attachment

Attach a copy of /sys/class/dmi/id/*

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches info on DMI
Plugin:

attachment
Requires:

dmi_present.state == ‘supported’
dmidecode_attachment

Attach output of dmidecode

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches dmidecode output
User:

root
Plugin:

attachment
Requires:

executable.name == ‘dmidecode’ dmi_present.state == ‘supported’
dpkg_attachment

Attach dpkg -l output

Category ID:

info
Status:

Non-blocking
Purpose:

Attach system debian packages status
Plugin:

attachment
Requires:

executable.name == ‘dpkg’
efi_attachment

Attaches firmware version info

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the firmware version
User:

root
Plugin:

attachment
info/buildstamp

Attaches the buildstamp identifier for the OS

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the buildstamp identifier for the OS
Plugin:

attachment
info/disk_partitions

Attaches info about disk partitions

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches information about disk partitions
User:

root
Plugin:

attachment
info/hdparm_name.txt

Attaches info from hdparm about {name}

Unit type:

template
Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

info/hdparm_name.txt
Template resource:

device
Template filter:

device.category == ‘DISK’
info/kernel-config-iommu-flag

Check if the kernel is compiled with IOMMU support

Category ID:

info
Status:

Non-blocking
Purpose:

Checks the value of the CONFIG_INTEL_IOMMU_DEFAULT_ON flag in the kernel configuration
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”)
info/secure-boot-check

Check secure boot state

Category ID:

info
Status:

Non-blocking
Purpose:

Output whether secure boot is enabled or disabled
Plugin:

attachment
Requires:

package.name == ‘mokutil’ manifest.has_secure_boot == ‘True’
For details about the required manifest entry, see has_secure_boot.
info/systemd-analyze

System boot-up performance statistics

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
info/systemd-analyze-critical-chain

Print the tree of the time-critical chain of SystemD

Category ID:

info
Status:

Non-blocking
Purpose:

This job prints a tree of the time-critical chain of SystemD units.
Plugin:

shell
info/touchpad_driver

Gather touchpad name, driver name, and driver version information.

Category ID:

info
Status:

Non-blocking
Purpose:

Returns the name, driver name and driver version of any touchpad discovered on the system.
Plugin:

attachment
Requires:

device.category == ‘TOUCHPAD’
installer_debug.gz

Attach the installer’s debug log for diagnostic purposes.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the installer debug log if it exists.
User:

root
Plugin:

attachment
kernel_cmdline_attachment

Attach a copy of /proc/cmdline

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the kernel command line used to boot
Plugin:

attachment
lsmod_attachment

Attach a list of currently running kernel modules

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a list of the currently running kernel modules.
Plugin:

attachment
lspci_attachment

Attach a list of PCI devices

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches very verbose lspci output.
Plugin:

attachment
lspci_standard_config_attachment

Attach PCI configuration space hex dump

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a hex dump of the standard part of the PCI configuration space for all PCI devices.
Plugin:

attachment
lstopo_visual_attachment

Attach the output of lstopo command to present system topology.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the system topology as presented by the lstopo command
Plugin:

attachment
Requires:

executable.name == ‘lstopo’
lsusb_attachment

Attach output of lsusb

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a list of detected USB devices.
After-suspend:

True
User:

root
Plugin:

attachment
manifest

Hardware Manifest

Category ID:

info
Status:

Non-blocking
Purpose:

This job loads the hardware manifest and exposes it as a resource.
Plugin:

resource
meminfo_attachment

Attach copy of /proc/meminfo

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches info on system memory as seen in /proc/meminfo.
Plugin:

attachment
modinfo_attachment

Attach modinfo information

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches modinfo information for all currently loaded modules
Plugin:

attachment
modprobe_attachment

Attach the contents of /etc/modprobe.*

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of the various modprobe conf files.
User:

root
Plugin:

attachment
modules_attachment

Attach the contents of /etc/modules

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of the /etc/modules file.
Plugin:

attachment
net_if_management_attachment

Collect logging from the net_if_management job

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Allows logging of debug info and errors by the associated resource job
Plugin:

attachment
parts_meta_info_attachment

Attaches an information about all parts that constituted this snap

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

CHECKBOX_RUNTIME, SNAP
User:

root
Plugin:

attachment
sysctl_attachment

Attach sysctl configuration files.

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of various sysctl config files.
User:

root
Plugin:

attachment
sysfs_attachment

Attach detailed sysfs property output from udev

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches a report of sysfs attributes.
Plugin:

attachment
Requires:

cpuinfo.platform not in (“aarch64”)
udev_attachment

Attach dump of udev database

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches a dump of the udev database showing system hardware information.
Plugin:

attachment

Intel Integrated Sensor Hub Transport Protocol (ISHTP)

The following test units are covered in this category:

eclite/module-detect

Verifies that the ishtp_eclite module is loaded

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_eclite == ‘True’
For details about the required manifest entry, see has_eclite.
eclite/temperature-reading-from-thermal-acpitz

Read the temperature of acpitz to ensure Eclite is functional

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

For testing the feature of Eclite, one way is to monitor the temperature of acpitz thermal.
User:

root
Plugin:

shell
ishtp/device-detect

Verify that at least 1 device entry exists in /sys/bus/ishtp/devices

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
ishtp/module-detect

Verifies that the intel_ish_ipc module for ISHTP is loaded

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_ishtp == ‘True’
For details about the required manifest entry, see has_ishtp.

Intel Management Engine Interface (MEI)

The following test units are covered in this category:

mei/check-device

Detect if the Intel MEI device is available

Category ID:

mei
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”) manifest.has_mei == ‘True’
For details about the required manifest entry, see has_mei.
mei/check-module

Detect if the Intel MEI kernel module is loaded

Category ID:

mei
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”) manifest.has_mei == ‘True’
For details about the required manifest entry, see has_mei.
mei/get-firmware-version

Retrieve MEI firmware version by MEI interface

Category ID:

mei
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell

Kernel snap tests

The following test units are covered in this category:

kernel-snap/booted-kernel-matches-current-name

The booted kernel image matches the image in the current kernel snap

Unit type:

template
Category ID:

kernel-snap
Status:

Non-blocking
Purpose:

On some Ubuntu Core devices, it is necessary for the kernel image to be extracted from the kernel snap and placed in the boot partition (notably devices using full disk encryption). This checks the images are in sync.
From template:

kernel-snap/booted-kernel-matches-current-name
Template resource:

bootloader
Template filter:

bootloader.booted_kernel_path != ‘unknown’

LED tests

The following test units are covered in this category:

led-indicator/gpio-controller-leds-name

Check control of {name} LED indicator.

Unit type:

template
Category ID:

led
Status:

Non-blocking
Purpose:

Check that {name} LED turns on and off.
After-suspend:

True
From template:

led-indicator/gpio-controller-leds-name
Template resource:

led-indicator/gpio-controller-leds
led-indicator/gpio-leds-name

Check control of {name} LED indicator.

Unit type:

template
Category ID:

led
Status:

Non-blocking
Purpose:

Check that {name} LED turns on and off.
After-suspend:

True
From template:

led-indicator/gpio-leds-name
Template resource:

led-indicator/gpio-leds
led-indicator/sysfs-leds-automated

Automated check control of {name} LED.

Unit type:

template
Category ID:

led
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Automated check that {name} LED turns on and off.
After-suspend:

True
From template:

led-indicator/sysfs-leds-automated
Template resource:

led-indicator/sysfs-leds
led-indicator/sysfs-leds-manual

Check control of {name} LED.

Unit type:

template
Category ID:

led
Status:

Non-blocking
Purpose:

Check that {name} LED turns on and off.
After-suspend:

True
From template:

led-indicator/sysfs-leds-manual
Template resource:

led-indicator/sysfs-leds
led_indicator/validate-sysfs-leds-variable

Validate the environment variable SYS_LEDS

Category ID:

led
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

validate the format of SYS_LEDS environment variable is correct.
Environment variable:

SYS_LEDS
Plugin:

shell
Requires:

manifest.has_sysfs_led == ‘True’
For details about the required manifest entry, see has_sysfs_led.

Media Card tests

The following test units are covered in this category:

mediacard/sdxc-storage-manual

Test SDXC card insertion + read/write + removal.

Category ID:

mediacard
Status:

Non-blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Secure Digital Extended Capacity (SDXC) media card Then it performs a read and write test on the SDXC card. Finally, it checks that the system detects the removal of the SDXC card.
Steps:

  1. Commence the test and then insert an UNLOCKED SDXC card into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the SDXC card from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
Requires:

manifest.has_card_reader == ‘True’
For details about the required manifest entry, see has_card_reader.

Miscellaneous tests

The following test units are covered in this category:

miscellanea/device_check

Device Check

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Device check
Steps:

  1. Commence the test 2. Compare items on System Manifest to the devices known to udev
Verification:

Do the devices reported by udev match the devices on the Manifest?
Plugin:

user-interact-verify
miscellanea/oops_results.log

Attach the FWTS oops results for submission.

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Attaches the FWTS oops results log to the submission
Plugin:

attachment
miscellanea/secure_boot_mode_gadget

Test that {gadget} Ubuntu Core system booted with Secure Boot active

Unit type:

template
Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Test to verify that the system booted with Secure Boot active.
From template:

miscellanea/secure_boot_mode_gadget
Template resource:

model_assertion
miscellanea/submission-resources

Check that data for a complete result are present

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

A meta-job that verifies the data necessary for a complete result submission are present. Failure indicates that the results are incomplete and may be rejected.
Plugin:

shell

Monitor tests

The following test units are covered in this category:

monitor/dvi-to-vga

Monitor works (DVI-to-VGA)

Category ID:

monitor
Status:

Non-blocking
Purpose:

Check output to display through VGA adaptor on DVI port
Steps:

  1. Connect display to VGA adaptor on DVI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_dvi == ‘True’
For details about the required manifest entry, see has_dvi.
monitor/hdmi-to-vga

Monitor works (HDMI-to-VGA)

Category ID:

monitor
Status:

Non-blocking
Purpose:

Check output to display through VGA adaptor on HDMI port
Steps:

  1. Connect display to VGA adaptor on HDMI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_hdmi == ‘True’
For details about the required manifest entry, see has_hdmi.
monitor/vga

Monitor works (VGA)

Category ID:

monitor
Status:

Non-blocking
Purpose:

Check output to display through VGA port
Steps:

  1. Connect display to VGA port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_vga == ‘True’
For details about the required manifest entry, see has_vga.

Non-device specific networking tests

The following test units are covered in this category:

ipv6_detect

Test if the kernel is IPv6 ready

Category ID:

networking
Status:

Non-blocking
Purpose:

Test if the running kernel supports IPv6.
Steps:

<missing steps>
Verification:

<missing verification>
After-suspend:

True
networking/predictable_names

Verify that all network interfaces have predictable names.

Category ID:

networking
Status:

Non-blocking
Purpose:

Verify that all network interfaces have predictable names.
Plugin:

shell
Requires:

environment.CHECKBOX_RUNNING_STRICT_SNAP != “1” or lsb.release >= ‘20’ environment.CHECKBOX_RUNNING_STRICT_SNAP != “1” or model_assertion.gadget != “pi” environment.CHECKBOX_RUNNING_STRICT_SNAP == “1” or lsb.release >= ‘18’

Power Management tests

The following test units are covered in this category:

power-management/fwts_wakealarm-log-attach

Attach and display fwts wakealarm test log.

Category ID:

power-management
Status:

Non-blocking
Purpose:

Attach log from fwts wakealarm test
Plugin:

attachment
watchdog/detect

Detect the presence of a Hardware Watchdog

Category ID:

power-management
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
Requires:

manifest.has_hardware_watchdog == ‘True’
For details about the required manifest entry, see has_hardware_watchdog.
watchdog/post-trigger-system-reset-auto

Post watchdog reset service check

Category ID:

power-management
Status:

Non-blocking
Purpose:

Check there are no failed services after the watchdog triggered
User:

root
Plugin:

shell
Requires:

manifest.has_hardware_watchdog == ‘True’
For details about the required manifest entry, see has_hardware_watchdog.
watchdog/systemd-config

Check if the hardware watchdog is properly configured

Category ID:

power-management
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
Requires:

manifest.has_hardware_watchdog == ‘True’
For details about the required manifest entry, see has_hardware_watchdog.
watchdog/trigger-system-reset-auto

Test that the watchdog module can trigger a system reset

Category ID:

power-management
Status:

Non-blocking
Purpose:

Ensure that the watchdog module can successfully initiate a system reset.
User:

root
Plugin:

shell

Quadrature Encoder Peripherals

The following test units are covered in this category:

qep/qep-device-driver-for-qep-device

Verify PCI Device {qep-device} is using the correct driver

Unit type:

template
Category ID:

qep
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Checks that the device exists in lspci and that the device driver associated is correct. Device ID should be 8086:{qep-device} and the driver is always intel_qep.
From template:

qep/qep-device-driver-for-qep-device
Template resource:

qep/qep-devices
qep/qep-device-node-for-qep-device

Verify device directory exists for {qep-device}

Unit type:

template
Category ID:

qep
Status:

Non-blocking
Purpose:

Detects if the device’s directory exists.
From template:

qep/qep-device-node-for-qep-device
Template resource:

qep/qep-devices

Real Time Clock (RTC)

The following test units are covered in this category:

rtc/rtc_alarm_rtc

Check that RTC alarm of {rtc} works

Unit type:

template
Category ID:

rtc_category
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

rtc/rtc_alarm_rtc
Template resource:

rtc_list
rtc/rtc_clock_rtc

Check that {rtc} clock is synchronized with system clock

Unit type:

template
Category ID:

rtc_category
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

rtc/rtc_clock_rtc
Template resource:

rtc_list
rtc/rtc_number

Check the number of RTC as expected.

Category ID:

rtc_category
Status:

Non-blocking
Purpose:

Check the number of RTC on platform as expected.
After-suspend:

True
Environment variable:

TOTAL_RTC_NUM
User:

root
Plugin:

shell

Snapd

The following test units are covered in this category:

snappy/snap-install

Test the snap install command is working

Category ID:

snapd
Status:

Non-blocking
Purpose:

The store should contain the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap. The test makes sure this can be downloaded and installed on the system.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-list

Test that the snap list command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

If snap list command is working then should at least find the ubuntu-core package.
Plugin:

shell
snappy/snap-refresh-automated

Test whether the snap refresh command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

The test will install the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap from the stable channel and then refreshes it to the edge channel and compares the revision before and after the refresh.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-remove

Test the snap remove command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

After having installed the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap, check it can be removed.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-reupdate-automated

Test the snap refresh command works after blacklisting.

Category ID:

snapd
Status:

Non-blocking
Purpose:

Checks that the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap can be refreshed after removal of the blacklisted revision.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-revert-automated

Test the snap revert command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

Checks if the edge channel {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap is reverted back to the one from stable.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-snaps-confinement

Test all the snaps’ confinement

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test all the snaps’ confinement, devmode, revision. Make sure the confinement is “strict”, devmode is “False”, and revision should not be sideloaded.
Plugin:

shell
snappy/test-store-config-store

Test that image is using the correct snappy store configuration.

Unit type:

template
Category ID:

snapd
Status:

Non-blocking
Purpose:

The image can be tied to using a particular store for the OEM. This tests the store for the image is as expected.
From template:

snappy/test-store-config-store
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.store not in (‘unknown’)
snappy/test-store-install-beta

Snappy install command - beta channel store

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test the snappy install command is able to install and remove snap in beta channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-store-install-edge

Snappy install command - edge channel store

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test the snappy install command is able to install a snap in the edge channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-system-confinement

Test if the system confinement is strict

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test if the system confinement is “strict” If not, list the missing features
Plugin:

shell

SocketCAN interface tests

The following test units are covered in this category:

socketcan/send_packet_local_eff_virtual

Virtual CAN device support test (Local test with raw socket and EFF)

Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_eff_interface

CAN device support test for {interface} (Raw, Local, EFF)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket, this is only a local test as the broadcast packet is received on the same device.
After-suspend:

True
From template:

socketcan/send_packet_local_eff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_local_fd_virtual

Virtual CAN device support test (Raw, Local, FD)

Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket; this is only a local test as the broadcast packet is received on the same device.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

manifest.has_socket_can_fd == ‘True’
For details about the required manifest entry, see has_socket_can_fd.
socketcan/send_packet_local_fd_interface

CAN device support test for {interface} (Raw, Local, FD)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
From template:

socketcan/send_packet_local_fd_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_local_sff_virtual

Virtual CAN device support test (Raw, Local)

Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_sff_interface

CAN device support test for {interface} (Raw, Local)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
From template:

socketcan/send_packet_local_sff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_eff_interface

CAN device support test {interface} (Raw, Remote, EFF)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket to a remote device. As a prerequisite the remote device should have can-echo-server installed so as to return the predicted packet.
After-suspend:

True
From template:

socketcan/send_packet_remote_eff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_fd_interface

CAN device support test {interface} (Raw, Remote, FD)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket to a remote device. As a prerequisite, the remote device should have can-echo-server installed so as to return the predicted packet.
After-suspend:

True
From template:

socketcan/send_packet_remote_fd_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_sff_interface

CAN device support test {interface} (Raw, Remote)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket to a remote device. As a prerequisite, the remote device should have can-echo-server installed so as to return the predicted packet.
After-suspend:

True
From template:

socketcan/send_packet_remote_sff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’

Suspend tests

The following test units are covered in this category:

suspend/any_suspend_failure

Tests if any device in this machine reports a suspend failure

Category ID:

suspend
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

If the DUT suspended successfully, /sys/power/suspend_stats/fail* and fail* in /sys/kernel/debug/suspend_stats will be zero. If a device failed /sys/power/suspend_stats/last_failed_dev and last_failed_dev in /sys/kernel/debug/suspend_stats will show the failed device
User:

root
Plugin:

shell
suspend/oops_results_after_suspend.log

Attach FWTS oops results log post-suspend.

Category ID:

suspend
Status:

Non-blocking
Purpose:

Attaches the FWTS oops results log to the submission after suspend
Plugin:

attachment

TPM 2.0 (Trusted Platform Module)

The following test units are covered in this category:

tpm2/fwts-event-log-dump

Dump the contents of the TPM Event Log

Category ID:

tpm2
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

The information in the TPM Event Log can be useful in debugging problems with TPM command support and adherance to standards. This can be of particular help when checking whether a device can support Ubuntu Core Full Disk Encryption.
User:

root
Plugin:

attachment
Requires:

manifest.has_tpm2_chip == ‘True’ executable.name == ‘fwts’
For details about the required manifest entry, see has_tpm2_chip.

Ubuntu Core OS feature tests

The following test units are covered in this category:

ubuntucore/kernel-failboot-kernel

Reboot with failboot kernel to verify system rollback function

Unit type:

template
Category ID:

ubuntucore
Status:

Non-blocking
Purpose:

Check if the system will rollback to the previous (functional) kernel
From template:

ubuntucore/kernel-failboot-kernel
Template resource:

model_assertion
ubuntucore/os-fail-boot-description

Automatically rollback after failed boot after upgrade

Unit type:

template
Category ID:

ubuntucore
Status:

Non-blocking
Purpose:

Check the system will rollback to the original core snap if it fails to boot the updated one.
From template:

ubuntucore/os-fail-boot-description
Template resource:

lsb
Template filter:

lsb.distributor_id == ‘Ubuntu Core’
ubuntucore/os-recovery-mode

Reboot into recovery mode and log into the system using prior credentials.

Category ID:

ubuntucore
Status:

Non-blocking
Purpose:

Check if the system will reboot to recovery mode successfully
Steps:

  1. Reboot the system to recovery mode $ sudo snap reboot –recover 2. The system should respond to the command and reboot itself 3. Wait until the system completes the reboot process 4. Check if the system is running in recovery mode $ cat /proc/cmdline 5. Reboot the system back to normal run mode $ sudo snap reboot 6. Check again if the system is running in run mode $ cat /proc/cmdline
Verification:

Check if kernel cmdline when the system is in recovery mode includes: ‘snapd_recovery_mode=recover’ Check if kernel cmdline when the system is in normal run mode includes: ‘snapd_recovery_mode=run’
Plugin:

manual
Requires:

lsb.release >= ‘20’
ubuntucore/os-reinstall-mode

Reboot into reinstall mode and trigger a factory reset on the device.

Category ID:

ubuntucore
Status:

Non-blocking
Purpose:

Check if the system will reboot to reinstall mode and reinitialize the device with a fresh factory reset.
Steps:

WARNING: ALL EXISTING DATA ON THIS DEVICE WILL BE WIPED!! 1. Check the current serial-assertion device-key $ ls /var/lib/snapd/save/device/private-keys-v1 2. Clear TPM first if this device has enabled secure boot & FDE For x86-based platforms: $ sudo su $ echo 5 > /sys/class/tpm/tpm0/ppi/request For ARM-based platforms: There is no generic command for ARM-based platforms, please refer to the device user manual. 3. Reboot the system to reinstall mode $ sudo snap reboot –install 4. The system should respond to the command and reboot itself. 5. Wait until the system completes the installation and initialization process. 6. Check the serial-assertion device-key after installation completes. $ ls /var/lib/snapd/save/device/private-keys-v1
Verification:

Check if a new serial-assertion device-key got generated after reinstallation completes.
Plugin:

manual
Requires:

lsb.release >= ‘20’
ubuntucore/sshd

SSH is enabled and operational

Category ID:

ubuntucore
Status:

Non-blocking
Purpose:

Check if a user can access the system through SSH from another machine
Steps:

  1. Execute the following command on another machine in the same network: $ ssh [user id]@[ip address of the testing system] 2. Enter the password to log in
Verification:

Can log into the system through SSH from another machine
Plugin:

manual

USB tests

The following test units are covered in this category:

usb-c-otg/g_ether

Check DUT can be detected as USB ethernet device

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

  1. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT. 2. Press Enter to configure the IP address for the USB OTG ethernet interface on the DUT. (IP will be 192.168.9.1/24) 3. On the host, configure the IP address for the USB OTG ethernet interface. It should be in the same subnet as the DUT. (e.g., 192.168.9.2/24) 4. Try to ping between the host and DUT.
Verification:

The host and DUT can ping each other.
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_ether-cleanup

Cleanup USB OTG ethernet interface setup after ethernet device test

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_mass_storage

Check DUT can be detected as a mass storage device

Category ID:

usb
Status:

Non-blocking
Purpose:

Check that after connecting the device under test (DUT) to another device (host), the DUT can be detected as a mass storage device by the host.
Steps:

  1. Press Enter to setup a 16 MB FAT32 image on the device. 2. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT.
Verification:

The host detects and mounts the mass storage device. It has read and write permissions on it.
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_mass_storage-cleanup

Cleanup mass storage setup after mass storage device test

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_serial

Check if USB OTG can work as a serial port.

Category ID:

usb
Status:

Non-blocking
Purpose:

Check if USB OTG can work as a serial port.
Steps:

  1. Press enter to probe USB OTG g_serial 2. Connect a USB cable from an Ubuntu host to the USB OTG (it’s the USB Type C connector) port on DUT. 3. On the host, check if ttyACM0 has been probed $ ls /dev/ttyACM0 4. On the host, listen to the ttyACM0 port as the receiving side. $ sudo su $ cat /dev/ttyACM0 5. On DUT, send a string via /dev/ttyGS0 as the sending side. $ sudo su $ echo 123 > /dev/ttyGS0 6. Check if the string “123” was received on the host side. 7. Check the string sending from the host to DUT by repeating steps 4-6, but swap the send and receive sides.
Verification:

Does string send and receive function correctly?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_serial-cleanup

Cleanup USB OTG serial interface setup after serial device test

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c/c-to-a-adapter/hid

USB HID work on USB Type-C port using a “USB Type-C to Type-A” adapter

Category ID:

usb
Status:

Non-blocking
Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port using a “USB Type-C to Type-A” adapter
Steps:

  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port using a “USB Type-C to Type-A” adapter 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text
Verification:

Did the device work as expected?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.
usb-c/c-to-a-adapter/storage-manual

Test USB 3 storage device insertion + read/write + removal using a “Type-C to Type-A” adapter.

Category ID:

usb
Status:

Non-blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3 storage device in a USB Type-C connector using a “Type-C to Type-A” adapter. Then it performs a read and write test on the device. Finally, it checks that the system correctly detects the removal of the USB 3 storage. NOTE: Make sure the USB 3 storage device has a partition before starting the test.
Steps:

  1. Commence and then insert a USB 3 storage device to a USB Type-C port using a “Type-C to Type-A” adapter. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 3 storage from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

usb.usb3 == ‘supported’ manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.

Wi-Fi access point

The following test units are covered in this category:

wireless/nmcli_wifi_ap_a_interface

Create 802.11a Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

wireless/nmcli_wifi_ap_a_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/nmcli_wifi_ap_bg_interface

Create 802.11b/g Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Create an 802.11b/g Wi-Fi access point on a specified interface using NetworkManager command-line tools.
After-suspend:

True
From template:

wireless/nmcli_wifi_ap_bg_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_b_no_sta_interface_auto

Create open 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11b Access Point without any STA connection on {interface} by configuring the system using wifi-ap snap and then checking status of the interface using iw command.
After-suspend:

True
From template:

wireless/wifi_ap_open_b_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_b_no_sta_interface_manual

Create open 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11b Access Point without any STA connection
After-suspend:

True
From template:

wireless/wifi_ap_open_b_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_g_no_sta_interface_auto

Create an open 802.11g Wi-Fi AP on {interface} with no STA connected.

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11g Access Point without any Station (STA) connection on {interface} by configuring the system using the wifi-ap snap and then checking the status of the interface using the iw command.
After-suspend:

True
From template:

wireless/wifi_ap_open_g_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_g_no_sta_interface_manual

Create open 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11g Access Point without any STA connection
After-suspend:

True
From template:

wireless/wifi_ap_open_g_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_setup_wizard_interface_auto

Create Access Point on {interface} using wifi-ap.setup-wizard

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create a WPA2 802.11g Access Point using wifi-ap.setup-wizard command on {interface}.
After-suspend:

True
From template:

wireless/wifi_ap_setup_wizard_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_no_sta_interface_auto

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11b Access Point without any STA connection on {interface} by configuring the system using wifi-ap snap and then checking the status of the interface using iw command.
After-suspend:

True
From template:

wireless/wifi_ap_wpa_b_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_no_sta_interface_manual

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create a WPA2 802.11b Access Point without any STA connected
After-suspend:

True
From template:

wireless/wifi_ap_wpa_b_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_with_sta_interface_auto

Create a WPA2 802.11b Wi-Fi Access Point on {interface} with active STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

wireless/wifi_ap_wpa_b_with_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_no_sta_interface_auto

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11g Access Point without any Station (STA) connection on {interface} by configuring the system using the wifi-ap snap and then checking the status of the interface using the iw command.
After-suspend:

True
From template:

wireless/wifi_ap_wpa_g_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_no_sta_interface_manual

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create a WPA2 802.11g Access Point without any STA connection
After-suspend:

True
From template:

wireless/wifi_ap_wpa_g_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_with_sta_interface_auto

Create WPA2 802.11g Wi-Fi Access Point on {interface} with active STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

wireless/wifi_ap_wpa_g_with_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless networking tests

The following test units are covered in this category:

wireless/check_iwlwifi_microcode_crash_interface

Check there have been no iwlwifi crashes

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
Purpose:

Ensure no crashes have occurred in the iwlwifi microcode.
After-suspend:

True
From template:

wireless/check_iwlwifi_microcode_crash_interface
Template resource:

device
Template filter:

device.driver == ‘iwlwifi’
wireless/wifi7_mlo_interface

Test Wi-Fi 7 Multi-Link-Operation (MLO) on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
Purpose:

Check if the DUT supports Wi-Fi 7 features like Multi-Link-Operation
After-suspend:

True
From template:

wireless/wifi7_mlo_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless Wide Area Network

The following test units are covered in this category:

wwan/3gpp-scan-manufacturer-model-hw_id-auto

Scan for available 3GPP networks with the {model} modem

Unit type:

template
Category ID:

wwan
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Scan for available 3GPP networks with the target modem
After-suspend:

True
From template:

wwan/3gpp-scan-manufacturer-model-hw_id-auto
Template resource:

wwan_resource

Manifest Entries

The following manifest entries are required for certification:

_dangerous_grade_core_image

Image is using ‘dangerous’ grade (should be set to ‘No’ unless you’re doing SRU testing)

Value type:

bool
Prompt:

Does this setup have the following configuration?
_ignore_disconnected_ethernet_interfaces

Ignore disconnected Ethernet interfaces

Value type:

bool
gpio_loopback

GPIO Loopback Connector

Value type:

bool
Prompt:

Does this device have the following?:
has_apste_support

Disk with Autonomous Power State Transition (APST) support

Value type:

bool
has_audio_capture

Audio capture: Machine can record sound. (For example, a desktop PC probably requires a microphone connected to it)

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_audio_loopback_connector

Audio Loopback Connector

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_audio_playback

Audio playback: Machine can emit sound. (For example, a desktop PC probably requires speakers connected to it)

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_bt_adapter

A Bluetooth Module

Value type:

bool
has_bt_obex_support

A Bluetooth Module with OBject EXchange (OBEX) Support

Value type:

bool
has_bt_smart

A Bluetooth Module with Smart (4.0 or later) Support

Value type:

bool
has_card_reader

Media Card Reader

Value type:

bool
has_dp

DisplayPort

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_dvi

DVI

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_eclite

Has support for Intel ISHTP eclite controller Driver (ECLITE)

Value type:

bool
has_edac_module

An Error Detection And Correction (EDAC) Module

Value type:

bool
has_eeprom

Does device supported EEPROM?

Value type:

bool
has_ethernet_adapter

An Ethernet Port

Value type:

bool
has_ethernet_wake_on_lan_support

Wake-on-LAN support through Ethernet port

Value type:

bool
has_fde

Full Disk Encryption (FDE)

Value type:

bool
has_hardware_watchdog

A Hardware Watchdog Timer

Value type:

bool
has_hdmi

HDMI

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_i2c

An I²C bus

Value type:

bool
has_ishtp

Has support for Intel Integrated Sensor Hub Tranport Protocol (ISHTP)

Value type:

bool
has_led_gpio_sysfs

LEDs controlled via GPIO or sysfs

Value type:

bool
Prompt:

Does this machine have the following LED indicators?
has_md_raid

Software RAID

Value type:

bool
has_mei

Has support for Intel Management Engine Interface (MEI)

Value type:

bool
has_qep

Has support for Quadrature Encoder Peripherals

Value type:

bool
has_secure_boot

Secure Boot support

Value type:

bool
has_sim_card

A working SIM card inserted

Value type:

bool
has_socket_can_fd

CAN FD capable devices

Value type:

bool
has_thunderbolt

Thunderbolt Support

Value type:

bool
has_thunderbolt3

Thunderbolt 3 Support

Value type:

bool
has_touchscreen

Touchscreen

Value type:

bool
has_touchscreen_multitouch

Touchscreen Multi-touch Support

Value type:

bool
has_tpm2_chip

TPM 2.0 Support

Value type:

bool
has_usb_dwc3_controller

A DesignWare Core SuperSpeed USB 3.0 Controller (DWC3)

Value type:

bool
has_usb_storage

USB Storage Device Connected

Value type:

bool
has_usbc_data

USB Type-C Data (e.g. HID, Drives, Ethernet)

Value type:

bool
has_usbc_otg

Does the platform support USB-C OTG?

Value type:

bool
has_va_api

Has support for hardware video acceleration (VA API)

Value type:

bool
has_vga

VGA

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_wifi_mlo_environment

Does the test environment have a WiFi 7 router with tri-band (2.4, 5, 6) multi-link-operation (MLO) enabled and are this router’s ssid and password specified in the environment variables MLO_SSID/MLO_PSK?

Value type:

bool
has_wlan_adapter

A Wi-Fi Module

Value type:

bool
has_wwan_module

A WWAN Module

Value type:

bool
socket_can_echo_server_running

A SocketCAN Echo Server

Value type:

bool
Prompt:

Is the device connected to the following?:

client-cert-iot-server-26-04

Note

The certification tests presented in this document are validated by Checkbox version 7.3.0.dev42.

Blocking

Audio tests

Output needs to be undistorted between 0%-100%. Output lines tested:

  • Internal speakers

  • 3.5mm headphones

  • HDMI audio output

  • DisplayPort audio output

Input needs to be recorded undistorted between 0%-100%. Input lines tested:

  • Internal microphone

  • 3.5mm microphone

Plug detection: when a new audio line input or output is plugged in the system, it needs to be recognized.

The following test units are covered in this category:

audio/alsa-loopback-automated

Captured sound matches played one (automated)

Category ID:

audio
Status:

Blocking
Purpose:

Check if sound that is ‘hearable’ by capture device. This test is intended mostly for devices without internal speakers/microphones, but with a line-in and a line-out ports to which a loopback cable can be connected.
After-suspend:

True
Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH, LD_LIBRARY_PATH
Plugin:

shell
Requires:

manifest.has_audio_loopback_connector == ‘True’
For details about the required manifest entry, see has_audio_loopback_connector.
audio/alsa-playback

Playback works

Category ID:

audio
Status:

Blocking
Purpose:

Check if sound is played through ALSA on the default output
Steps:

  1. Make sure speakers or headphones are connected to the device 2. Commence the test
Verification:

Did you hear the sound?
After-suspend:

True
Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH
Plugin:

user-interact-verify
audio/playback_displayport

<missing summary>

Category ID:

audio
Status:

Blocking
Purpose:

DisplayPort audio interface verification
Steps:

  1. Plug an external DisplayPort device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Commence the test
Verification:

Did you hear the sound from the DisplayPort device?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_dp == ‘True’ manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_dp, has_audio_playback.
audio/playback_hdmi

<missing summary>

Category ID:

audio
Status:

Blocking
Purpose:

HDMI audio interface verification
Steps:

  1. Plug an external HDMI device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Commence the test
Verification:

Did you hear the sound from the HDMI device?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_hdmi == ‘True’ manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_hdmi, has_audio_playback.
audio/playback_thunderbolt3

<missing summary>

Category ID:

audio
Status:

Blocking
Purpose:

Thunderbolt audio interface verification
Steps:

  1. Plug an external Thunderbolt device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Commence the test
Verification:

Did you hear the sound from the Thunderbolt device?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_thunderbolt3 == ‘True’ manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_thunderbolt3, has_audio_playback.
audio/playback_type-c_displayport

<missing summary>

Category ID:

audio
Status:

Blocking
Purpose:

DisplayPort audio via USB Type-C port interface verification
Steps:

  1. Plug an external DisplayPort device with sound on a USB Type-C port using a “USB Type-C to DisplayPort” adapter (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Commence the test
Verification:

Did you hear the sound from the DisplayPort device?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_video == ‘True’ manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_usbc_video, has_audio_playback.
audio/playback_type-c_hdmi

Verify HDMI audio playback through the USB Type-C port using a “USB Type-C to HDMI” adapter and confirm sound output.

Category ID:

audio
Status:

Blocking
Purpose:

HDMI audio via USB Type-C port interface verification
Steps:

  1. Plug an external HDMI device with sound into a USB Type-C port using a “USB Type-C to HDMI” adapter (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Begin the test
Verification:

Did you hear the sound from the HDMI device?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_video == ‘True’ manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_usbc_video, has_audio_playback.

Bluetooth tests

Bluetooth LE (Smart and Smart Ready) is tested for device scanning and pairing. Apart from pairing, several profiles are specifically tested and required:

  • Eddystone Beacon

  • HID Over GATT Profile (HOGP), Low-Energy keyboard or mouse with basic functionality

The following test units are covered in this category:

bluetooth/audio-a2dp

Verify Bluetooth device’s High Fidelity Playback (A2DP) capability.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check the High Fidelity Playback (A2DP) capability of your Bluetooth device, to see if you can hear audio from it.
Steps:

  1. Enable and pair the Bluetooth headset 2. Click “Test” to play a brief tone on your Bluetooth device, if it failed to set the Mode to A2DP, please select the device and change it manually in the “Sound Settings”
Verification:

Did you hear the tone?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_bt_smart == ‘True’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_bt_smart.
bluetooth/audio_record_playback

Verify Bluetooth HSP/HFP profile capability for recording and playback.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check the Headset Head Unit (HSP/HFP) capability of your Bluetooth device, to check if you can record sounds.
Steps:

  1. Enable and pair the bluetooth headset. 2. Click “Test”, then speak into your Bluetooth microphone. 3. After a few seconds, your speech will be played back to you.
Verification:

Did you hear your speech played back?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_bt_smart == ‘True’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_bt_smart.
bluetooth/bluetooth_obex_send

Bluetooth OBEX send

Category ID:

bluetooth
Status:

Blocking
Purpose:

This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable
After-suspend:

True
Environment variable:

BTDEVADDR, PLAINBOX_PROVIDER_DATA
Plugin:

shell
Requires:

package.name == ‘bluez’ or snap.name == ‘bluez’ executable.name == ‘obexftp’ and executable.name == ‘hcitool’ device.category == ‘BLUETOOTH’ manifest.has_bt_obex_support == ‘True’
For details about the required manifest entry, see has_bt_obex_support.
bluetooth/detect

Make sure at least one bluetooth device is detected

Category ID:

bluetooth
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_bt_adapter == ‘True’
For details about the required manifest entry, see has_bt_adapter.
bluetooth/detect-output

Store Bluetooth device information for reports.

Category ID:

bluetooth
Status:

Blocking
Purpose:

Automated test to store Bluetooth device information in the Checkbox report
After-suspend:

True
Plugin:

shell
Requires:

package.name == ‘bluez’ or snap.name == ‘bluez’ device.category == ‘BLUETOOTH’
bluetooth4/HOGP-keyboard

Verify HOGP keyboard functionality with Bluetooth Smart.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart keyboard.
Steps:

  1. Enable a Bluetooth Smart keyboard, and put it into pairing mode. 2. Commence the test to do the auto-pairing, you will be asked to select the targeting keyboard from the list. 3. After it’s paired and connected, enter some text with your keyboard.
Verification:

Did the Bluetooth Smart keyboard work as expected?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_bt_smart == ‘True’ package.name == ‘bluez’
For details about the required manifest entry, see has_bt_smart.

CPU tests

x86_64 and ARM processors are tested to ensure proper functionality. We will test specific features as:

  • CPU’s performance states (frequency up and down in runtime)

  • CPU’s sleep states (cpu on and off in runtime)

  • Running CPU at its maximum frequency

We will also include a general stress test performed for 120 minutes to verify that the system can handle a sustained high load for a period of time. This test uses the tool “stress-ng” available in the Universe repositories.

For Intel CPU’s, the IPDT (Intel Processor Diagnostic Tool) test suite will be run. The diagnostic checks for brand identification, verifies the processor operating frequency, tests specific processor features, and performs a stress test on the processor.

The following test units are covered in this category:

cpu/cstates

Run C-States tests

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Uses the Firmware Test Suite (fwts) to test the power saving states of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’ cpuinfo.platform not in (“aarch64”, “armv7l”, “ppc64el”, “ppc64le”, “s390x”)
cpu/maxfreq_test

Test that the CPU can run at its max frequency

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.
Environment variable:

LD_LIBRARY_PATH, SNAP
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’ cpuinfo.platform in (“i386”, “x86_64”)
cpu/offlining_test

Test offlining of each CPU core

Category ID:

cpu
Status:

Blocking
Purpose:

Attempts to offline each core in a multicore system.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

cpu_offlining.state == ‘supported’
cpu/scaling_test

Test the CPU scaling capabilities

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use Firmware Test Suite (fwts cpufreq) to test the scaling capabilities of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’ ‘userspace’ in cpuinfo.governors cpuinfo.platform not in (“ppc64el”, “ppc64le”, “s390x”)
cpu/topology

Check CPU topology for accuracy between proc and sysfs

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.
After-suspend:

True
Plugin:

shell
Requires:

int(cpuinfo.count) > 1 and (cpuinfo.platform == ‘i386’ or cpuinfo.platform == ‘x86_64’)

Disk tests

The following test units are covered in this category:

disk/read_performance_name

Disk performance test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
Purpose:

Verify that disk storage performs at or above baseline performance
After-suspend:

True
From template:

disk/read_performance_name
Template resource:

device
Template filter:

device.category == ‘DISK’
disk/storage_device_name

Disk I/O stress test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
Purpose:

Take the path of the storage device and test if it is a block device.
After-suspend:

True
From template:

disk/storage_device_name
Template resource:

device
Template filter:

device.category == ‘DISK’
thunderbolt3/daisy-chain

Daisy-chain testing for Thunderbolt 3 storage and display device

Category ID:

disk
Status:

Blocking
Purpose:

This test will check if your system can support daisy-chaining of a storage and a monitor over Thunderbolt 3 port
Steps:

  1. Connect your Thunderbolt monitor to your system 2. Connect and mount your Thunderbolt HDD to another Thunderbolt 3 port of the monitor (you can do this with HDD first as well) 3. Click ‘Test’ to perform the storage test on the Thunderbolt HDD
Verification:

  1. The verification for storage is automated, please select the result combined with the result for the display. 2. Was the desktop displayed correctly on the Thunderbolt-connected screen?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

manifest.has_thunderbolt3 == ‘True’
For details about the required manifest entry, see has_thunderbolt3.
thunderbolt3/storage-manual

Thunderbolt 3 HDD storage insertion + read/write + removal

Category ID:

disk
Status:

Blocking
Purpose:

This test will check if the connection of a Thunderbolt 3 HDD could be detected, then performs read/write operations on the attached Thunderbolt 3 HDD storage and checks if the removal of the Thunderbolt 3 HDD can be detected.
Steps:

  1. Click ‘Test’ to begin the test. This test will timeout and fail if the insertion has not been detected within 30 seconds. 2. Plug a Thunderbolt 3 HDD into an available Thunderbolt 3 port, if it’s not mounted automatically, please click the HDD icon to mount it. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the previously attached Thunderbolt 3 HDD.
Verification:

The verification of this test is automated. Do not change the automatically selected result
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

manifest.has_thunderbolt3 == ‘True’
For details about the required manifest entry, see has_thunderbolt3.

Ethernet Device tests

Connections are tested for functionality, but not for performance.

The following test units are covered in this category:

ethernet/detect

Detect if at least one ethernet device is detected

Category ID:

ethernet
Status:

Blocking
Purpose:

Test to detect and return information about available network controllers on the system under test.
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_ethernet_adapter == ‘True’
For details about the required manifest entry, see has_ethernet_adapter.
ethernet/hotplug-interface

Ensure hotplugging works on port {{ interface }}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

Check that hotplugging works on port {{ interface }}
After-suspend:

True
From template:

ethernet/hotplug-interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/iperf3_reverse_interface

Iperf3 stress testing for {interface} (reverse)

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

This test uses iperf3 to ensure network devices pass data at an acceptable minimum percentage of advertised speed (Reverse).
From template:

ethernet/iperf3_reverse_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/iperf3_interface

Iperf3 stress testing for {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

This test uses iperf3 to ensure network devices pass data at an acceptable minimum percentage of advertised speed.
From template:

ethernet/iperf3_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/ping-with-any-cable-interface

Can ping the gateway with any cable Ethernet interface

Category ID:

ethernet
Status:

Blocking
Purpose:

Check any of the cable Ethernet ports available on the system can ping its default gateway.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

device.category == ‘NETWORK’
ethernet/ping_interface

Can ping another machine over Ethernet port {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

Check Ethernet works by pinging another machine
After-suspend:

True
From template:

ethernet/ping_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’

Graphics tests

The following test units are covered in this category:

graphics/cycle_resolution_primary

Test resolution cycling

Category ID:

graphics
Status:

Blocking
Purpose:

This test cycles through the detected video modes for the graphics card
Steps:

  1. Click “Test” to start cycling through the video modes
Verification:

Did the screen appear to be working for each mode?
After-suspend:

True
Environment variable:

XDG_CURRENT_DESKTOP, XDG_SESSION_TYPE
Plugin:

user-interact-verify
Requires:

package.name == ‘xorg’
graphics/maximum_resolution_primary

Test maximum supported resolution

Category ID:

graphics
Status:

Blocking
Purpose:

This test will verify the maximum supported resolution on the primary graphics card.
Steps:

  1. Consult the system’s specifications and locate the screen’s maximum supported resolution. 2. Click on Test to display the maximum resolution that can be used by Ubuntu on the current display.
Verification:

Is this the maximum resolution for the display connected to the graphics card?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

device.category == ‘VIDEO’
graphics/rotation_primary

Test rotation

Category ID:

graphics
Status:

Blocking
Purpose:

This test will test display rotation on the graphics card
Steps:

  1. Click “Test” to test display rotation. The display will be rotated every 4 seconds. 2. Try moving the mouse or try opening multiple terminals via ‘Ctrl+Alt+T’ every time the screen automatically turns. 3. Check if all rotations (normal, right, inverted, left) took place without permanent screen corruption.
Verification:

Did the display rotation take place without permanent screen corruption?
After-suspend:

True
Environment variable:

XDG_CURRENT_DESKTOP, XDG_SESSION_TYPE
Plugin:

user-interact-verify
graphics/video_primary

Test that video can be displayed

Category ID:

graphics
Status:

Blocking
Purpose:

This test will test the default display with a sample video
Steps:

  1. Click “Test” to display a video test.
Verification:

Do you see color bars and static?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’
graphics/index_valid_glxgears_fullscreen_product_slug

Test that glxgears works in full screen mode for {vendor} {product} (manual)

Unit type:

template
Category ID:

graphics
Status:

Blocking
Purpose:

This test tests the basic fullscreen 3D capabilities of your {vendor} {product} video card
After-suspend:

True
From template:

graphics/index_valid_glxgears_fullscreen_product_slug
Template resource:

graphics_card
graphics/index_valid_glxgears_product_slug

Test that glxgears works for {vendor} {product} (manual)

Unit type:

template
Category ID:

graphics
Status:

Blocking
Purpose:

This test tests the basic 3D capabilities of your {vendor} {product} video card
After-suspend:

True
From template:

graphics/index_valid_glxgears_product_slug
Template resource:

graphics_card

I²C (Inter-Integrated Circuit)

All devices attached to the I2C bus must be detectable. This includes:

  • Temperature sensors

  • Humidity sensors

  • Accelerometers

The following test units are covered in this category:

i2c/i2c-bus-detect

Check presence of an I²C bus

Category ID:

i2c
Status:

Blocking
Purpose:

If an expected number of I²C buses is provided, the job will verify the detected number is correct. If the expected number of buses is not provided, the job will pass if at least one I²C bus is detected.
Environment variable:

I2C_BUS_NUMBER
User:

root
Plugin:

shell
Requires:

manifest.has_i2c == ‘True’
For details about the required manifest entry, see has_i2c.
i2c/i2c-device-detect

Check if any I²C devices can be detected

Category ID:

i2c
Status:

Blocking
Purpose:

The test will pass if there’s at least one I²C device detected on any I²C bus.
Environment variable:

IGNORED_I2C_BUSES
User:

root
Plugin:

shell

Media Card tests

Media Card readers are tested for read and write for the following type of cards:

  • CF

  • MMC

  • MS

  • MSP

  • SD

  • SDHC

  • SDXC

  • XD

The following test units are covered in this category:

mediacard/sdhc-storage-manual

Test SDHC card insertion + read/write + removal.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a UNLOCKED Secure Digital High-Capacity (SDHC) media card Then it performs a read and write test on the SDHC card. Finally, it checks that the system detects the removal of the SDHC card.
Steps:

  1. Commence the test and then insert an UNLOCKED SDHC card into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the SDHC card from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

manifest.has_card_reader == ‘True’
For details about the required manifest entry, see has_card_reader.

Memory tests

Proper detection of the amount of memory installed is required (the amount of memory installed is the memory seen by the OS).

The following test units are covered in this category:

memory/info

Check the amount of memory reported by meminfo against DMI

Category ID:

memory
Status:

Blocking
Purpose:

This test checks the amount of memory which is reported in meminfo against the size of the memory modules detected by DMI.
User:

root
Plugin:

shell

Miscellaneous tests

The following test units are covered in this category:

miscellanea/check_prerelease

Test that the system is not a pre-release version

Category ID:

miscellanea
Status:

Blocking
Purpose:

Test to verify that the system uses production, rather than pre-release, versions of the kernel and the OS.
Plugin:

shell
Requires:

“Ubuntu Core” not in lsb.description
miscellanea/debsums

Check the MD5 sums of installed Debian packages

Category ID:

miscellanea
Status:

Blocking
Purpose:

Verify installed Debian package files against MD5 checksum lists from /var/lib/dpkg/info/*.md5sums.
User:

root
Plugin:

shell
Requires:

executable.name == ‘debsums’
miscellanea/grub_file_check

Check if the file core.efi exists to make sure shim and grub can be upgraded

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

“Ubuntu Core” not in lsb.description cpuinfo.platform in (“x86_64”, “aarch64”, “armhf”) bootloader.name == “grub”
miscellanea/oops

Run FWTS OOPS check

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (FWTS) oops tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’
miscellanea/ubuntu-desktop-minimal-recommends

Check that all the recommended packages for ubuntu-desktop-minimal are installed

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

package.name == ‘ubuntu-desktop-minimal’
miscellanea/ubuntu-desktop-recommends

Check that all the recommended packages for ubuntu-desktop are installed

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

package.name == ‘ubuntu-desktop’

Monitor tests

Each of the available external video ports (supported ports are HDMI, DisplayPort, DVI) are tested one by one. Output to the display must work i.e. a console is presented.

The following test units are covered in this category:

monitor/dim_brightness

Examine screen brightness adjustment capabilities.

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check changes to screen brightness.
Steps:

  1. Change screen brightness to maximum. 2. Click “Test” to attempt to dim the screen. 3. Check if the screen was dimmed approximately to half of the maximum brightness. 4. The screen will go back to the original brightness in 2 seconds.
Verification:

Was your screen dimmed approximately to half of the maximum brightness?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

dmi.display_type == “integrated”
monitor/displayport

Test the DisplayPort port functionality with mirrored, extended, external, or onboard display modes.

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your DisplayPort port as a monitor interconnect.
Steps:

  1. Connect a display (if not already connected) to the DisplayPort port on your system. 2. Switch display modes in your Display Settings; check if it can be set to mirrored, extended, displayed on external or onboard only.
Verification:

Was the desktop displayed correctly on the DisplayPort-connected screen in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_dp == ‘True’
For details about the required manifest entry, see has_dp.
monitor/displayport_hotplug

Can hotplug monitor (DisplayPort)

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the DisplayPort port and the ability to do hotplugging.
Steps:

Skip this test if your system does not have a DisplayPort port. 1. If a display is already connected, unplug it. 2. (Re-)Connect a display to the DisplayPort port on your system
Verification:

Was the interface displayed correctly on the screen?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_dp == ‘True’
For details about the required manifest entry, see has_dp.
monitor/dvi

Check DVI port functionality by connecting a display and verifying different display modes.

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your DVI port as a monitor interconnect.
Steps:

  1. Connect a display (if not already connected) to the DVI port on your system. 2. Switch display modes within your Display Settings, check if it can be set to mirrored, extended, display-only on external or onboard only.
Verification:

Was the desktop displayed correctly on the DVI-connected screen in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_dvi == ‘True’
For details about the required manifest entry, see has_dvi.
monitor/hdmi

Check HDMI port functionality as a monitor interconnect.

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your HDMI port as a monitor interconnect.
Steps:

  1. Connect a display (if not already connected) to the HDMI port on your system. 2. Switch display modes within your Display Settings, check if it can be set to mirrored, extended, displayed on external or onboard only.
Verification:

Was the desktop displayed correctly on the HDMI-connected screen in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_hdmi == ‘True’
For details about the required manifest entry, see has_hdmi.
monitor/hdmi_hotplug

Can hotplug monitor (HDMI)

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the HDMI port and the ability to do hotplugging.
Steps:

Skip this test if your system does not have a HDMI port. 1. If a display is already connected, unplug it. 2. (Re-)Connect a display to the HDMI port on your system
Verification:

Was the interface displayed correctly on the screen?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_hdmi == ‘True’
For details about the required manifest entry, see has_hdmi.
monitor/powersaving

<missing summary>

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your monitor power saving capabilities
Steps:

  1. Start the test to try the power saving capabilities of your monitor
Verification:

Did the monitor go blank and turn on again after a few seconds?
After-suspend:

True
Plugin:

user-interact-verify
monitor/thunderbolt3

Display connected via Thunderbolt 3

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your Thunderbolt 3 port as a monitor interconnect.
Steps:

  1. Connect a display (if not already connected) to the Thunderbolt 3 port on your system 2. Switch display modes between in your Display Settings, check if it can be set to mirrored, extended, displayed on external or onboard only
Verification:

Was the desktop displayed correctly on the Thunderbolt-connected screen in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_thunderbolt3 == ‘True’
For details about the required manifest entry, see has_thunderbolt3.
monitor/type-c_displayport

Display connected via DisplayPort using a USB Type-C port

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the connection of a screen using a “USB Type-C to DisplayPort” adapter.
Steps:

  1. Connect a display (if not already connected) to the USB Type-C port on your system using a “USB Type-C to DisplayPort” adapter. 2. Switch display modes within your Display Settings, check if it can be set to mirrored, extended, displayed on external or onboard only.
Verification:

Was the desktop displayed correctly on the screen connected using a “USB Type-C to DisplayPort” adapter in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_usbc_video == ‘True’
For details about the required manifest entry, see has_usbc_video.
monitor/type-c_hdmi

Display connection via HDMI using a USB Type-C port

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the connection of a display using a “USB Type-C to HDMI” adapter.
Steps:

  1. Connect a display (if not already connected) to the USB Type-C port on your system using a “USB Type-C to HDMI” adapter. 2. Switch display modes in your Display Settings, check if it can be set to mirrored, extended, displayed on external or onboard only.
Verification:

Was the desktop displayed correctly on the screen connected using a “USB Type-C to HDMI” adapter in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_usbc_video == ‘True’
For details about the required manifest entry, see has_usbc_video.
monitor/type-c_vga

Display connected via VGA using an USB Type-C port

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the connection of a screen using a “USB Type-C to VGA” adapter.
Steps:

  1. Connect a display (if not already connected) to the USB Type-C port on your system using a “USB Type-C to VGA” adapter 2. Switch display modes between in your Display Settings, check if it can be set to mirrored, extended, displayed on external or onboard only
Verification:

Was the desktop displayed correctly on the screen connected using a “USB Type-C to VGA” adapter in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_usbc_video == ‘True’
For details about the required manifest entry, see has_usbc_video.

Non-device specific networking tests

The following test units are covered in this category:

networking/gateway_ping

Verifies if the system can connect to the Internet.

Category ID:

networking
Status:

Blocking
Purpose:

Tests whether the system has a working Internet connection.
After-suspend:

True
User:

root
Plugin:

shell
networking/info_device__index___interface

Network Information of device {__index__} ({interface})

Unit type:

template
Category ID:

networking
Status:

Blocking
Purpose:

This test will check the network device {__index__} ({interface})
After-suspend:

True
From template:

networking/info_device__index___interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’
networking/ntp

Test NTP server synchronization capability.

Category ID:

networking
Status:

Blocking
Purpose:

Test to see if we can sync local clock to an NTP server
After-suspend:

True
User:

root
Plugin:

shell
Requires:

(package.name == ‘ntpdate’ or package.name == ‘ntpsec-ntpdate’)

Power Management tests

Warm reboot is tested such that the system must be able to perform the reboot command and services must be restarted such that systemctl does not identify a failed state.

Cold reboot is performed where an RTC is available (see next section). The wakealarm is used to reboot the system after a period of rest and services must be restarted such that systemctl does not identify a failed state.

The following test units are covered in this category:

power-management/cold-reboot

Cold reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This test powers off the system and then powers it on using RTC
Environment variable:

COLD_REBOOT_DELAY, RTC_DEVICE_FILE
User:

root
Plugin:

shell
Requires:

rtc.state == ‘supported’ rtc.wakealarm == ‘supported’
power-management/fwts_wakealarm

Executes ACPI Wakealarm test to validate functionality.

Category ID:

power-management
Status:

Blocking
Purpose:

Test ACPI Wakealarm (fwts wakealarm)
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’
power-management/post-cold-reboot

Post cold reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the cold reboot
User:

root
Plugin:

shell
power-management/post-warm-reboot

Post warm reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the warm reboot
User:

root
Plugin:

shell
power-management/rtc

Test that RTC functions properly (if present)

Category ID:

power-management
Status:

Blocking
Purpose:

Verify that the Real-time clock (RTC) device functions properly, if present.
After-suspend:

True
Environment variable:

RTC_DEVICE_FILE
User:

root
Plugin:

shell
Requires:

rtc.state == ‘supported’ cpuinfo.other != ‘emulated by qemu’
power-management/warm-reboot

Warm reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This tests reboots the system using the reboot command
User:

root
Plugin:

shell

Serial Port

Tests are carried out on ports that provide access via the Linux tty layer. The exact tests performed depend on the physical characteristics of the driver/receiver hardware. The possible tests include:

  • Ensure expected number of devices are available

  • Looped tests:

  • RS232 Ports: perform loopback test to ensure RX/TX

  • RS422/485 Ports: connect together to ensure RX/TX

  • Machine to Machine tests: confirm that a connection can be made to another PC device and RX/TX is operational

The following test units are covered in this category:

serial/loopback-dev

Serial loopback test of {dev}

Unit type:

template
Category ID:

serial
Status:

Blocking
Purpose:

Check if serial port is working hardwired
After-suspend:

True
From template:

serial/loopback-dev
Template resource:

serial_ports_static

Suspend tests

The following test units are covered in this category:

suspend/audio_after_suspend_auto

Verify mixer settings consistency after system suspend.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that mixer settings after suspend are the same as before suspend.
Plugin:

shell
Requires:

manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_audio_playback.
suspend/audio_before_suspend

Record mixer settings before suspending.

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_audio_playback.
suspend/cpu_after_suspend_auto

Ensure all CPUs are online post-resumption.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that all CPUs are online after resuming.
Plugin:

shell
suspend/cpu_before_suspend

Verify that all the CPUs are online before suspending

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/display_after_suspend

Verify if the display functions normally after a suspend and resume cycle.

Category ID:

suspend
Status:

Blocking
Purpose:

This test will check that the display is correct after suspend and resume
Steps:

  1. Check that your display does not show up visual artifacts after resuming.
Verification:

Does the display work normally after resuming from suspend?
Plugin:

manual
suspend/memory_after_suspend_auto

Ensure all memory is accessible after waking from suspend mode.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that all memory is available after resuming from suspend.
Plugin:

shell
suspend/memory_before_suspend

Dumps memory info to a file for comparison after suspend

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/network_after_suspend_auto

Verify the network functionality after system resume.

Category ID:

suspend
Status:

Blocking
Purpose:

Test the network after resuming.
User:

root
Plugin:

shell
suspend/network_before_suspend

Record the current network before suspending.

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
suspend/oops_after_suspend

Run fwts oops tests post-suspension to ensure system stability.

Category ID:

suspend
Status:

Blocking
Purpose:

Run Firmware Test Suite (fwts) oops tests after suspend.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’
suspend/validate_suspend_status

Tests that the machine suspended correctly

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

Query sysfs and debugfs for the amount of times the system was suspended. We expect the count in /sys/power/suspend_stats/success and suscess in /sys/kernel/debugfs/suspend_stats to be non zero
User:

root
Plugin:

shell

Touchscreen tests

The following test units are covered in this category:

touchscreen/evdev/2-touch-tap-product_slug

Validate proper detection of a 2-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that 2-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/2-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/3-touch-tap-product_slug

Validate proper detection of a 3-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that 3-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/3-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/4-touch-tap-product_slug

Validate the detection of a 4-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that 4-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/4-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/single-touch-tap-product_slug

Validate the detection of a single-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that single-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/single-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’

TPM 2.0 (Trusted Platform Module)

On Intel and AMD x86 platforms that include TPM 2.0 compliant modules, it is required that all commands necessary to support Ubuntu’s Full Disk Encryption functionality are supported.

The following test units are covered in this category:

clevis-encrypt-tpm2/detect-ecc-capabilities

Ensure the TPM has required capabilities for clevis ECC test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/detect-rsa-capabilities

Ensure the TPM has required capabilities for clevis RSA test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/ecc

clevis encrypt/decrypt key ecc

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/precheck

clevis encrypt/decrypt precheck

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_tpm2_chip == ‘True’ executable.name == ‘clevis-encrypt-tpm2’
For details about the required manifest entry, see has_tpm2_chip.
clevis-encrypt-tpm2/rsa

clevis encrypt/decrypt key rsa

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell

USB tests

USB 2.0

USB storage devices must work on all available USB ports. USB Human Interface Devices (HID), specifically keyboard or mouse, should be working properly on any USB port.

USB 3.0

USB storage devices must work on all available USB ports. USB Human Interface Devices (HID), specifically keyboard or mouse, should be working properly on any USB port.

USB Type C (USB 3.1)

USB Type C (USB 3.1) supports various types of devices (e.g. Video, Power) through the use of adapters or peripherals. The following adapters/peripherals should work:

  • Storage devices

  • Keyboard or mouse (basic functionality)

  • When DisplayPort over USB Type-C is advertised:

  • Display hot plugging and the following display are required to work:

    mirrored, extended, internal only, external only.

  • Audio output needs to be undistorted over this port.

The following test units are covered in this category:

usb-c/c-to-ethernet-adapter-insert

Check if USB Type-C to Ethernet adapter works

Category ID:

usb
Status:

Blocking
Purpose:

This test will check if system detects network interface of the Type-C to ethernet adapter.
Steps:

  1. Prepare USB Type-C to Ethernet adapter 2. Start the test 3. When the message “INSERT NOW” is shown, plug in the adapter to Type-C port
Verification:

<missing verification>
After-suspend:

True
Plugin:

user-interact
Requires:

manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.
usb-c/storage-manual

USB 3.0 storage device insertion + read/write + removal on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3.0 storage device in a USB Type-C connector. Then it performs a read and write test on the device. Finally, it checks that the system correctly detects the removal of the USB 3 storage. NOTE: Make sure the USB 3 storage device has a partition before starting the test.
Steps:

  1. Commence the test and then insert a USB 3.0 storage device to a USB Type-C port. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 3 storage from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

usb.usb3 == ‘supported’ manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.
usb/detect

Display USB devices attached to SUT

Category ID:

usb
Status:

Blocking
Purpose:

Detects and shows USB devices attached to this system.
After-suspend:

True
Plugin:

shell
Requires:

cpuinfo.platform != ‘s390x’
usb/hid

USB keyboard works

Category ID:

usb
Status:

Blocking
Purpose:

Check USB input device works
Steps:

  1. Connect USB keyboard 2. Input something with USB keyboard
Verification:

What was just input is displayed correctly
After-suspend:

True
Plugin:

manual
usb/storage-detect

Detect storage partitions on a device on the USB bus

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_usb_storage == ‘True’
For details about the required manifest entry, see has_usb_storage.
usb/storage-manual

Test USB 2.0 storage device insertion + read/write + removal.

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect insertion of a USB 2.0 storage device. Then it performs a read and write test on the USB 2.0. Finally, it checks that the system correctly detects the removal of the USB 2.0. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test and then insert an USB 2.0 into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 2.0 from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb3/storage-manual

Test USB 3.0 storage device insertion + read/write + removal.

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect insertion of a USB 3.0 storage device. Then it performs a read and write test on the USB 3.0. Finally, it checks that the system correctly detects the removal of the USB 3.0. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test and then insert an USB 3.0 into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 3.0 from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

usb.usb3 == ‘supported’

Wireless networking tests

Wi-Fi interfaces are tested for connection to access points configured for 802.11 b/g/n/ac/ax protocols.

The following test units are covered in this category:

wireless/detect

Detect if at least one Wireless LAN device is detected

Category ID:

wireless
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_wlan_adapter == ‘True’
For details about the required manifest entry, see has_wlan_adapter.
wireless/wireless_connection_open_ac_nm_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ac AP
After-suspend:

True
From template:

wireless/wireless_connection_open_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ac_np_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ac AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_ac_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_nm_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ax AP
After-suspend:

True
From template:

wireless/wireless_connection_open_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_np_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ax AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_nm_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11be AP
After-suspend:

True
From template:

wireless/wireless_connection_open_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_np_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11be AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_nm_interface

Connect to an unencrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check the system can connect to an insecure 802.11b/g AP
After-suspend:

True
From template:

wireless/wireless_connection_open_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_np_interface

Connect to unencrypted 802.11b/g Wi-Fi network on {{ interface }} using netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check the system can connect to an unsecured 802.11b/g access point using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_bg_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_nm_interface

Connect to an unencrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check if the system can connect to an unsecured 802.11n access point.
After-suspend:

True
From template:

wireless/wireless_connection_open_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_np_interface

Connect to unencrypted 802.11n Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11n AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_n_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_nm_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa3 security
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_np_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa3 security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_nm_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa3 security
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_np_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa3 security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_nm_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ac AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_np_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ac AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ac_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_nm_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_np_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_nm_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_np_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_nm_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11b/g AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_np_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11b/g AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_bg_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_nm_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check if the system can connect to an 802.11n AP with WPA security.
After-suspend:

True
From template:

wireless/wireless_connection_wpa_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_np_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }} using netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check if the system can connect to an 802.11n AP with WPA security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_n_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_scanning_interface

Test system can discover Wi-Fi networks on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can find a wireless network AP nearby
After-suspend:

True
From template:

wireless/wireless_scanning_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless Wide Area Network

WWAN interfaces are tested for connection to 3G/4G/LTE services.

The following test units are covered in this category:

wwan/check-sim-present-manufacturer-model-hw_id-auto

Check if a SIM card is present in a slot connected to the modem

Unit type:

template
Category ID:

wwan
Status:

Blocking
Purpose:

<missing purpose>
Description:

Check if a SIM card is present in a slot connected to the modem
After-suspend:

True
From template:

wwan/check-sim-present-manufacturer-model-hw_id-auto
Template resource:

wwan_resource
wwan/detect

Identify if WWAN module is missing

Category ID:

wwan
Status:

Blocking
Purpose:

Tests that there is a WWAN module present and indicates that testing of it should follow.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

manifest.has_wwan_module == ‘True’ snap.name == ‘modem-manager’ or package.name == ‘modemmanager’
For details about the required manifest entry, see has_wwan_module.
wwan/gsm-connection-manufacturer-model-hw_id-auto

Verify a GSM broadband modem can create a data connection multiple times

Unit type:

template
Category ID:

wwan
Status:

Blocking
Purpose:

Any modems discovered by the resource job that list GSM support will be tested to ensure a data connection can be made.
After-suspend:

True
From template:

wwan/gsm-connection-manufacturer-model-hw_id-auto
Template resource:

wwan_resource

Non-blocking

Advanced Configuration and Power Interface

The following test units are covered in this category:

acpi/oem_osi

Test ACPI OEM _OSI strings

Category ID:

acpi
Status:

Non-blocking
Purpose:

This checks if the deprecated OEM _OSI strings are still used by checking the ACPI DSDT and SSDT tables
User:

root
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”)

Audio tests

The following test units are covered in this category:

audio/detect-capture-devices

Check that at least one audio capture device exists

Category ID:

audio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_audio_capture == ‘True’
For details about the required manifest entry, see has_audio_capture.
audio/detect-playback-devices

Check that at least one audio playback device exists

Category ID:

audio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_audio_playback == ‘True’
For details about the required manifest entry, see has_audio_playback.

Bluetooth - BlueZ Self Tests

The following test units are covered in this category:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test

BlueZ-{bluez-internal-bnep-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

Runs a specific test from the bnep test suite.
After-suspend:

True
From template:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test
Template resource:

bluez-internal-bnep-tests
bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test

BlueZ-{bluez-internal-hci-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Runs a specific test from the hci test suite
After-suspend:

True
From template:

bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test
Template resource:

bluez-internal-hci-tests
bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test

BlueZ-{bluez-internal-rfcomm-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

Runs a specific test from the rfcomm test suite
After-suspend:

True
From template:

bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test
Template resource:

bluez-internal-rfcomm-tests
bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test

BlueZ-{bluez-internal-uc-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

Runs a specific test from the user channel test suite
After-suspend:

True
From template:

bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test
Template resource:

bluez-internal-uc-tests

Bluetooth tests

The following test units are covered in this category:

bluetooth/bluez-controller-detect

Check bluez lists a controller if rfkill detects one

Category ID:

bluetooth
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

package.name == ‘bluez’ or snap.name == ‘bluez’ environment.CHECKBOX_RUNNING_STRICT_SNAP != “1” or (connections.slot == ‘bluez:service’ and connections.plug == ‘{{ __system_env__[“SNAP_NAME”] }}:bluez’)
bluetooth4/beacon_eddystone_url_interface

Test system can get beacon EddyStone URL advertisements on the {{ interface }} adapter

Unit type:

template
Category ID:

bluetooth
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

bluetooth4/beacon_eddystone_url_interface
Template resource:

device
Template filter:

device.category == ‘BLUETOOTH’

Camera tests

The following test units are covered in this category:

camera/camera-quality-rpi_name

Webcam BRISQUE score for rpi devices on {name}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

Uses an image of the camera to get the quality based on a No-Reference image quality assessment algorithm called BRISQUE. This test will timeout and fail if the quality has not been computed within 120 seconds.
From template:

camera/camera-quality-rpi_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images-rpi-attachment_name

Attach an image from the multiple resolution images test on rpi

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

This test will attach one of the images used for the multiple resolution images test on rpi.
From template:

camera/multiple-resolution-images-rpi-attachment_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images-rpi_name

Webcam multiple resolution capture test for Pi Camera

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

Takes multiple pictures based on the resolutions supported by the camera and validates their size and that they are of a valid format.
From template:

camera/multiple-resolution-images-rpi_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images_name

Webcam multiple resolution capture test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

Takes multiple pictures based on the resolutions supported by the camera and validates their size and that they are of a valid format.
After-suspend:

True
From template:

camera/multiple-resolution-images_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

CPU tests

The following test units are covered in this category:

cpu/arm64_vfp_support_platform

Validate that the Floating Point Unit is running on {platform} device

Unit type:

template
Category ID:

cpu
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Validate that the Floating Point Unit is running on {platform} device.
From template:

cpu/arm64_vfp_support_platform
Template resource:

cpuinfo
Template filter:

cpuinfo.platform == ‘aarch64’
cpu/armhf_vfp_support_platform

Validate that the Vector Floating Point Unit is running on {platform} device

Unit type:

template
Category ID:

cpu
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Validate that the Vector Floating Point Unit is running on {platform} device.
From template:

cpu/armhf_vfp_support_platform
Template resource:

cpuinfo
Template filter:

cpuinfo.platform == ‘armv7l’
cpu/clocktest

Tests the CPU for clock jitter

Category ID:

cpu
Status:

Non-blocking
Purpose:

Runs a test for clock jitter on SMP machines.
After-suspend:

True
Plugin:

shell
Requires:

cpuinfo.platform not in (“s390x”)
cpu/cstates_results.log

Attach C-States test log

Category ID:

cpu
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission.
Plugin:

attachment
Requires:

cpuinfo.platform not in (“aarch64”, “armv7l”, “s390x”)
cpu/maxfreq_test-log-attach

Attach CPU max frequency log

Category ID:

cpu
Status:

Non-blocking
Purpose:

Attaches the log generated by cpu/maxfreq_test to the results submission.
Plugin:

attachment
cpu/scaling_test-log-attach

Attach CPU scaling capabilities log

Category ID:

cpu
Status:

Non-blocking
Purpose:

Attaches the log generated by cpu/scaling_test to the results submission.
Plugin:

attachment

DesignWare Core SuperSpeed USB 3.0 Controller

The following test units are covered in this category:

usb-dwc3/driver-detect

Detect if the USB DWC3 drivers are loaded

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_usb_dwc3_controller == ‘True’
For details about the required manifest entry, see has_usb_dwc3_controller.
usb-dwc3/mass-storage

Check DUT can be detected as mass storage device

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

Check that after connecting the device under test (DUT) to another device (host), DUT can be detected as mass storage device by the host.
Steps:

  1. In BIOS, enable USB xDCI support. 2. Press Enter to setup a 256 MB FAT32 image on the device. 3. From an Ubuntu (or Windows) host, connect a cable to the USB OTG port of the DUT.
Verification:

The host detects and mounts a mass storage device. It has read and write permissions on it.
User:

root
Plugin:

user-interact-verify
usb-dwc3/mass-storage-cleanup

Cleanup mass storage setup after mass storage device test

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
usb-dwc3/module-detect

Detect if the USB DWC3 module is loaded

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_usb_dwc3_controller == ‘True’
For details about the required manifest entry, see has_usb_dwc3_controller.

Disk tests

The following test units are covered in this category:

disk/apste_support_on_name

Check support for Autonomous Power State Transition on {name}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

disk/apste_support_on_name
Template resource:

device
Template filter:

device.driver == ‘nvme’ and device.category == ‘OTHER’
disk/check-software-raid

Validate the configuration of software RAID devices are expected

Category ID:

disk
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Examine the system to detect Software RAID devices are created and the RAID mode are expected the SOFTWARE_RAID_LEVEL variable is needed for this tests. e.g. SOFTWARE_RAID_LEVEL=”raid0 raid1 raid5”
Environment variable:

SOFTWARE_RAID_LEVEL
User:

root
Plugin:

shell
Requires:

executable.name == ‘mdadm’ manifest.has_md_raid == ‘True’
For details about the required manifest entry, see has_md_raid.
disk/disk_stress_ng_name

Disk stress-ng test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
Purpose:

Disk stress-ng test for {product_slug}
From template:

disk/disk_stress_ng_name
Template resource:

device
Template filter:

device.category == ‘DISK’
disk/stats_name

Disk statistics for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
Purpose:

This test checks disk stats, generates some activity and rechecks stats to verify they’ve changed. It also verifies that disks appear in the various files they’re supposed to. . This test will inspect the following disk: . product name: {product_slug} sysfs path: {path} device node path: /dev/{name}
After-suspend:

True
From template:

disk/stats_name
Template resource:

device
Template filter:

device.category == ‘DISK’ and device.name != ‘’

Docker containers

The following test units are covered in this category:

docker/build-single_arch

Test docker build with a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/build-single_arch
Template resource:

docker_resource
docker/commit_arch

Test docker commit a change to a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/commit_arch
Template resource:

docker_resource
docker/compose-and-basic_arch

Test docker compose and basic command

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/compose-and-basic_arch
Template resource:

docker_resource
docker/compose-restart_arch

Test compose a container with restart policy applied

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/compose-restart_arch
Template resource:

docker_resource
docker/compose-single_arch

Test docker compose with a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/compose-single_arch
Template resource:

docker_resource
docker/copy_arch

Test copy a file bwtween a container and local filesystem

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/copy_arch
Template resource:

docker_resource
docker/deploy-registry_arch

Deploy a registry server and run it on localhost

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/deploy-registry_arch
Template resource:

docker_resource
docker/diff_arch

Test changes to files in Ubuntu container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/diff_arch
Template resource:

docker_resource
docker/export-and-import_arch

Test docker import and export a docker container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/export-and-import_arch
Template resource:

docker_resource
docker/info

Display system-wide information about docker

Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
User:

root
Requires:

executable.name == ‘docker’
docker/inspect_arch

Test query low-level information on a docker object

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/inspect_arch
Template resource:

docker_resource
docker/interative_arch

Test an interactive shell in Ubuntu container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/interative_arch
Template resource:

docker_resource
docker/kill_arch

Test killing containers

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/kill_arch
Template resource:

docker_resource
docker/restart-always_arch

Test container restart policy with always applied

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/restart-always_arch
Template resource:

docker_resource
docker/restart-on-failure_arch

Test container restart policy with on_failure applied

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/restart-on-failure_arch
Template resource:

docker_resource
docker/run_arch

Download and run ubuntu container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/run_arch
Template resource:

docker_resource
docker/save-and-load_arch

Test docker save and load a docker image

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/save-and-load_arch
Template resource:

docker_resource
docker/start-stop_arch

Start and stop a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/start-stop_arch
Template resource:

docker_resource
docker/update_arch

Test update configuration of one container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/update_arch
Template resource:

docker_resource
docker/version

Display docker version information

Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
User:

root
Requires:

executable.name == ‘docker’

Eeprom

The following test units are covered in this category:

eeprom/read-write

Test EEPROM read and write functions for the system.

Category ID:

eeprom
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

shell
Requires:

manifest.has_eeprom == ‘True’
For details about the required manifest entry, see has_eeprom.

Error Detection And Correction (EDAC) Memory Controllers

The following test units are covered in this category:

edac/default-report

Attach the default report from EDAC util

Category ID:

edac
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

attachment
Requires:

executable.name == ‘edac-util’ manifest.has_edac_module == ‘True’
For details about the required manifest entry, see has_edac_module.
edac/detect

Detect if the EDAC drivers are loaded and Memory Controllers are found

Category ID:

edac
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
Requires:

executable.name == ‘edac-util’ manifest.has_edac_module == ‘True’
For details about the required manifest entry, see has_edac_module.

Ethernet Device tests

The following test units are covered in this category:

ethernet/wol_S3_interface

Wake on LAN (WOL) test from S3 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Non-blocking
Purpose:

Check that another system can wake the System Under Test (SUT) up from S3 using the Ethernet port {interface}’s WOL function.
From template:

ethernet/wol_S3_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’ and device.interface != ‘UNKNOWN’
ethernet/wol_S4_interface

Wake on LAN (WOL) test from S4 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Non-blocking
Purpose:

Check that another system can wake up from S4 the System Under Test (SUT) using ethernet port {interface} Wake-on-LAN (WOL) function.
From template:

ethernet/wol_S4_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’
ethernet/wol_S5_interface

Wake on LAN (WOL) test from S5 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Non-blocking
Purpose:

Check that another system can wake the System Under Test (SUT) from S5 using the Ethernet port {interface} WOL function.
From template:

ethernet/wol_S5_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’

Firmware tests

The following test units are covered in this category:

firmware/fwts_desktop_diagnosis

Run FWTS QA-concerned desktop-specific diagnosis tests.

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (fwts) QA-concerned desktop-specific diagnosis tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’
firmware/fwts_desktop_diagnosis_results.log.gz

Attach FWTS desktop diagnosis log to submission

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission
Plugin:

attachment
firmware/fwts_dump_acpi_attachment.gz

Collect the ACPI tables dump from FWTS

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

attachment
firmware/fwupdmgr_get_devices

Collect the device firmware update information

Category ID:

firmware
Status:

Non-blocking
Purpose:

Attach information about the devices, as reported by the fwupdmgr
Plugin:

attachment
Requires:

executable.name in (“fwupdmgr”, “fwupd.fwupdmgr”)

Gathers information about the DUT

The following test units are covered in this category:

cpuinfo

Collect information about the CPU

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Gets CPU resource info from /proc/cpuinfo
User:

root
Plugin:

resource
dpkg

Collect information about dpkg version

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Gets info on the version of dpkg installed
Plugin:

resource
interface

Collect information about interfaces

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of interface declarations on the device
Plugin:

resource
lsb

[DEPRECATED, use ‘os’ instead] Collect information about installed operating system (os-release)

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates release info based on /etc/os-release
Plugin:

resource
meminfo

Collect information about system memory (/proc/meminfo)

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates resource info based on /proc/meminfo
Plugin:

resource
package

Collect information about installed software packages

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of packages
Plugin:

resource
snap

Collect information about installed snap packages

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of snap packages
Plugin:

resource
uname

Collect information about the running kernel

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Creates resource info from uname output
Plugin:

resource

General Purpose I/O

The following test units are covered in this category:

gpio/gpiomem_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via /dev/gpiomem

Unit type:

template
Category ID:

gpio
Status:

Non-blocking
Purpose:

Ensure GPIO lines exposed on headers are controllable via /dev/gpiomem using the RPi python module for specific models.
After-suspend:

True
From template:

gpio/gpiomem_loopback_pairs_model
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)
gpio/sysfs_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template
Category ID:

gpio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

gpio/sysfs_loopback_pairs_model
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)
gpio/sysfs_loopback_pairs_vendor_product

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template
Category ID:

gpio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

gpio/sysfs_loopback_pairs_vendor_product
Template resource:

dmi
Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’, ‘UPN-EHL01’)

Hardware video acceleration

The following test units are covered in this category:

va-api/va-initialize

Detect if the VA API could be loaded

Category ID:

va-api
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

package.name == ‘vainfo’ cpuinfo.platform == ‘x86_64’ manifest.has_va_api == ‘True’
For details about the required manifest entry, see has_va_api.

Informational tests

The following test units are covered in this category:

acpi_sleep_attachment

Attach the contents of /proc/acpi/sleep for further analysis.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the contents of /proc/acpi/sleep if it exists.
Plugin:

attachment
codecs_attachment

Attach a report of installed codecs for Intel HDA.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a report of installed codecs for Intel HDA
Plugin:

attachment
cpuinfo_attachment

Attach a copy of /proc/cpuinfo

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a report of CPU information
Plugin:

attachment
dkms_info_attachment

Attaches json dumps of installed dkms package information.

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches json dumps of installed dkms package information.
Plugin:

attachment
Requires:

package.name == ‘dkms’
dmesg_attachment

Attach a copy of dmesg or the current dmesg buffer to the test results.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a copy of /var/log/dmesg or the current dmesg buffer to the test results
User:

root
Plugin:

attachment
dmi_attachment

Attach a copy of /sys/class/dmi/id/*

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches info on DMI
Plugin:

attachment
Requires:

dmi_present.state == ‘supported’
dmidecode_attachment

Attach output of dmidecode

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches dmidecode output
User:

root
Plugin:

attachment
Requires:

executable.name == ‘dmidecode’ dmi_present.state == ‘supported’
dpkg_attachment

Attach dpkg -l output

Category ID:

info
Status:

Non-blocking
Purpose:

Attach system debian packages status
Plugin:

attachment
Requires:

executable.name == ‘dpkg’
efi_attachment

Attaches firmware version info

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the firmware version
User:

root
Plugin:

attachment
info/buildstamp

Attaches the buildstamp identifier for the OS

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the buildstamp identifier for the OS
Plugin:

attachment
info/disk_partitions

Attaches info about disk partitions

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches information about disk partitions
User:

root
Plugin:

attachment
info/hdparm_name.txt

Attaches info from hdparm about {name}

Unit type:

template
Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

info/hdparm_name.txt
Template resource:

device
Template filter:

device.category == ‘DISK’
info/kernel-config-iommu-flag

Check if the kernel is compiled with IOMMU support

Category ID:

info
Status:

Non-blocking
Purpose:

Checks the value of the CONFIG_INTEL_IOMMU_DEFAULT_ON flag in the kernel configuration
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”)
info/secure-boot-check

Check secure boot state

Category ID:

info
Status:

Non-blocking
Purpose:

Output whether secure boot is enabled or disabled
Plugin:

attachment
Requires:

package.name == ‘mokutil’ manifest.has_secure_boot == ‘True’
For details about the required manifest entry, see has_secure_boot.
info/systemd-analyze

System boot-up performance statistics

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
info/systemd-analyze-critical-chain

Print the tree of the time-critical chain of SystemD

Category ID:

info
Status:

Non-blocking
Purpose:

This job prints a tree of the time-critical chain of SystemD units.
Plugin:

shell
info/touchpad_driver

Gather touchpad name, driver name, and driver version information.

Category ID:

info
Status:

Non-blocking
Purpose:

Returns the name, driver name and driver version of any touchpad discovered on the system.
Plugin:

attachment
Requires:

device.category == ‘TOUCHPAD’
installer_debug.gz

Attach the installer’s debug log for diagnostic purposes.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the installer debug log if it exists.
User:

root
Plugin:

attachment
kernel_cmdline_attachment

Attach a copy of /proc/cmdline

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the kernel command line used to boot
Plugin:

attachment
lsmod_attachment

Attach a list of currently running kernel modules

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a list of the currently running kernel modules.
Plugin:

attachment
lspci_attachment

Attach a list of PCI devices

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches very verbose lspci output.
Plugin:

attachment
lspci_standard_config_attachment

Attach PCI configuration space hex dump

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a hex dump of the standard part of the PCI configuration space for all PCI devices.
Plugin:

attachment
lstopo_visual_attachment

Attach the output of lstopo command to present system topology.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the system topology as presented by the lstopo command
Plugin:

attachment
Requires:

executable.name == ‘lstopo’
lsusb_attachment

Attach output of lsusb

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a list of detected USB devices.
After-suspend:

True
User:

root
Plugin:

attachment
manifest

Hardware Manifest

Category ID:

info
Status:

Non-blocking
Purpose:

This job loads the hardware manifest and exposes it as a resource.
Plugin:

resource
meminfo_attachment

Attach copy of /proc/meminfo

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches info on system memory as seen in /proc/meminfo.
Plugin:

attachment
modinfo_attachment

Attach modinfo information

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches modinfo information for all currently loaded modules
Plugin:

attachment
modprobe_attachment

Attach the contents of /etc/modprobe.*

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of the various modprobe conf files.
User:

root
Plugin:

attachment
modules_attachment

Attach the contents of /etc/modules

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of the /etc/modules file.
Plugin:

attachment
sysctl_attachment

Attach sysctl configuration files.

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of various sysctl config files.
User:

root
Plugin:

attachment
sysfs_attachment

Attach detailed sysfs property output from udev

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches a report of sysfs attributes.
Plugin:

attachment
Requires:

cpuinfo.platform not in (“aarch64”)
udev_attachment

Attach dump of udev database

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches a dump of the udev database showing system hardware information.
Plugin:

attachment

Intel Integrated Sensor Hub Transport Protocol (ISHTP)

The following test units are covered in this category:

eclite/module-detect

Verifies that the ishtp_eclite module is loaded

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_eclite == ‘True’
For details about the required manifest entry, see has_eclite.
eclite/temperature-reading-from-thermal-acpitz

Read the temperature of acpitz to ensure Eclite is functional

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

For testing the feature of Eclite, one way is to monitor the temperature of acpitz thermal.
User:

root
Plugin:

shell
ishtp/device-detect

Verify that at least 1 device entry exists in /sys/bus/ishtp/devices

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
ishtp/module-detect

Verifies that the intel_ish_ipc module for ISHTP is loaded

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_ishtp == ‘True’
For details about the required manifest entry, see has_ishtp.

Intel Management Engine Interface (MEI)

The following test units are covered in this category:

mei/check-device

Detect if the Intel MEI device is available

Category ID:

mei
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”) manifest.has_mei == ‘True’
For details about the required manifest entry, see has_mei.
mei/check-module

Detect if the Intel MEI kernel module is loaded

Category ID:

mei
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”) manifest.has_mei == ‘True’
For details about the required manifest entry, see has_mei.
mei/get-firmware-version

Retrieve MEI firmware version by MEI interface

Category ID:

mei
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell

LED tests

The following LEDS will be tested for functionality:

  • Cloud LED

  • Wireless LED

  • Bluetooth LED

  • WWAN LED

The following test units are covered in this category:

led-indicator/gpio-controller-leds-name

Check control of {name} LED indicator.

Unit type:

template
Category ID:

led
Status:

Non-blocking
Purpose:

Check that {name} LED turns on and off.
After-suspend:

True
From template:

led-indicator/gpio-controller-leds-name
Template resource:

led-indicator/gpio-controller-leds
led-indicator/gpio-leds-name

Check control of {name} LED indicator.

Unit type:

template
Category ID:

led
Status:

Non-blocking
Purpose:

Check that {name} LED turns on and off.
After-suspend:

True
From template:

led-indicator/gpio-leds-name
Template resource:

led-indicator/gpio-leds
led-indicator/sysfs-leds-manual

Check control of {name} LED.

Unit type:

template
Category ID:

led
Status:

Non-blocking
Purpose:

Check that {name} LED turns on and off.
After-suspend:

True
From template:

led-indicator/sysfs-leds-manual
Template resource:

led-indicator/sysfs-leds

Media Card tests

The following test units are covered in this category:

mediacard/sdxc-storage-manual

Test SDXC card insertion + read/write + removal.

Category ID:

mediacard
Status:

Non-blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Secure Digital Extended Capacity (SDXC) media card Then it performs a read and write test on the SDXC card. Finally, it checks that the system detects the removal of the SDXC card.
Steps:

  1. Commence the test and then insert an UNLOCKED SDXC card into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the SDXC card from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
Requires:

manifest.has_card_reader == ‘True’
For details about the required manifest entry, see has_card_reader.

Miscellaneous tests

The following test units are covered in this category:

install/apt-get-gets-updates

Ensure apt can access repositories and get updates without installing them, to aid in recovery from broken updates.

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Tests to see that apt can access repositories and get updates (does not install updates). This is done to confirm that you could recover from an incomplete or broken update.
User:

root
Plugin:

shell
Requires:

package.name == ‘apt’
miscellanea/device_check

Device Check

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Device check
Steps:

  1. Commence the test 2. Compare items on System Manifest to the devices known to udev
Verification:

Do the devices reported by udev match the devices on the Manifest?
Plugin:

user-interact-verify
miscellanea/oops_results.log

Attach the FWTS oops results for submission.

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Attaches the FWTS oops results log to the submission
Plugin:

attachment
miscellanea/submission-resources

Check that data for a complete result are present

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

A meta-job that verifies the data necessary for a complete result submission are present. Failure indicates that the results are incomplete and may be rejected.
Plugin:

shell

Monitor tests

The following test units are covered in this category:

monitor/dvi-to-vga

Monitor works (DVI-to-VGA)

Category ID:

monitor
Status:

Non-blocking
Purpose:

Check output to display through VGA adaptor on DVI port
Steps:

  1. Connect display to VGA adaptor on DVI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_dvi == ‘True’
For details about the required manifest entry, see has_dvi.
monitor/hdmi-to-vga

Monitor works (HDMI-to-VGA)

Category ID:

monitor
Status:

Non-blocking
Purpose:

Check output to display through VGA adaptor on HDMI port
Steps:

  1. Connect display to VGA adaptor on HDMI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_hdmi == ‘True’
For details about the required manifest entry, see has_hdmi.
monitor/vga

Monitor works (VGA)

Category ID:

monitor
Status:

Non-blocking
Purpose:

Check output to display through VGA port
Steps:

  1. Connect display to VGA port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_vga == ‘True’
For details about the required manifest entry, see has_vga.

Non-device specific networking tests

The following test units are covered in this category:

ipv6_detect

Test if the kernel is IPv6 ready

Category ID:

networking
Status:

Non-blocking
Purpose:

Test if the running kernel supports IPv6.
Steps:

<missing steps>
Verification:

<missing verification>
After-suspend:

True
networking/predictable_names

Verify that all network interfaces have predictable names.

Category ID:

networking
Status:

Non-blocking
Purpose:

Verify that all network interfaces have predictable names.
Plugin:

shell
Requires:

environment.CHECKBOX_RUNNING_STRICT_SNAP != “1” or lsb.release >= ‘20’ environment.CHECKBOX_RUNNING_STRICT_SNAP != “1” or model_assertion.gadget != “pi” environment.CHECKBOX_RUNNING_STRICT_SNAP == “1” or lsb.release >= ‘18’

Power Management tests

Suspend/Resume

For x86 devices, a 30 cycle suspend/resume stress test is performed using the FWTS. The suspend mode (e.g. S3, S2Idle) used during the test will be the default for the system under test. The test is passed if all 30 cycles complete without failure. Any errors reported in the fwts log for the 30 cycle suspend/resume stress test are informational only and do not affect the outcome of the test, however, we do recommend examining and fixing any failures noted, as they indicate firmware non-compliance with standards.

In addition a single suspend is performed across which the following features and devices are tested:

  • CPU

  • Memory

  • Networking (Wifi, Ethernet)

  • Audio

  • Bluetooth

  • USB controllers

  • Input devices

  • Mediacards

The following test units are covered in this category:

power-management/fwts_wakealarm-log-attach

Attach and display fwts wakealarm test log.

Category ID:

power-management
Status:

Non-blocking
Purpose:

Attach log from fwts wakealarm test
Plugin:

attachment
watchdog/detect

Detect the presence of a Hardware Watchdog

Category ID:

power-management
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
Requires:

manifest.has_hardware_watchdog == ‘True’
For details about the required manifest entry, see has_hardware_watchdog.
watchdog/post-trigger-system-reset-auto

Post watchdog reset service check

Category ID:

power-management
Status:

Non-blocking
Purpose:

Check there are no failed services after the watchdog triggered
User:

root
Plugin:

shell
Requires:

manifest.has_hardware_watchdog == ‘True’
For details about the required manifest entry, see has_hardware_watchdog.
watchdog/systemd-config

Check if the hardware watchdog is properly configured

Category ID:

power-management
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
Requires:

manifest.has_hardware_watchdog == ‘True’
For details about the required manifest entry, see has_hardware_watchdog.
watchdog/trigger-system-reset-auto

Test that the watchdog module can trigger a system reset

Category ID:

power-management
Status:

Non-blocking
Purpose:

Ensure that the watchdog module can successfully initiate a system reset.
User:

root
Plugin:

shell

Quadrature Encoder Peripherals

The following test units are covered in this category:

qep/qep-device-driver-for-qep-device

Verify PCI Device {qep-device} is using the correct driver

Unit type:

template
Category ID:

qep
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Checks that the device exists in lspci and that the device driver associated is correct. Device ID should be 8086:{qep-device} and the driver is always intel_qep.
From template:

qep/qep-device-driver-for-qep-device
Template resource:

qep/qep-devices
qep/qep-device-node-for-qep-device

Verify device directory exists for {qep-device}

Unit type:

template
Category ID:

qep
Status:

Non-blocking
Purpose:

Detects if the device’s directory exists.
From template:

qep/qep-device-node-for-qep-device
Template resource:

qep/qep-devices

Real Time Clock (RTC)

The following test units are covered in this category:

rtc/rtc_alarm_rtc

Check that RTC alarm of {rtc} works

Unit type:

template
Category ID:

rtc_category
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

rtc/rtc_alarm_rtc
Template resource:

rtc_list
rtc/rtc_clock_rtc

Check that {rtc} clock is synchronized with system clock

Unit type:

template
Category ID:

rtc_category
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

rtc/rtc_clock_rtc
Template resource:

rtc_list
rtc/rtc_number

Check the number of RTC as expected.

Category ID:

rtc_category
Status:

Non-blocking
Purpose:

Check the number of RTC on platform as expected.
After-suspend:

True
Environment variable:

TOTAL_RTC_NUM
User:

root
Plugin:

shell

Snapd

The following test units are covered in this category:

snappy/snap-install

Test the snap install command is working

Category ID:

snapd
Status:

Non-blocking
Purpose:

The store should contain the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap. The test makes sure this can be downloaded and installed on the system.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-list

Test that the snap list command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

If snap list command is working then should at least find the ubuntu-core package.
Plugin:

shell
snappy/snap-refresh-automated

Test whether the snap refresh command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

The test will install the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap from the stable channel and then refreshes it to the edge channel and compares the revision before and after the refresh.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-remove

Test the snap remove command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

After having installed the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap, check it can be removed.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-reupdate-automated

Test the snap refresh command works after blacklisting.

Category ID:

snapd
Status:

Non-blocking
Purpose:

Checks that the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap can be refreshed after removal of the blacklisted revision.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-revert-automated

Test the snap revert command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

Checks if the edge channel {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap is reverted back to the one from stable.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-snaps-confinement

Test all the snaps’ confinement

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test all the snaps’ confinement, devmode, revision. Make sure the confinement is “strict”, devmode is “False”, and revision should not be sideloaded.
Plugin:

shell
snappy/test-store-config-store

Test that image is using the correct snappy store configuration.

Unit type:

template
Category ID:

snapd
Status:

Non-blocking
Purpose:

The image can be tied to using a particular store for the OEM. This tests the store for the image is as expected.
From template:

snappy/test-store-config-store
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.store not in (‘unknown’)
snappy/test-store-install-beta

Snappy install command - beta channel store

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test the snappy install command is able to install and remove snap in beta channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-store-install-edge

Snappy install command - edge channel store

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test the snappy install command is able to install a snap in the edge channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-system-confinement

Test if the system confinement is strict

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test if the system confinement is “strict” If not, list the missing features
Plugin:

shell

SocketCAN interface tests

Devices that support the SocketCAN standard are tested to ensure that the adapter is present and can be communicated with via CANBus configuration commands.

The following test units are covered in this category:

socketcan/send_packet_local_eff_virtual

Virtual CAN device support test (Local test with raw socket and EFF)

Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_eff_interface

CAN device support test for {interface} (Raw, Local, EFF)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket, this is only a local test as the broadcast packet is received on the same device.
After-suspend:

True
From template:

socketcan/send_packet_local_eff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_local_fd_virtual

Virtual CAN device support test (Raw, Local, FD)

Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket; this is only a local test as the broadcast packet is received on the same device.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

manifest.has_socket_can_fd == ‘True’
For details about the required manifest entry, see has_socket_can_fd.
socketcan/send_packet_local_fd_interface

CAN device support test for {interface} (Raw, Local, FD)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
From template:

socketcan/send_packet_local_fd_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_local_sff_virtual

Virtual CAN device support test (Raw, Local)

Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_sff_interface

CAN device support test for {interface} (Raw, Local)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
From template:

socketcan/send_packet_local_sff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_eff_interface

CAN device support test {interface} (Raw, Remote, EFF)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket to a remote device. As a prerequisite the remote device should have can-echo-server installed so as to return the predicted packet.
After-suspend:

True
From template:

socketcan/send_packet_remote_eff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_fd_interface

CAN device support test {interface} (Raw, Remote, FD)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket to a remote device. As a prerequisite, the remote device should have can-echo-server installed so as to return the predicted packet.
After-suspend:

True
From template:

socketcan/send_packet_remote_fd_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_sff_interface

CAN device support test {interface} (Raw, Remote)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket to a remote device. As a prerequisite, the remote device should have can-echo-server installed so as to return the predicted packet.
After-suspend:

True
From template:

socketcan/send_packet_remote_sff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’

Suspend tests

The following test units are covered in this category:

suspend/any_suspend_failure

Tests if any device in this machine reports a suspend failure

Category ID:

suspend
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

If the DUT suspended successfully, /sys/power/suspend_stats/fail* and fail* in /sys/kernel/debug/suspend_stats will be zero. If a device failed /sys/power/suspend_stats/last_failed_dev and last_failed_dev in /sys/kernel/debug/suspend_stats will show the failed device
User:

root
Plugin:

shell
suspend/oops_results_after_suspend.log

Attach FWTS oops results log post-suspend.

Category ID:

suspend
Status:

Non-blocking
Purpose:

Attaches the FWTS oops results log to the submission after suspend
Plugin:

attachment
suspend/xrandr_screens_after_suspend_primary

Attach screenshots from suspend/cycle resolutions after suspend test to results.

Category ID:

suspend
Status:

Non-blocking
Purpose:

This attaches screenshots from the suspend/cycle_resolutions_after_suspend test to the results submission.
Plugin:

attachment

TPM 2.0 (Trusted Platform Module)

The following test units are covered in this category:

tpm2/fwts-event-log-dump

Dump the contents of the TPM Event Log

Category ID:

tpm2
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

The information in the TPM Event Log can be useful in debugging problems with TPM command support and adherance to standards. This can be of particular help when checking whether a device can support Ubuntu Core Full Disk Encryption.
User:

root
Plugin:

attachment
Requires:

manifest.has_tpm2_chip == ‘True’ executable.name == ‘fwts’
For details about the required manifest entry, see has_tpm2_chip.

USB tests

The following test units are covered in this category:

usb-c-otg/g_ether

Check DUT can be detected as USB ethernet device

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

  1. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT. 2. Press Enter to configure the IP address for the USB OTG ethernet interface on the DUT. (IP will be 192.168.9.1/24) 3. On the host, configure the IP address for the USB OTG ethernet interface. It should be in the same subnet as the DUT. (e.g., 192.168.9.2/24) 4. Try to ping between the host and DUT.
Verification:

The host and DUT can ping each other.
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_ether-cleanup

Cleanup USB OTG ethernet interface setup after ethernet device test

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_mass_storage

Check DUT can be detected as a mass storage device

Category ID:

usb
Status:

Non-blocking
Purpose:

Check that after connecting the device under test (DUT) to another device (host), the DUT can be detected as a mass storage device by the host.
Steps:

  1. Press Enter to setup a 16 MB FAT32 image on the device. 2. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT.
Verification:

The host detects and mounts the mass storage device. It has read and write permissions on it.
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_mass_storage-cleanup

Cleanup mass storage setup after mass storage device test

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_serial

Check if USB OTG can work as a serial port.

Category ID:

usb
Status:

Non-blocking
Purpose:

Check if USB OTG can work as a serial port.
Steps:

  1. Press enter to probe USB OTG g_serial 2. Connect a USB cable from an Ubuntu host to the USB OTG (it’s the USB Type C connector) port on DUT. 3. On the host, check if ttyACM0 has been probed $ ls /dev/ttyACM0 4. On the host, listen to the ttyACM0 port as the receiving side. $ sudo su $ cat /dev/ttyACM0 5. On DUT, send a string via /dev/ttyGS0 as the sending side. $ sudo su $ echo 123 > /dev/ttyGS0 6. Check if the string “123” was received on the host side. 7. Check the string sending from the host to DUT by repeating steps 4-6, but swap the send and receive sides.
Verification:

Does string send and receive function correctly?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_serial-cleanup

Cleanup USB OTG serial interface setup after serial device test

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c/c-to-a-adapter/hid

USB HID work on USB Type-C port using a “USB Type-C to Type-A” adapter

Category ID:

usb
Status:

Non-blocking
Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port using a “USB Type-C to Type-A” adapter
Steps:

  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port using a “USB Type-C to Type-A” adapter 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text
Verification:

Did the device work as expected?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.
usb-c/c-to-a-adapter/storage-manual

Test USB 3 storage device insertion + read/write + removal using a “Type-C to Type-A” adapter.

Category ID:

usb
Status:

Non-blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3 storage device in a USB Type-C connector using a “Type-C to Type-A” adapter. Then it performs a read and write test on the device. Finally, it checks that the system correctly detects the removal of the USB 3 storage. NOTE: Make sure the USB 3 storage device has a partition before starting the test.
Steps:

  1. Commence and then insert a USB 3 storage device to a USB Type-C port using a “Type-C to Type-A” adapter. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 3 storage from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

usb.usb3 == ‘supported’ manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.

Wi-Fi access point

The following test units are covered in this category:

wireless/nmcli_wifi_ap_a_interface

Create 802.11a Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

wireless/nmcli_wifi_ap_a_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/nmcli_wifi_ap_bg_interface

Create 802.11b/g Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Create an 802.11b/g Wi-Fi access point on a specified interface using NetworkManager command-line tools.
After-suspend:

True
From template:

wireless/nmcli_wifi_ap_bg_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_b_no_sta_interface_auto

Create open 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11b Access Point without any STA connection on {interface} by configuring the system using wifi-ap snap and then checking status of the interface using iw command.
After-suspend:

True
From template:

wireless/wifi_ap_open_b_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_b_no_sta_interface_manual

Create open 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11b Access Point without any STA connection
After-suspend:

True
From template:

wireless/wifi_ap_open_b_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_g_no_sta_interface_auto

Create an open 802.11g Wi-Fi AP on {interface} with no STA connected.

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11g Access Point without any Station (STA) connection on {interface} by configuring the system using the wifi-ap snap and then checking the status of the interface using the iw command.
After-suspend:

True
From template:

wireless/wifi_ap_open_g_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_g_no_sta_interface_manual

Create open 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11g Access Point without any STA connection
After-suspend:

True
From template:

wireless/wifi_ap_open_g_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_setup_wizard_interface_auto

Create Access Point on {interface} using wifi-ap.setup-wizard

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create a WPA2 802.11g Access Point using wifi-ap.setup-wizard command on {interface}.
After-suspend:

True
From template:

wireless/wifi_ap_setup_wizard_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_no_sta_interface_auto

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11b Access Point without any STA connection on {interface} by configuring the system using wifi-ap snap and then checking the status of the interface using iw command.
After-suspend:

True
From template:

wireless/wifi_ap_wpa_b_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_no_sta_interface_manual

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create a WPA2 802.11b Access Point without any STA connected
After-suspend:

True
From template:

wireless/wifi_ap_wpa_b_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_with_sta_interface_auto

Create a WPA2 802.11b Wi-Fi Access Point on {interface} with active STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

wireless/wifi_ap_wpa_b_with_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_no_sta_interface_auto

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11g Access Point without any Station (STA) connection on {interface} by configuring the system using the wifi-ap snap and then checking the status of the interface using the iw command.
After-suspend:

True
From template:

wireless/wifi_ap_wpa_g_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_no_sta_interface_manual

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create a WPA2 802.11g Access Point without any STA connection
After-suspend:

True
From template:

wireless/wifi_ap_wpa_g_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_with_sta_interface_auto

Create WPA2 802.11g Wi-Fi Access Point on {interface} with active STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

wireless/wifi_ap_wpa_g_with_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless networking tests

The following test units are covered in this category:

wireless/check_iwlwifi_microcode_crash_interface

Check there have been no iwlwifi crashes

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
Purpose:

Ensure no crashes have occurred in the iwlwifi microcode.
After-suspend:

True
From template:

wireless/check_iwlwifi_microcode_crash_interface
Template resource:

device
Template filter:

device.driver == ‘iwlwifi’
wireless/wifi7_mlo_interface

Test Wi-Fi 7 Multi-Link-Operation (MLO) on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
Purpose:

Check if the DUT supports Wi-Fi 7 features like Multi-Link-Operation
After-suspend:

True
From template:

wireless/wifi7_mlo_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless Wide Area Network

The following test units are covered in this category:

wwan/3gpp-scan-manufacturer-model-hw_id-auto

Scan for available 3GPP networks with the {model} modem

Unit type:

template
Category ID:

wwan
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Scan for available 3GPP networks with the target modem
After-suspend:

True
From template:

wwan/3gpp-scan-manufacturer-model-hw_id-auto
Template resource:

wwan_resource

Manifest Entries

The following manifest entries are required for certification:

_ignore_disconnected_ethernet_interfaces

Ignore disconnected Ethernet interfaces

Value type:

bool
gpio_loopback

GPIO Loopback Connector

Value type:

bool
Prompt:

Does this device have the following?:
has_apste_support

Disk with Autonomous Power State Transition (APST) support

Value type:

bool
has_audio_capture

Audio capture: Machine can record sound. (For example, a desktop PC probably requires a microphone connected to it)

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_audio_loopback_connector

Audio Loopback Connector

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_audio_playback

Audio playback: Machine can emit sound. (For example, a desktop PC probably requires speakers connected to it)

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_bt_adapter

A Bluetooth Module

Value type:

bool
has_bt_obex_support

A Bluetooth Module with OBject EXchange (OBEX) Support

Value type:

bool
has_bt_smart

A Bluetooth Module with Smart (4.0 or later) Support

Value type:

bool
has_card_reader

Media Card Reader

Value type:

bool
has_dp

DisplayPort

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_dvi

DVI

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_eclite

Has support for Intel ISHTP eclite controller Driver (ECLITE)

Value type:

bool
has_edac_module

An Error Detection And Correction (EDAC) Module

Value type:

bool
has_eeprom

Does device supported EEPROM?

Value type:

bool
has_ethernet_adapter

An Ethernet Port

Value type:

bool
has_ethernet_wake_on_lan_support

Wake-on-LAN support through Ethernet port

Value type:

bool
has_hardware_watchdog

A Hardware Watchdog Timer

Value type:

bool
has_hdmi

HDMI

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_i2c

An I²C bus

Value type:

bool
has_ishtp

Has support for Intel Integrated Sensor Hub Tranport Protocol (ISHTP)

Value type:

bool
has_led_gpio_sysfs

LEDs controlled via GPIO or sysfs

Value type:

bool
Prompt:

Does this machine have the following LED indicators?
has_md_raid

Software RAID

Value type:

bool
has_mei

Has support for Intel Management Engine Interface (MEI)

Value type:

bool
has_qep

Has support for Quadrature Encoder Peripherals

Value type:

bool
has_secure_boot

Secure Boot support

Value type:

bool
has_sim_card

A working SIM card inserted

Value type:

bool
has_socket_can_fd

CAN FD capable devices

Value type:

bool
has_thunderbolt

Thunderbolt Support

Value type:

bool
has_thunderbolt3

Thunderbolt 3 Support

Value type:

bool
has_touchscreen

Touchscreen

Value type:

bool
has_touchscreen_multitouch

Touchscreen Multi-touch Support

Value type:

bool
has_tpm2_chip

TPM 2.0 Support

Value type:

bool
has_usb_dwc3_controller

A DesignWare Core SuperSpeed USB 3.0 Controller (DWC3)

Value type:

bool
has_usb_storage

USB Storage Device Connected

Value type:

bool
has_usbc_data

USB Type-C Data (e.g. HID, Drives, Ethernet)

Value type:

bool
has_usbc_otg

Does the platform support USB-C OTG?

Value type:

bool
has_usbc_video

USB Type-C Video Adapter (USB-C to HDMI/DP/…)

Value type:

bool
has_va_api

Has support for hardware video acceleration (VA API)

Value type:

bool
has_vga

VGA

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_wifi_mlo_environment

Does the test environment have a WiFi 7 router with tri-band (2.4, 5, 6) multi-link-operation (MLO) enabled and are this router’s ssid and password specified in the environment variables MLO_SSID/MLO_PSK?

Value type:

bool
has_wlan_adapter

A Wi-Fi Module

Value type:

bool
has_wwan_module

A WWAN Module

Value type:

bool
socket_can_echo_server_running

A SocketCAN Echo Server

Value type:

bool
Prompt:

Is the device connected to the following?:

client-cert-iot-desktop-26-04

Note

The certification tests presented in this document are validated by Checkbox version 7.3.0.dev42.

Blocking

Audio tests

Output needs to be undistorted between 0%-100%. Output lines tested:

  • Internal speakers

  • 3.5mm headphones

  • HDMI audio output

  • DisplayPort audio output

Input needs to be recorded undistorted between 0%-100%. Input lines tested:

  • Internal microphone

  • 3.5mm microphone

Plug detection: when a new audio line input or output is plugged in the system, it needs to be recognized.

The following test units are covered in this category:

audio/alsa-loopback-automated

Captured sound matches played one (automated)

Category ID:

audio
Status:

Blocking
Purpose:

Check if sound that is ‘hearable’ by capture device. This test is intended mostly for devices without internal speakers/microphones, but with a line-in and a line-out ports to which a loopback cable can be connected.
After-suspend:

True
Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH, LD_LIBRARY_PATH
Plugin:

shell
Requires:

manifest.has_audio_loopback_connector == ‘True’
For details about the required manifest entry, see has_audio_loopback_connector.
audio/alsa_record_playback_automated

Test playback and recording functionality on the default audio input and output.

Category ID:

audio
Status:

Blocking
Purpose:

Play back a sound on the default output and listen for it on the default input. This test is intended mostly for laptops, where the default output will be the internal speakers and the default input will be the internal microphone.
After-suspend:

True
Plugin:

shell
Requires:

package.name == ‘python3-gi’ package.name == ‘gir1.2-gstreamer-1.0’ package.name == ‘libgstreamer1.0-0’ package.name == ‘gstreamer1.0-plugins-good’ package.name in [‘gstreamer1.0-pulseaudio’, ‘gstreamer1.0-pipewire’] package.name == ‘alsa-base’ package.name in [‘pulseaudio-utils’, ‘pipewire’] manifest.has_audio_playback == ‘True’
For details about the required manifest entry, see has_audio_playback.
audio/alsa_record_playback_external

Verify external microphone sound recording and playback.

Category ID:

audio
Status:

Blocking
Purpose:

This test will check that recording sound using an external microphone works correctly
Steps:

  1. Connect a microphone to your microphone port 2. If the port is a 3.5mm combo jack (mic + playback), make sure BOTH the input and output are set to the headphones. 3. Click “Test”, then speak into the external microphone 4. After a few seconds, your speech will be played back to you
Verification:

Did you hear your speech played back? If the port is a combo jack, your speech must be played back through the same headphones used for recording.
After-suspend:

True
Plugin:

user-interact-verify
Requires:

(manifest.has_line_in == ‘True’ or manifest.has_headset == ‘True’) manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gstreamer1.0-plugins-good’ or package.name == ‘gstreamer0.10-plugins-good’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_line_in, has_headset, has_audio_playback.
audio/alsa_record_playback_internal

Test internal microphone recording and playback functionality.

Category ID:

audio
Status:

Blocking
Purpose:

This test will check that recording sound using the onboard microphone works correctly
Steps:

  1. Disconnect any external microphones that you have plugged in 2. Click “Test”, then speak into your internal microphone 3. After a few seconds, your speech will be played back to you.
Verification:

Did you hear your speech played back?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_audio_playback == ‘True’ manifest.has_audio_capture == ‘True’ manifest.has_internal_microphone == ‘True’ package.name == ‘alsa-base’ package.name == ‘gstreamer1.0-plugins-good’ or package.name == ‘gstreamer0.10-plugins-good’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_audio_playback, has_audio_capture, has_internal_microphone.
audio/channels

Verify that all audio channels are functioning correctly by hearing a voice clearly from them.

Category ID:

audio
Status:

Blocking
Purpose:

Check that the various audio channels are working properly
Steps:

  1. Commence the test
Verification:

You should clearly hear a voice from the different audio channels
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_internal_speakers == ‘True’
For details about the required manifest entry, see has_internal_speakers.
audio/detect_sinks

Ensure audio sinks are available for detection.

Category ID:

audio
Status:

Blocking
Purpose:

Test to detect if there are available sinks
After-suspend:

True
Plugin:

shell
Requires:

package.name in [‘pulseaudio-utils’, ‘pipewire’] manifest.has_audio_playback == ‘True’
For details about the required manifest entry, see has_audio_playback.
audio/external-linein

Verify external line-in connection functionality by recording and playback testing.

Category ID:

audio
Status:

Blocking
Purpose:

Check that external line-in connection works correctly
Steps:

  1. Use a cable to connect the line-in port to an external line-out source. 2. Open system sound preferences, ‘Input’ tab, select ‘Line-in’ on the connector list. Commence the test. 3. After a few seconds, your recording will be played back to you.
Verification:

Did you hear your recording?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_line_in == ‘True’ dmi.sane_product == “non-portable” manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gstreamer1.0-plugins-good’ or package.name == ‘gstreamer0.10-plugins-good’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_line_in, has_audio_playback.
audio/external-lineout

Verify external line out connection functionality by inserting a cable to speakers, selecting ‘Line Out’ in system sound preferences, and testing sound channels.

Category ID:

audio
Status:

Blocking
Purpose:

Check that external line out connection works correctly
Steps:

  1. Insert cable to speakers (with built-in amplifiers) on the line out port 2. Open system sound preferences, ‘Output’ tab, select ‘Line Out’ on the connector list. Commence the test 3. On the system sound preferences, click ‘Test Sound’ to check left and right channel
Verification:

  1. Do you see internal speakers? The internal speakers should be replaced by external speakers from the Line out port, if any 2. Do you hear the sound in the internal speakers? The internal speakers should be muted automatically 3. Do you hear the sound coming out on the corresponding channel by external speakers from the Line out port?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_line_out == ‘True’ dmi.sane_product == “non-portable”
For details about the required manifest entry, see has_line_out.
audio/microphone-plug-detection

Ensure that the system can detect when a microphone is plugged in.

Category ID:

audio
Status:

Blocking
Purpose:

Check that system detects a microphone being plugged in
Steps:

  1. Prepare a microphone with a standard 3.5mm jack 2. Locate the microphone jack on the device under test. Keep in mind that it may be shared with the headphone jack. 3. Run the test (you have 30 seconds from now on) 4. Plug the microphone into the appropriate jack 5. Unplug the device for subsequent tests.
Verification:

Verification is automatic, no action is required. The test times out after 30 seconds (and fails in that case).
After-suspend:

True
Plugin:

user-interact
Requires:

(manifest.has_line_in == ‘True’ or manifest.has_headset == ‘True’) manifest.has_audio_playback == ‘True’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_line_in, has_headset, has_audio_playback.
audio/playback_auto

Ensure the internal speakers are working by playing a tone.

Category ID:

audio
Status:

Blocking
Purpose:

This test will check that internal speakers work correctly
Steps:

  1. Make sure that no external speakers or headphones are connected 2. Commence the test to play a brief tone on your audio device
Verification:

Did you hear a tone?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_internal_speakers == ‘True’ manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_internal_speakers, has_audio_playback.
audio/playback_displayport

<missing summary>

Category ID:

audio
Status:

Blocking
Purpose:

DisplayPort audio interface verification
Steps:

  1. Plug an external DisplayPort device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Commence the test
Verification:

Did you hear the sound from the DisplayPort device?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_dp == ‘True’ manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_dp, has_audio_playback.
audio/playback_hdmi

<missing summary>

Category ID:

audio
Status:

Blocking
Purpose:

HDMI audio interface verification
Steps:

  1. Plug an external HDMI device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Commence the test
Verification:

Did you hear the sound from the HDMI device?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_hdmi == ‘True’ manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_hdmi, has_audio_playback.
audio/playback_headphones

Verify headphone connectivity and audio playback quality.

Category ID:

audio
Status:

Blocking
Purpose:

This test will check that the headphones connector works correctly.
Steps:

  1. Connect a pair of headphones to your audio device. 2. If the headphones port is a 3.5mm combo jack (mic + playback), make sure BOTH the input and output are set to the headphones. 3. Commence the test to play a sound through your audio device.
Verification:

Did you hear a sound through the headphones, and did the sound play without any distortion, clicks, or other strange noises from your headphones?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

(manifest.has_line_in == ‘True’ or manifest.has_headset == ‘True’) manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_line_in, has_headset, has_audio_playback.
audio/playback_thunderbolt3

<missing summary>

Category ID:

audio
Status:

Blocking
Purpose:

Thunderbolt audio interface verification
Steps:

  1. Plug an external Thunderbolt device with sound (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Commence the test
Verification:

Did you hear the sound from the Thunderbolt device?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_thunderbolt3 == ‘True’ manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_thunderbolt3, has_audio_playback.
audio/playback_type-c_displayport

<missing summary>

Category ID:

audio
Status:

Blocking
Purpose:

DisplayPort audio via USB Type-C port interface verification
Steps:

  1. Plug an external DisplayPort device with sound on a USB Type-C port using a “USB Type-C to DisplayPort” adapter (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Commence the test
Verification:

Did you hear the sound from the DisplayPort device?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_video == ‘True’ manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_usbc_video, has_audio_playback.
audio/playback_type-c_hdmi

Verify HDMI audio playback through the USB Type-C port using a “USB Type-C to HDMI” adapter and confirm sound output.

Category ID:

audio
Status:

Blocking
Purpose:

HDMI audio via USB Type-C port interface verification
Steps:

  1. Plug an external HDMI device with sound into a USB Type-C port using a “USB Type-C to HDMI” adapter (Use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Begin the test
Verification:

Did you hear the sound from the HDMI device?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_video == ‘True’ manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_usbc_video, has_audio_playback.
audio/speaker-headphone-plug-detection

Ensure the system automatically detects when speakers or headphones are plugged in.

Category ID:

audio
Status:

Blocking
Purpose:

Check that system detects speakers or headphones being plugged in
Steps:

  1. Prepare a pair of headphones or speakers with a standard 3.5mm jack 2. Locate the speaker / headphone jack on the device under test 3. Run the test (you have 30 seconds from now on) 4. Plug headphones or speakers into the appropriate jack 5. Unplug the device for subsequent tests.
Verification:

Verification is automatic, no action is required. The test times out after 30 seconds (and fails in that case).
After-suspend:

True
Plugin:

user-interact
Requires:

(manifest.has_line_out == ‘True’ or manifest.has_headset == ‘True’) manifest.has_audio_playback == ‘True’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_line_out, has_headset, has_audio_playback.

Bluetooth tests

Bluetooth LE (Smart and Smart Ready) is tested for device scanning and pairing. Apart from pairing, several profiles are specifically tested and required:

  • Eddystone Beacon

  • HID Over GATT Profile (HOGP), Low-Energy keyboard or mouse with basic functionality

The following test units are covered in this category:

bluetooth/audio-a2dp

Verify Bluetooth device’s High Fidelity Playback (A2DP) capability.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check the High Fidelity Playback (A2DP) capability of your Bluetooth device, to see if you can hear audio from it.
Steps:

  1. Enable and pair the Bluetooth headset 2. Click “Test” to play a brief tone on your Bluetooth device, if it failed to set the Mode to A2DP, please select the device and change it manually in the “Sound Settings”
Verification:

Did you hear the tone?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_bt_smart == ‘True’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_bt_smart.
bluetooth/audio_record_playback

Verify Bluetooth HSP/HFP profile capability for recording and playback.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check the Headset Head Unit (HSP/HFP) capability of your Bluetooth device, to check if you can record sounds.
Steps:

  1. Enable and pair the bluetooth headset. 2. Click “Test”, then speak into your Bluetooth microphone. 3. After a few seconds, your speech will be played back to you.
Verification:

Did you hear your speech played back?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_bt_smart == ‘True’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_bt_smart.
bluetooth/bluetooth_obex_send

Bluetooth OBEX send

Category ID:

bluetooth
Status:

Blocking
Purpose:

This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable
After-suspend:

True
Environment variable:

BTDEVADDR, PLAINBOX_PROVIDER_DATA
Plugin:

shell
Requires:

package.name == ‘bluez’ or snap.name == ‘bluez’ executable.name == ‘obexftp’ and executable.name == ‘hcitool’ device.category == ‘BLUETOOTH’ manifest.has_bt_obex_support == ‘True’
For details about the required manifest entry, see has_bt_obex_support.
bluetooth/detect

Make sure at least one bluetooth device is detected

Category ID:

bluetooth
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_bt_adapter == ‘True’
For details about the required manifest entry, see has_bt_adapter.
bluetooth/detect-output

Store Bluetooth device information for reports.

Category ID:

bluetooth
Status:

Blocking
Purpose:

Automated test to store Bluetooth device information in the Checkbox report
After-suspend:

True
Plugin:

shell
Requires:

package.name == ‘bluez’ or snap.name == ‘bluez’ device.category == ‘BLUETOOTH’
bluetooth4/HOGP-keyboard

Verify HOGP keyboard functionality with Bluetooth Smart.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart keyboard.
Steps:

  1. Enable a Bluetooth Smart keyboard, and put it into pairing mode. 2. Commence the test to do the auto-pairing, you will be asked to select the targeting keyboard from the list. 3. After it’s paired and connected, enter some text with your keyboard.
Verification:

Did the Bluetooth Smart keyboard work as expected?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_bt_smart == ‘True’ package.name == ‘bluez’
For details about the required manifest entry, see has_bt_smart.
bluetooth4/HOGP-mouse

Test the functionality of a Bluetooth Smart mouse using HID Over GATT Profile.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart mouse.
Steps:

  1. Enable a Bluetooth Smart mouse, and put it into pairing mode. 2. Commence the test to do the auto-pairing; you will be asked to select the targeting mouse from the list. 3. After it’s paired and connected, perform actions such as moving the pointer, right and left button clicks, and double clicks.
Verification:

Did the Bluetooth Smart mouse work as expected?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_bt_smart == ‘True’ package.name == ‘bluez’
For details about the required manifest entry, see has_bt_smart.

CPU tests

x86_64 and ARM processors are tested to ensure proper functionality. We will test specific features as:

  • CPU’s performance states (frequency up and down in runtime)

  • CPU’s sleep states (cpu on and off in runtime)

  • Running CPU at its maximum frequency

We will also include a general stress test performed for 120 minutes to verify that the system can handle a sustained high load for a period of time. This test uses the tool “stress-ng” available in the Universe repositories.

For Intel CPU’s, the IPDT (Intel Processor Diagnostic Tool) test suite will be run. The diagnostic checks for brand identification, verifies the processor operating frequency, tests specific processor features, and performs a stress test on the processor.

The following test units are covered in this category:

cpu/cstates

Run C-States tests

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Uses the Firmware Test Suite (fwts) to test the power saving states of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’ cpuinfo.platform not in (“aarch64”, “armv7l”, “ppc64el”, “ppc64le”, “s390x”)
cpu/maxfreq_test

Test that the CPU can run at its max frequency

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.
Environment variable:

LD_LIBRARY_PATH, SNAP
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’ cpuinfo.platform in (“i386”, “x86_64”)
cpu/offlining_test

Test offlining of each CPU core

Category ID:

cpu
Status:

Blocking
Purpose:

Attempts to offline each core in a multicore system.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

cpu_offlining.state == ‘supported’
cpu/scaling_test

Test the CPU scaling capabilities

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use Firmware Test Suite (fwts cpufreq) to test the scaling capabilities of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’ ‘userspace’ in cpuinfo.governors cpuinfo.platform not in (“ppc64el”, “ppc64le”, “s390x”)
cpu/topology

Check CPU topology for accuracy between proc and sysfs

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.
After-suspend:

True
Plugin:

shell
Requires:

int(cpuinfo.count) > 1 and (cpuinfo.platform == ‘i386’ or cpuinfo.platform == ‘x86_64’)

Disk tests

The following test units are covered in this category:

disk/read_performance_name

Disk performance test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
Purpose:

Verify that disk storage performs at or above baseline performance
After-suspend:

True
From template:

disk/read_performance_name
Template resource:

device
Template filter:

device.category == ‘DISK’
disk/storage_device_name

Disk I/O stress test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
Purpose:

Take the path of the storage device and test if it is a block device.
After-suspend:

True
From template:

disk/storage_device_name
Template resource:

device
Template filter:

device.category == ‘DISK’
thunderbolt3/daisy-chain

Daisy-chain testing for Thunderbolt 3 storage and display device

Category ID:

disk
Status:

Blocking
Purpose:

This test will check if your system can support daisy-chaining of a storage and a monitor over Thunderbolt 3 port
Steps:

  1. Connect your Thunderbolt monitor to your system 2. Connect and mount your Thunderbolt HDD to another Thunderbolt 3 port of the monitor (you can do this with HDD first as well) 3. Click ‘Test’ to perform the storage test on the Thunderbolt HDD
Verification:

  1. The verification for storage is automated, please select the result combined with the result for the display. 2. Was the desktop displayed correctly on the Thunderbolt-connected screen?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

manifest.has_thunderbolt3 == ‘True’
For details about the required manifest entry, see has_thunderbolt3.
thunderbolt3/storage-manual

Thunderbolt 3 HDD storage insertion + read/write + removal

Category ID:

disk
Status:

Blocking
Purpose:

This test will check if the connection of a Thunderbolt 3 HDD could be detected, then performs read/write operations on the attached Thunderbolt 3 HDD storage and checks if the removal of the Thunderbolt 3 HDD can be detected.
Steps:

  1. Click ‘Test’ to begin the test. This test will timeout and fail if the insertion has not been detected within 30 seconds. 2. Plug a Thunderbolt 3 HDD into an available Thunderbolt 3 port, if it’s not mounted automatically, please click the HDD icon to mount it. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the previously attached Thunderbolt 3 HDD.
Verification:

The verification of this test is automated. Do not change the automatically selected result
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

manifest.has_thunderbolt3 == ‘True’
For details about the required manifest entry, see has_thunderbolt3.

Ethernet Device tests

Connections are tested for functionality, but not for performance.

The following test units are covered in this category:

ethernet/detect

Detect if at least one ethernet device is detected

Category ID:

ethernet
Status:

Blocking
Purpose:

Test to detect and return information about available network controllers on the system under test.
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_ethernet_adapter == ‘True’
For details about the required manifest entry, see has_ethernet_adapter.
ethernet/hotplug-interface

Ensure hotplugging works on port {{ interface }}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

Check that hotplugging works on port {{ interface }}
After-suspend:

True
From template:

ethernet/hotplug-interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/iperf3_reverse_interface

Iperf3 stress testing for {interface} (reverse)

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

This test uses iperf3 to ensure network devices pass data at an acceptable minimum percentage of advertised speed (Reverse).
From template:

ethernet/iperf3_reverse_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/iperf3_interface

Iperf3 stress testing for {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

This test uses iperf3 to ensure network devices pass data at an acceptable minimum percentage of advertised speed.
From template:

ethernet/iperf3_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/ping-with-any-cable-interface

Can ping the gateway with any cable Ethernet interface

Category ID:

ethernet
Status:

Blocking
Purpose:

Check any of the cable Ethernet ports available on the system can ping its default gateway.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

device.category == ‘NETWORK’
ethernet/ping_interface

Can ping another machine over Ethernet port {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
Purpose:

Check Ethernet works by pinging another machine
After-suspend:

True
From template:

ethernet/ping_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’

Graphics tests

The following test units are covered in this category:

graphics/VESA_drivers_not_in_use

Test that VESA drivers are not in use

Category ID:

graphics
Status:

Blocking
Purpose:

<missing purpose>
Description:

Check that VESA drivers are not in use
After-suspend:

True
Plugin:

shell
graphics/cycle_resolution_primary

Test resolution cycling

Category ID:

graphics
Status:

Blocking
Purpose:

This test cycles through the detected video modes for the graphics card
Steps:

  1. Click “Test” to start cycling through the video modes
Verification:

Did the screen appear to be working for each mode?
After-suspend:

True
Environment variable:

XDG_CURRENT_DESKTOP, XDG_SESSION_TYPE
Plugin:

user-interact-verify
Requires:

package.name == ‘xorg’
graphics/gl_support_primary

Test OpenGL support

Category ID:

graphics
Status:

Blocking
Purpose:

Check that hardware is able to run a desktop session (OpenGL)
After-suspend:

True
Environment variable:

CHECKBOX_RUNTIME
User:

root
Plugin:

shell
graphics/maximum_resolution_primary

Test maximum supported resolution

Category ID:

graphics
Status:

Blocking
Purpose:

This test will verify the maximum supported resolution on the primary graphics card.
Steps:

  1. Consult the system’s specifications and locate the screen’s maximum supported resolution. 2. Click on Test to display the maximum resolution that can be used by Ubuntu on the current display.
Verification:

Is this the maximum resolution for the display connected to the graphics card?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

device.category == ‘VIDEO’
graphics/rotation_primary

Test rotation

Category ID:

graphics
Status:

Blocking
Purpose:

This test will test display rotation on the graphics card
Steps:

  1. Click “Test” to test display rotation. The display will be rotated every 4 seconds. 2. Try moving the mouse or try opening multiple terminals via ‘Ctrl+Alt+T’ every time the screen automatically turns. 3. Check if all rotations (normal, right, inverted, left) took place without permanent screen corruption.
Verification:

Did the display rotation take place without permanent screen corruption?
After-suspend:

True
Environment variable:

XDG_CURRENT_DESKTOP, XDG_SESSION_TYPE
Plugin:

user-interact-verify
graphics/video_primary

Test that video can be displayed

Category ID:

graphics
Status:

Blocking
Purpose:

This test will test the default display with a sample video
Steps:

  1. Click “Test” to display a video test.
Verification:

Do you see color bars and static?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

package.name == ‘gir1.2-gst-plugins-base-0.10’ or package.name == ‘gir1.2-gst-plugins-base-1.0’
graphics/vulkaninfo-api-version

Test that vulkaninfo api version is right

Category ID:

graphics
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run vulkaninfo - api version check
Environment variable:

CHECKBOX_EXPECTED_VK_API_VERSION
User:

root
Plugin:

shell
Requires:

executable.name == ‘vulkaninfo’ dmi.sane_product in [‘portable’, ‘not-portable’]
graphics/vulkaninfo-debug

Test that vulkaninfo works

Category ID:

graphics
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run vulkaninfo with debug info
User:

root
Plugin:

shell
Requires:

executable.name == ‘vulkaninfo’ dmi.sane_product in [‘portable’, ‘not-portable’]
graphics/index_auto_glxgears_fullscreen_product_slug

Test that glxgears works in full screen mode for {vendor} {product} (automated)

Unit type:

template
Category ID:

graphics
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run glxgears in full screen mode while offloading rendering to the {vendor} {product} GPU and validate the outcome automatically. This test covers all devices with an integrated display, such as laptops and all-in-one (AIO) devices.
After-suspend:

True
From template:

graphics/index_auto_glxgears_fullscreen_product_slug
Template resource:

graphics_card
graphics/index_auto_glxgears_product_slug

Test that glxgears works for {vendor} {product} (automated)

Unit type:

template
Category ID:

graphics
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run glxgears while offloading rendering to the {vendor} {product} GPU and validate the outcome automatically. This test covers all devices with an integrated display, such as laptops and all-in-one (AIO) devices.
After-suspend:

True
From template:

graphics/index_auto_glxgears_product_slug
Template resource:

graphics_card
graphics/index_driver_version_product_slug

Test X driver/version for {vendor} {product}

Unit type:

template
Category ID:

graphics
Status:

Blocking
Purpose:

Parses Xorg.0.log and discovers the running X driver and version for the {vendor} {product} graphics card
After-suspend:

True
From template:

graphics/index_driver_version_product_slug
Template resource:

graphics_card
Template filter:

graphics_card.prime_gpu_offload == ‘Off’
graphics/index_minimum_resolution_product_slug

Test that {vendor} {product} meets minimum resolution requirement

Unit type:

template
Category ID:

graphics
Status:

Blocking
Purpose:

Ensure the current resolution meets or exceeds the recommended minimum resolution (800x600) on the {vendor} {product} graphics card. See here for details: https://help.ubuntu.com/community/Installation/SystemRequirements
After-suspend:

True
From template:

graphics/index_minimum_resolution_product_slug
Template resource:

graphics_card
Template filter:

graphics_card.prime_gpu_offload == ‘Off’
graphics/index_valid_glxgears_fullscreen_product_slug

Test that glxgears works in full screen mode for {vendor} {product} (manual)

Unit type:

template
Category ID:

graphics
Status:

Blocking
Purpose:

This test tests the basic fullscreen 3D capabilities of your {vendor} {product} video card
After-suspend:

True
From template:

graphics/index_valid_glxgears_fullscreen_product_slug
Template resource:

graphics_card
graphics/index_valid_glxgears_product_slug

Test that glxgears works for {vendor} {product} (manual)

Unit type:

template
Category ID:

graphics
Status:

Blocking
Purpose:

This test tests the basic 3D capabilities of your {vendor} {product} video card
After-suspend:

True
From template:

graphics/index_valid_glxgears_product_slug
Template resource:

graphics_card
graphics/index_vkcube_product_slug

Test that vkcube works for {vendor} {product}

Unit type:

template
Category ID:

graphics
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run vkcube while offloading rendering to the {vendor} {product} GPU
From template:

graphics/index_vkcube_product_slug
Template resource:

graphics_card

I²C (Inter-Integrated Circuit)

All devices attached to the I2C bus must be detectable. This includes:

  • Temperature sensors

  • Humidity sensors

  • Accelerometers

The following test units are covered in this category:

i2c/i2c-bus-detect

Check presence of an I²C bus

Category ID:

i2c
Status:

Blocking
Purpose:

If an expected number of I²C buses is provided, the job will verify the detected number is correct. If the expected number of buses is not provided, the job will pass if at least one I²C bus is detected.
Environment variable:

I2C_BUS_NUMBER
User:

root
Plugin:

shell
Requires:

manifest.has_i2c == ‘True’
For details about the required manifest entry, see has_i2c.
i2c/i2c-device-detect

Check if any I²C devices can be detected

Category ID:

i2c
Status:

Blocking
Purpose:

The test will pass if there’s at least one I²C device detected on any I²C bus.
Environment variable:

IGNORED_I2C_BUSES
User:

root
Plugin:

shell

Media Card tests

Media Card readers are tested for read and write for the following type of cards:

  • CF

  • MMC

  • MS

  • MSP

  • SD

  • SDHC

  • SDXC

  • XD

The following test units are covered in this category:

mediacard/sdhc-storage-manual

Test SDHC card insertion + read/write + removal.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a UNLOCKED Secure Digital High-Capacity (SDHC) media card Then it performs a read and write test on the SDHC card. Finally, it checks that the system detects the removal of the SDHC card.
Steps:

  1. Commence the test and then insert an UNLOCKED SDHC card into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the SDHC card from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

manifest.has_card_reader == ‘True’
For details about the required manifest entry, see has_card_reader.

Memory tests

Proper detection of the amount of memory installed is required (the amount of memory installed is the memory seen by the OS).

The following test units are covered in this category:

memory/info

Check the amount of memory reported by meminfo against DMI

Category ID:

memory
Status:

Blocking
Purpose:

This test checks the amount of memory which is reported in meminfo against the size of the memory modules detected by DMI.
User:

root
Plugin:

shell

Miscellaneous tests

The following test units are covered in this category:

miscellanea/check_prerelease

Test that the system is not a pre-release version

Category ID:

miscellanea
Status:

Blocking
Purpose:

Test to verify that the system uses production, rather than pre-release, versions of the kernel and the OS.
Plugin:

shell
Requires:

“Ubuntu Core” not in lsb.description
miscellanea/debsums

Check the MD5 sums of installed Debian packages

Category ID:

miscellanea
Status:

Blocking
Purpose:

Verify installed Debian package files against MD5 checksum lists from /var/lib/dpkg/info/*.md5sums.
User:

root
Plugin:

shell
Requires:

executable.name == ‘debsums’
miscellanea/grub_file_check

Check if the file core.efi exists to make sure shim and grub can be upgraded

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

“Ubuntu Core” not in lsb.description cpuinfo.platform in (“x86_64”, “aarch64”, “armhf”) bootloader.name == “grub”
miscellanea/oops

Run FWTS OOPS check

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (FWTS) oops tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’
miscellanea/ubuntu-desktop-minimal-recommends

Check that all the recommended packages for ubuntu-desktop-minimal are installed

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

package.name == ‘ubuntu-desktop-minimal’
miscellanea/ubuntu-desktop-recommends

Check that all the recommended packages for ubuntu-desktop are installed

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

package.name == ‘ubuntu-desktop’

Monitor tests

Each of the available external video ports (supported ports are HDMI, DisplayPort, DVI) are tested one by one. Output to the display must work i.e. a console is presented.

The following test units are covered in this category:

monitor/dim_brightness

Examine screen brightness adjustment capabilities.

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check changes to screen brightness.
Steps:

  1. Change screen brightness to maximum. 2. Click “Test” to attempt to dim the screen. 3. Check if the screen was dimmed approximately to half of the maximum brightness. 4. The screen will go back to the original brightness in 2 seconds.
Verification:

Was your screen dimmed approximately to half of the maximum brightness?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

dmi.display_type == “integrated”
monitor/displayport

Test the DisplayPort port functionality with mirrored, extended, external, or onboard display modes.

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your DisplayPort port as a monitor interconnect.
Steps:

  1. Connect a display (if not already connected) to the DisplayPort port on your system. 2. Switch display modes in your Display Settings; check if it can be set to mirrored, extended, displayed on external or onboard only.
Verification:

Was the desktop displayed correctly on the DisplayPort-connected screen in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_dp == ‘True’
For details about the required manifest entry, see has_dp.
monitor/displayport_hotplug

Can hotplug monitor (DisplayPort)

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the DisplayPort port and the ability to do hotplugging.
Steps:

Skip this test if your system does not have a DisplayPort port. 1. If a display is already connected, unplug it. 2. (Re-)Connect a display to the DisplayPort port on your system
Verification:

Was the interface displayed correctly on the screen?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_dp == ‘True’
For details about the required manifest entry, see has_dp.
monitor/dvi

Check DVI port functionality by connecting a display and verifying different display modes.

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your DVI port as a monitor interconnect.
Steps:

  1. Connect a display (if not already connected) to the DVI port on your system. 2. Switch display modes within your Display Settings, check if it can be set to mirrored, extended, display-only on external or onboard only.
Verification:

Was the desktop displayed correctly on the DVI-connected screen in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_dvi == ‘True’
For details about the required manifest entry, see has_dvi.
monitor/hdmi

Check HDMI port functionality as a monitor interconnect.

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your HDMI port as a monitor interconnect.
Steps:

  1. Connect a display (if not already connected) to the HDMI port on your system. 2. Switch display modes within your Display Settings, check if it can be set to mirrored, extended, displayed on external or onboard only.
Verification:

Was the desktop displayed correctly on the HDMI-connected screen in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_hdmi == ‘True’
For details about the required manifest entry, see has_hdmi.
monitor/hdmi_hotplug

Can hotplug monitor (HDMI)

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the HDMI port and the ability to do hotplugging.
Steps:

Skip this test if your system does not have a HDMI port. 1. If a display is already connected, unplug it. 2. (Re-)Connect a display to the HDMI port on your system
Verification:

Was the interface displayed correctly on the screen?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_hdmi == ‘True’
For details about the required manifest entry, see has_hdmi.
monitor/multi-head

Verify multi-monitor output functionality on desktop systems.

Category ID:

monitor
Status:

Blocking
Purpose:

This test verifies that multi-monitor output works on your desktop system. This is not the same test as the external monitor tests you would run on your laptop. You will need two monitors to perform this test.
Steps:

Skip this test if your video card does not support multiple monitors. 1. If your second monitor is not already connected, connect it now. 2. Open the “Displays” tool (open the dash and search for “Displays”). 3. Configure your output to provide one desktop across both monitors. 4. Open any application and drag it from one monitor to the next.
Verification:

Was the stretched desktop displayed correctly across both screens?
After-suspend:

True
Plugin:

manual
Requires:

dmi.sane_product == “non-portable”
monitor/powersaving

<missing summary>

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your monitor power saving capabilities
Steps:

  1. Start the test to try the power saving capabilities of your monitor
Verification:

Did the monitor go blank and turn on again after a few seconds?
After-suspend:

True
Plugin:

user-interact-verify
monitor/thunderbolt3

Display connected via Thunderbolt 3

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check your Thunderbolt 3 port as a monitor interconnect.
Steps:

  1. Connect a display (if not already connected) to the Thunderbolt 3 port on your system 2. Switch display modes between in your Display Settings, check if it can be set to mirrored, extended, displayed on external or onboard only
Verification:

Was the desktop displayed correctly on the Thunderbolt-connected screen in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_thunderbolt3 == ‘True’
For details about the required manifest entry, see has_thunderbolt3.
monitor/type-c_displayport

Display connected via DisplayPort using a USB Type-C port

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the connection of a screen using a “USB Type-C to DisplayPort” adapter.
Steps:

  1. Connect a display (if not already connected) to the USB Type-C port on your system using a “USB Type-C to DisplayPort” adapter. 2. Switch display modes within your Display Settings, check if it can be set to mirrored, extended, displayed on external or onboard only.
Verification:

Was the desktop displayed correctly on the screen connected using a “USB Type-C to DisplayPort” adapter in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_usbc_video == ‘True’
For details about the required manifest entry, see has_usbc_video.
monitor/type-c_hdmi

Display connection via HDMI using a USB Type-C port

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the connection of a display using a “USB Type-C to HDMI” adapter.
Steps:

  1. Connect a display (if not already connected) to the USB Type-C port on your system using a “USB Type-C to HDMI” adapter. 2. Switch display modes in your Display Settings, check if it can be set to mirrored, extended, displayed on external or onboard only.
Verification:

Was the desktop displayed correctly on the screen connected using a “USB Type-C to HDMI” adapter in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_usbc_video == ‘True’
For details about the required manifest entry, see has_usbc_video.
monitor/type-c_vga

Display connected via VGA using an USB Type-C port

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the connection of a screen using a “USB Type-C to VGA” adapter.
Steps:

  1. Connect a display (if not already connected) to the USB Type-C port on your system using a “USB Type-C to VGA” adapter 2. Switch display modes between in your Display Settings, check if it can be set to mirrored, extended, displayed on external or onboard only
Verification:

Was the desktop displayed correctly on the screen connected using a “USB Type-C to VGA” adapter in every mode?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_usbc_video == ‘True’
For details about the required manifest entry, see has_usbc_video.

Non-device specific networking tests

The following test units are covered in this category:

networking/gateway_ping

Verifies if the system can connect to the Internet.

Category ID:

networking
Status:

Blocking
Purpose:

Tests whether the system has a working Internet connection.
After-suspend:

True
User:

root
Plugin:

shell
networking/info_device__index___interface

Network Information of device {__index__} ({interface})

Unit type:

template
Category ID:

networking
Status:

Blocking
Purpose:

This test will check the network device {__index__} ({interface})
After-suspend:

True
From template:

networking/info_device__index___interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’
networking/ntp

Test NTP server synchronization capability.

Category ID:

networking
Status:

Blocking
Purpose:

Test to see if we can sync local clock to an NTP server
After-suspend:

True
User:

root
Plugin:

shell
Requires:

(package.name == ‘ntpdate’ or package.name == ‘ntpsec-ntpdate’)

Power Management tests

Warm reboot is tested such that the system must be able to perform the reboot command and services must be restarted such that systemctl does not identify a failed state.

Cold reboot is performed where an RTC is available (see next section). The wakealarm is used to reboot the system after a period of rest and services must be restarted such that systemctl does not identify a failed state.

The following test units are covered in this category:

power-management/cold-reboot

Cold reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This test powers off the system and then powers it on using RTC
Environment variable:

COLD_REBOOT_DELAY, RTC_DEVICE_FILE
User:

root
Plugin:

shell
Requires:

rtc.state == ‘supported’ rtc.wakealarm == ‘supported’
power-management/fwts_wakealarm

Executes ACPI Wakealarm test to validate functionality.

Category ID:

power-management
Status:

Blocking
Purpose:

Test ACPI Wakealarm (fwts wakealarm)
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’
power-management/post-cold-reboot

Post cold reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the cold reboot
User:

root
Plugin:

shell
power-management/post-warm-reboot

Post warm reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the warm reboot
User:

root
Plugin:

shell
power-management/rtc

Test that RTC functions properly (if present)

Category ID:

power-management
Status:

Blocking
Purpose:

Verify that the Real-time clock (RTC) device functions properly, if present.
After-suspend:

True
Environment variable:

RTC_DEVICE_FILE
User:

root
Plugin:

shell
Requires:

rtc.state == ‘supported’ cpuinfo.other != ‘emulated by qemu’
power-management/warm-reboot

Warm reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This tests reboots the system using the reboot command
User:

root
Plugin:

shell

Serial Port

Tests are carried out on ports that provide access via the Linux tty layer. The exact tests performed depend on the physical characteristics of the driver/receiver hardware. The possible tests include:

  • Ensure expected number of devices are available

  • Looped tests:

  • RS232 Ports: perform loopback test to ensure RX/TX

  • RS422/485 Ports: connect together to ensure RX/TX

  • Machine to Machine tests: confirm that a connection can be made to another PC device and RX/TX is operational

The following test units are covered in this category:

serial/loopback-dev

Serial loopback test of {dev}

Unit type:

template
Category ID:

serial
Status:

Blocking
Purpose:

Check if serial port is working hardwired
After-suspend:

True
From template:

serial/loopback-dev
Template resource:

serial_ports_static

Suspend tests

The following test units are covered in this category:

suspend/audio_after_suspend_auto

Verify mixer settings consistency after system suspend.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that mixer settings after suspend are the same as before suspend.
Plugin:

shell
Requires:

manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_audio_playback.
suspend/audio_before_suspend

Record mixer settings before suspending.

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’ package.name in [‘pulseaudio-utils’, ‘pipewire’]
For details about the required manifest entry, see has_audio_playback.
suspend/cpu_after_suspend_auto

Ensure all CPUs are online post-resumption.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that all CPUs are online after resuming.
Plugin:

shell
suspend/cpu_before_suspend

Verify that all the CPUs are online before suspending

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/display_after_suspend

Verify if the display functions normally after a suspend and resume cycle.

Category ID:

suspend
Status:

Blocking
Purpose:

This test will check that the display is correct after suspend and resume
Steps:

  1. Check that your display does not show up visual artifacts after resuming.
Verification:

Does the display work normally after resuming from suspend?
Plugin:

manual
suspend/memory_after_suspend_auto

Ensure all memory is accessible after waking from suspend mode.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that all memory is available after resuming from suspend.
Plugin:

shell
suspend/memory_before_suspend

Dumps memory info to a file for comparison after suspend

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/network_after_suspend_auto

Verify the network functionality after system resume.

Category ID:

suspend
Status:

Blocking
Purpose:

Test the network after resuming.
User:

root
Plugin:

shell
suspend/network_before_suspend

Record the current network before suspending.

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
suspend/oops_after_suspend

Run fwts oops tests post-suspension to ensure system stability.

Category ID:

suspend
Status:

Blocking
Purpose:

Run Firmware Test Suite (fwts) oops tests after suspend.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’
suspend/resolution_after_suspend

Verify resolution consistency after resuming from suspension.

Category ID:

suspend
Status:

Blocking
Purpose:

Test to see that we have the same resolution after resuming as before.
Plugin:

shell
suspend/resolution_before_suspend

Record the current resolution before suspending.

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/validate_suspend_status

Tests that the machine suspended correctly

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

Query sysfs and debugfs for the amount of times the system was suspended. We expect the count in /sys/power/suspend_stats/success and suscess in /sys/kernel/debugfs/suspend_stats to be non zero
User:

root
Plugin:

shell

Touchscreen tests

The following test units are covered in this category:

touchscreen/drag-n-drop

Assess touchscreen functionality for drag & drop tasks.

Category ID:

touchscreen
Status:

Blocking
Purpose:

Check touchscreen drag & drop
Steps:

  1. Press ‘PrtScn’ key to create a screenshot 2. Tap ‘Files’ on dock (launcher bar) to open Home folder 3. Double Tap ‘Pictures’ folder to open Pictures folder 4. Tap and hold the Screenshot* files in the Pictures folder 5. Drag and drop the Screenshot* files to Home folder
Verification:

Does drag and drop work?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_touchscreen == ‘True’
For details about the required manifest entry, see has_touchscreen.
touchscreen/evdev/2-touch-tap-product_slug

Validate proper detection of a 2-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that 2-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/2-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/3-touch-tap-product_slug

Validate proper detection of a 3-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that 3-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/3-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/4-touch-tap-product_slug

Validate the detection of a 4-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that 4-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/4-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/single-touch-tap-product_slug

Validate the detection of a single-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
Purpose:

Validate that single-touch tap is properly detected
After-suspend:

True
From template:

touchscreen/evdev/single-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/multitouch-rotate

Check touchscreen pinch gesture for rotate

Category ID:

touchscreen
Status:

Blocking
Purpose:

Check touchscreen pinch gesture for rotate
Steps:

  1. Commence the test 2. Using 2 fingers, rotate the blue square until it turns green, then release it.
Verification:

Did the blue square rotate following the gesture?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_touchscreen_multitouch == ‘True’
For details about the required manifest entry, see has_touchscreen_multitouch.
touchscreen/multitouch-zoom

Check touchscreen pinch gesture for zoom

Category ID:

touchscreen
Status:

Blocking
Purpose:

Check touchscreen pinch gesture for zoom
Steps:

  1. Commence the test 2. Using 2 fingers, resize the blue square until it turns green, then release it.
Verification:

Did the blue square change size following the gesture?
After-suspend:

True
Plugin:

user-interact-verify
Requires:

manifest.has_touchscreen_multitouch == ‘True’
For details about the required manifest entry, see has_touchscreen_multitouch.

TPM 2.0 (Trusted Platform Module)

On Intel and AMD x86 platforms that include TPM 2.0 compliant modules, it is required that all commands necessary to support Ubuntu’s Full Disk Encryption functionality are supported.

The following test units are covered in this category:

clevis-encrypt-tpm2/detect-ecc-capabilities

Ensure the TPM has required capabilities for clevis ECC test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/detect-rsa-capabilities

Ensure the TPM has required capabilities for clevis RSA test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/ecc

clevis encrypt/decrypt key ecc

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/precheck

clevis encrypt/decrypt precheck

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_tpm2_chip == ‘True’ executable.name == ‘clevis-encrypt-tpm2’
For details about the required manifest entry, see has_tpm2_chip.
clevis-encrypt-tpm2/rsa

clevis encrypt/decrypt key rsa

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell

USB tests

USB 2.0

USB storage devices must work on all available USB ports. USB Human Interface Devices (HID), specifically keyboard or mouse, should be working properly on any USB port.

USB 3.0

USB storage devices must work on all available USB ports. USB Human Interface Devices (HID), specifically keyboard or mouse, should be working properly on any USB port.

USB Type C (USB 3.1)

USB Type C (USB 3.1) supports various types of devices (e.g. Video, Power) through the use of adapters or peripherals. The following adapters/peripherals should work:

  • Storage devices

  • Keyboard or mouse (basic functionality)

  • When DisplayPort over USB Type-C is advertised:

  • Display hot plugging and the following display are required to work:

    mirrored, extended, internal only, external only.

  • Audio output needs to be undistorted over this port.

The following test units are covered in this category:

usb-c/c-to-ethernet-adapter-insert

Check if USB Type-C to Ethernet adapter works

Category ID:

usb
Status:

Blocking
Purpose:

This test will check if system detects network interface of the Type-C to ethernet adapter.
Steps:

  1. Prepare USB Type-C to Ethernet adapter 2. Start the test 3. When the message “INSERT NOW” is shown, plug in the adapter to Type-C port
Verification:

<missing verification>
After-suspend:

True
Plugin:

user-interact
Requires:

manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.
usb-c/storage-manual

USB 3.0 storage device insertion + read/write + removal on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3.0 storage device in a USB Type-C connector. Then it performs a read and write test on the device. Finally, it checks that the system correctly detects the removal of the USB 3 storage. NOTE: Make sure the USB 3 storage device has a partition before starting the test.
Steps:

  1. Commence the test and then insert a USB 3.0 storage device to a USB Type-C port. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 3 storage from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

usb.usb3 == ‘supported’ manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.
usb/detect

Display USB devices attached to SUT

Category ID:

usb
Status:

Blocking
Purpose:

Detects and shows USB devices attached to this system.
After-suspend:

True
Plugin:

shell
Requires:

cpuinfo.platform != ‘s390x’
usb/hid

USB keyboard works

Category ID:

usb
Status:

Blocking
Purpose:

Check USB input device works
Steps:

  1. Connect USB keyboard 2. Input something with USB keyboard
Verification:

What was just input is displayed correctly
After-suspend:

True
Plugin:

manual
usb/storage-detect

Detect storage partitions on a device on the USB bus

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_usb_storage == ‘True’
For details about the required manifest entry, see has_usb_storage.
usb/storage-manual

Test USB 2.0 storage device insertion + read/write + removal.

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect insertion of a USB 2.0 storage device. Then it performs a read and write test on the USB 2.0. Finally, it checks that the system correctly detects the removal of the USB 2.0. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test and then insert an USB 2.0 into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 2.0 from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb3/storage-manual

Test USB 3.0 storage device insertion + read/write + removal.

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect insertion of a USB 3.0 storage device. Then it performs a read and write test on the USB 3.0. Finally, it checks that the system correctly detects the removal of the USB 3.0. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test and then insert an USB 3.0 into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 3.0 from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

usb.usb3 == ‘supported’

Wireless networking tests

Wi-Fi interfaces are tested for connection to access points configured for 802.11 b/g/n/ac/ax protocols.

The following test units are covered in this category:

wireless/detect

Detect if at least one Wireless LAN device is detected

Category ID:

wireless
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_wlan_adapter == ‘True’
For details about the required manifest entry, see has_wlan_adapter.
wireless/wireless_connection_open_ac_nm_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ac AP
After-suspend:

True
From template:

wireless/wireless_connection_open_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ac_np_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ac AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_ac_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_nm_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ax AP
After-suspend:

True
From template:

wireless/wireless_connection_open_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_np_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11ax AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_nm_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11be AP
After-suspend:

True
From template:

wireless/wireless_connection_open_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_np_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11be AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_nm_interface

Connect to an unencrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check the system can connect to an insecure 802.11b/g AP
After-suspend:

True
From template:

wireless/wireless_connection_open_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_np_interface

Connect to unencrypted 802.11b/g Wi-Fi network on {{ interface }} using netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check the system can connect to an unsecured 802.11b/g access point using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_bg_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_nm_interface

Connect to an unencrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check if the system can connect to an unsecured 802.11n access point.
After-suspend:

True
From template:

wireless/wireless_connection_open_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_np_interface

Connect to unencrypted 802.11n Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to insecure 802.11n AP using netplan
After-suspend:

True
From template:

wireless/wireless_connection_open_n_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_nm_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa3 security
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_np_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa3 security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_nm_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa3 security
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_np_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa3 security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_nm_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ac AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_np_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ac AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ac_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_nm_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_np_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11ax AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_nm_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_np_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11be AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_nm_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11b/g AP with wpa security
After-suspend:

True
From template:

wireless/wireless_connection_wpa_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_np_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can connect to 802.11b/g AP with wpa security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_bg_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_nm_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check if the system can connect to an 802.11n AP with WPA security.
After-suspend:

True
From template:

wireless/wireless_connection_wpa_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_np_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }} using netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check if the system can connect to an 802.11n AP with WPA security using netplan
After-suspend:

True
From template:

wireless/wireless_connection_wpa_n_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_scanning_interface

Test system can discover Wi-Fi networks on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
Purpose:

Check system can find a wireless network AP nearby
After-suspend:

True
From template:

wireless/wireless_scanning_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless Wide Area Network

WWAN interfaces are tested for connection to 3G/4G/LTE services.

The following test units are covered in this category:

wwan/check-sim-present-manufacturer-model-hw_id-auto

Check if a SIM card is present in a slot connected to the modem

Unit type:

template
Category ID:

wwan
Status:

Blocking
Purpose:

<missing purpose>
Description:

Check if a SIM card is present in a slot connected to the modem
After-suspend:

True
From template:

wwan/check-sim-present-manufacturer-model-hw_id-auto
Template resource:

wwan_resource
wwan/detect

Identify if WWAN module is missing

Category ID:

wwan
Status:

Blocking
Purpose:

Tests that there is a WWAN module present and indicates that testing of it should follow.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

manifest.has_wwan_module == ‘True’ snap.name == ‘modem-manager’ or package.name == ‘modemmanager’
For details about the required manifest entry, see has_wwan_module.
wwan/gsm-connection-manufacturer-model-hw_id-auto

Verify a GSM broadband modem can create a data connection multiple times

Unit type:

template
Category ID:

wwan
Status:

Blocking
Purpose:

Any modems discovered by the resource job that list GSM support will be tested to ensure a data connection can be made.
After-suspend:

True
From template:

wwan/gsm-connection-manufacturer-model-hw_id-auto
Template resource:

wwan_resource

Non-blocking

Advanced Configuration and Power Interface

The following test units are covered in this category:

acpi/oem_osi

Test ACPI OEM _OSI strings

Category ID:

acpi
Status:

Non-blocking
Purpose:

This checks if the deprecated OEM _OSI strings are still used by checking the ACPI DSDT and SSDT tables
User:

root
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”)

Audio tests

The following test units are covered in this category:

audio/alsa_info_attachment

Attach audio hardware data collection log to the results.

Category ID:

audio
Status:

Non-blocking
Purpose:

Attaches the audio hardware data collection log to the results.
Plugin:

attachment
audio/alsa_info_collect

Collect audio-related system information for simulation and detailed testing.

Category ID:

audio
Status:

Non-blocking
Purpose:

Collect audio-related system information. This data can be used to simulate this computer’s audio subsystem and perform more detailed tests under a controlled environment.
User:

root
Plugin:

shell
audio/detect-capture-devices

Check that at least one audio capture device exists

Category ID:

audio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_audio_capture == ‘True’
For details about the required manifest entry, see has_audio_capture.
audio/detect-playback-devices

Check that at least one audio playback device exists

Category ID:

audio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
Requires:

manifest.has_audio_playback == ‘True’
For details about the required manifest entry, see has_audio_playback.
audio/list_devices

Check if audio devices can be detected.

Category ID:

audio
Status:

Non-blocking
Purpose:

Test to detect audio devices
Plugin:

shell
Requires:

manifest.has_audio_playback == ‘True’ package.name == ‘alsa-base’
For details about the required manifest entry, see has_audio_playback.
audio/post-suspend-audio-occupied

Verify the test audio I/O after suspend with previously occupied devices.

Category ID:

audio
Status:

Non-blocking
Purpose:

Validate the audio could input and output after suspending while audio devices are occupied before suspending
Steps:

  1. Make sure the internal speaker device working normally 2. Commence the test 3. The sound will last for 90 seconds. Please execute the suspend after hearing the sound about 30 seconds 4. Make sure the system enter suspend 5. After the system enter suspend 15 seconds, resume the system back 6. You will hear the remaining 60 seconds of sound
Verification:

Did you hear the sound after suspend?
Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH, LD_LIBRARY_PATH
Plugin:

user-interact-verify
audio/valid-sof-firmware-sig

Ensure SOF firmware signature is valid.

Category ID:

audio
Status:

Non-blocking
Purpose:

Validate SOF firmware signature.
Plugin:

shell
Requires:

manifest.has_audio_playback == ‘True’ package.name == ‘linux-firmware’
For details about the required manifest entry, see has_audio_playback.

Bluetooth - BlueZ Self Tests

The following test units are covered in this category:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test

BlueZ-{bluez-internal-bnep-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

Runs a specific test from the bnep test suite.
After-suspend:

True
From template:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test
Template resource:

bluez-internal-bnep-tests
bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test

BlueZ-{bluez-internal-hci-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Runs a specific test from the hci test suite
After-suspend:

True
From template:

bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test
Template resource:

bluez-internal-hci-tests
bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test

BlueZ-{bluez-internal-rfcomm-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

Runs a specific test from the rfcomm test suite
After-suspend:

True
From template:

bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test
Template resource:

bluez-internal-rfcomm-tests
bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test

BlueZ-{bluez-internal-uc-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Non-blocking
Purpose:

Runs a specific test from the user channel test suite
After-suspend:

True
From template:

bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test
Template resource:

bluez-internal-uc-tests

Bluetooth tests

The following test units are covered in this category:

bluetooth/bluez-controller-detect

Check bluez lists a controller if rfkill detects one

Category ID:

bluetooth
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

package.name == ‘bluez’ or snap.name == ‘bluez’ environment.CHECKBOX_RUNNING_STRICT_SNAP != “1” or (connections.slot == ‘bluez:service’ and connections.plug == ‘{{ __system_env__[“SNAP_NAME”] }}:bluez’)
bluetooth4/beacon_eddystone_url_interface

Test system can get beacon EddyStone URL advertisements on the {{ interface }} adapter

Unit type:

template
Category ID:

bluetooth
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

bluetooth4/beacon_eddystone_url_interface
Template resource:

device
Template filter:

device.category == ‘BLUETOOTH’

Camera tests

The following test units are covered in this category:

camera/camera-quality-rpi_name

Webcam BRISQUE score for rpi devices on {name}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

Uses an image of the camera to get the quality based on a No-Reference image quality assessment algorithm called BRISQUE. This test will timeout and fail if the quality has not been computed within 120 seconds.
From template:

camera/camera-quality-rpi_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images-rpi-attachment_name

Attach an image from the multiple resolution images test on rpi

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

This test will attach one of the images used for the multiple resolution images test on rpi.
From template:

camera/multiple-resolution-images-rpi-attachment_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images-rpi_name

Webcam multiple resolution capture test for Pi Camera

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

Takes multiple pictures based on the resolutions supported by the camera and validates their size and that they are of a valid format.
From template:

camera/multiple-resolution-images-rpi_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images_name

Webcam multiple resolution capture test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Purpose:

Takes multiple pictures based on the resolutions supported by the camera and validates their size and that they are of a valid format.
After-suspend:

True
From template:

camera/multiple-resolution-images_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

CPU tests

The following test units are covered in this category:

cpu/arm64_vfp_support_platform

Validate that the Floating Point Unit is running on {platform} device

Unit type:

template
Category ID:

cpu
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Validate that the Floating Point Unit is running on {platform} device.
From template:

cpu/arm64_vfp_support_platform
Template resource:

cpuinfo
Template filter:

cpuinfo.platform == ‘aarch64’
cpu/armhf_vfp_support_platform

Validate that the Vector Floating Point Unit is running on {platform} device

Unit type:

template
Category ID:

cpu
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Validate that the Vector Floating Point Unit is running on {platform} device.
From template:

cpu/armhf_vfp_support_platform
Template resource:

cpuinfo
Template filter:

cpuinfo.platform == ‘armv7l’
cpu/clocktest

Tests the CPU for clock jitter

Category ID:

cpu
Status:

Non-blocking
Purpose:

Runs a test for clock jitter on SMP machines.
After-suspend:

True
Plugin:

shell
Requires:

cpuinfo.platform not in (“s390x”)
cpu/cstates_results.log

Attach C-States test log

Category ID:

cpu
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission.
Plugin:

attachment
Requires:

cpuinfo.platform not in (“aarch64”, “armv7l”, “s390x”)
cpu/maxfreq_test-log-attach

Attach CPU max frequency log

Category ID:

cpu
Status:

Non-blocking
Purpose:

Attaches the log generated by cpu/maxfreq_test to the results submission.
Plugin:

attachment
cpu/scaling_test-log-attach

Attach CPU scaling capabilities log

Category ID:

cpu
Status:

Non-blocking
Purpose:

Attaches the log generated by cpu/scaling_test to the results submission.
Plugin:

attachment

DesignWare Core SuperSpeed USB 3.0 Controller

The following test units are covered in this category:

usb-dwc3/driver-detect

Detect if the USB DWC3 drivers are loaded

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_usb_dwc3_controller == ‘True’
For details about the required manifest entry, see has_usb_dwc3_controller.
usb-dwc3/mass-storage

Check DUT can be detected as mass storage device

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

Check that after connecting the device under test (DUT) to another device (host), DUT can be detected as mass storage device by the host.
Steps:

  1. In BIOS, enable USB xDCI support. 2. Press Enter to setup a 256 MB FAT32 image on the device. 3. From an Ubuntu (or Windows) host, connect a cable to the USB OTG port of the DUT.
Verification:

The host detects and mounts a mass storage device. It has read and write permissions on it.
User:

root
Plugin:

user-interact-verify
usb-dwc3/mass-storage-cleanup

Cleanup mass storage setup after mass storage device test

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
usb-dwc3/module-detect

Detect if the USB DWC3 module is loaded

Category ID:

usb-dwc3
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_usb_dwc3_controller == ‘True’
For details about the required manifest entry, see has_usb_dwc3_controller.

Disk tests

The following test units are covered in this category:

disk/apste_support_on_name

Check support for Autonomous Power State Transition on {name}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

disk/apste_support_on_name
Template resource:

device
Template filter:

device.driver == ‘nvme’ and device.category == ‘OTHER’
disk/check-software-raid

Validate the configuration of software RAID devices are expected

Category ID:

disk
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Examine the system to detect Software RAID devices are created and the RAID mode are expected the SOFTWARE_RAID_LEVEL variable is needed for this tests. e.g. SOFTWARE_RAID_LEVEL=”raid0 raid1 raid5”
Environment variable:

SOFTWARE_RAID_LEVEL
User:

root
Plugin:

shell
Requires:

executable.name == ‘mdadm’ manifest.has_md_raid == ‘True’
For details about the required manifest entry, see has_md_raid.
disk/disk_stress_ng_name

Disk stress-ng test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
Purpose:

Disk stress-ng test for {product_slug}
From template:

disk/disk_stress_ng_name
Template resource:

device
Template filter:

device.category == ‘DISK’
disk/stats_name

Disk statistics for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
Purpose:

This test checks disk stats, generates some activity and rechecks stats to verify they’ve changed. It also verifies that disks appear in the various files they’re supposed to. . This test will inspect the following disk: . product name: {product_slug} sysfs path: {path} device node path: /dev/{name}
After-suspend:

True
From template:

disk/stats_name
Template resource:

device
Template filter:

device.category == ‘DISK’ and device.name != ‘’

Docker containers

The following test units are covered in this category:

docker/build-single_arch

Test docker build with a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/build-single_arch
Template resource:

docker_resource
docker/commit_arch

Test docker commit a change to a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/commit_arch
Template resource:

docker_resource
docker/compose-and-basic_arch

Test docker compose and basic command

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/compose-and-basic_arch
Template resource:

docker_resource
docker/compose-restart_arch

Test compose a container with restart policy applied

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/compose-restart_arch
Template resource:

docker_resource
docker/compose-single_arch

Test docker compose with a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/compose-single_arch
Template resource:

docker_resource
docker/copy_arch

Test copy a file bwtween a container and local filesystem

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/copy_arch
Template resource:

docker_resource
docker/deploy-registry_arch

Deploy a registry server and run it on localhost

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/deploy-registry_arch
Template resource:

docker_resource
docker/diff_arch

Test changes to files in Ubuntu container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/diff_arch
Template resource:

docker_resource
docker/export-and-import_arch

Test docker import and export a docker container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/export-and-import_arch
Template resource:

docker_resource
docker/info

Display system-wide information about docker

Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
User:

root
Requires:

executable.name == ‘docker’
docker/inspect_arch

Test query low-level information on a docker object

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/inspect_arch
Template resource:

docker_resource
docker/interative_arch

Test an interactive shell in Ubuntu container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/interative_arch
Template resource:

docker_resource
docker/kill_arch

Test killing containers

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/kill_arch
Template resource:

docker_resource
docker/restart-always_arch

Test container restart policy with always applied

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/restart-always_arch
Template resource:

docker_resource
docker/restart-on-failure_arch

Test container restart policy with on_failure applied

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/restart-on-failure_arch
Template resource:

docker_resource
docker/run_arch

Download and run ubuntu container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/run_arch
Template resource:

docker_resource
docker/save-and-load_arch

Test docker save and load a docker image

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/save-and-load_arch
Template resource:

docker_resource
docker/start-stop_arch

Start and stop a single container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/start-stop_arch
Template resource:

docker_resource
docker/update_arch

Test update configuration of one container

Unit type:

template
Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

docker/update_arch
Template resource:

docker_resource
docker/version

Display docker version information

Category ID:

docker
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
User:

root
Requires:

executable.name == ‘docker’

Eeprom

The following test units are covered in this category:

eeprom/read-write

Test EEPROM read and write functions for the system.

Category ID:

eeprom
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

shell
Requires:

manifest.has_eeprom == ‘True’
For details about the required manifest entry, see has_eeprom.

Error Detection And Correction (EDAC) Memory Controllers

The following test units are covered in this category:

edac/default-report

Attach the default report from EDAC util

Category ID:

edac
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

attachment
Requires:

executable.name == ‘edac-util’ manifest.has_edac_module == ‘True’
For details about the required manifest entry, see has_edac_module.
edac/detect

Detect if the EDAC drivers are loaded and Memory Controllers are found

Category ID:

edac
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
Requires:

executable.name == ‘edac-util’ manifest.has_edac_module == ‘True’
For details about the required manifest entry, see has_edac_module.

Ethernet Device tests

The following test units are covered in this category:

ethernet/wol_S3_interface

Wake on LAN (WOL) test from S3 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Non-blocking
Purpose:

Check that another system can wake the System Under Test (SUT) up from S3 using the Ethernet port {interface}’s WOL function.
From template:

ethernet/wol_S3_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’ and device.interface != ‘UNKNOWN’
ethernet/wol_S4_interface

Wake on LAN (WOL) test from S4 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Non-blocking
Purpose:

Check that another system can wake up from S4 the System Under Test (SUT) using ethernet port {interface} Wake-on-LAN (WOL) function.
From template:

ethernet/wol_S4_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’
ethernet/wol_S5_interface

Wake on LAN (WOL) test from S5 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Non-blocking
Purpose:

Check that another system can wake the System Under Test (SUT) from S5 using the Ethernet port {interface} WOL function.
From template:

ethernet/wol_S5_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’

Firmware tests

The following test units are covered in this category:

firmware/fwts_desktop_diagnosis

Run FWTS QA-concerned desktop-specific diagnosis tests.

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (fwts) QA-concerned desktop-specific diagnosis tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
Requires:

executable.name == ‘fwts’
firmware/fwts_desktop_diagnosis_results.log.gz

Attach FWTS desktop diagnosis log to submission

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission
Plugin:

attachment
firmware/fwts_dump_acpi_attachment.gz

Collect the ACPI tables dump from FWTS

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

attachment
firmware/fwupdmgr_get_devices

Collect the device firmware update information

Category ID:

firmware
Status:

Non-blocking
Purpose:

Attach information about the devices, as reported by the fwupdmgr
Plugin:

attachment
Requires:

executable.name in (“fwupdmgr”, “fwupd.fwupdmgr”)

Gathers information about the DUT

The following test units are covered in this category:

cpuinfo

Collect information about the CPU

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Gets CPU resource info from /proc/cpuinfo
User:

root
Plugin:

resource
dpkg

Collect information about dpkg version

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Gets info on the version of dpkg installed
Plugin:

resource
interface

Collect information about interfaces

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of interface declarations on the device
Plugin:

resource
lsb

[DEPRECATED, use ‘os’ instead] Collect information about installed operating system (os-release)

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates release info based on /etc/os-release
Plugin:

resource
meminfo

Collect information about system memory (/proc/meminfo)

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates resource info based on /proc/meminfo
Plugin:

resource
package

Collect information about installed software packages

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of packages
Plugin:

resource
snap

Collect information about installed snap packages

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of snap packages
Plugin:

resource
uname

Collect information about the running kernel

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Creates resource info from uname output
Plugin:

resource

General Purpose I/O

The following test units are covered in this category:

gpio/gpiomem_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via /dev/gpiomem

Unit type:

template
Category ID:

gpio
Status:

Non-blocking
Purpose:

Ensure GPIO lines exposed on headers are controllable via /dev/gpiomem using the RPi python module for specific models.
After-suspend:

True
From template:

gpio/gpiomem_loopback_pairs_model
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)
gpio/sysfs_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template
Category ID:

gpio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

gpio/sysfs_loopback_pairs_model
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)
gpio/sysfs_loopback_pairs_vendor_product

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template
Category ID:

gpio
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

gpio/sysfs_loopback_pairs_vendor_product
Template resource:

dmi
Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’, ‘UPN-EHL01’)

Hardware video acceleration

The following test units are covered in this category:

va-api/va-initialize

Detect if the VA API could be loaded

Category ID:

va-api
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

package.name == ‘vainfo’ cpuinfo.platform == ‘x86_64’ manifest.has_va_api == ‘True’
For details about the required manifest entry, see has_va_api.

Informational tests

The following test units are covered in this category:

acpi_sleep_attachment

Attach the contents of /proc/acpi/sleep for further analysis.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the contents of /proc/acpi/sleep if it exists.
Plugin:

attachment
codecs_attachment

Attach a report of installed codecs for Intel HDA.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a report of installed codecs for Intel HDA
Plugin:

attachment
cpuinfo_attachment

Attach a copy of /proc/cpuinfo

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a report of CPU information
Plugin:

attachment
dkms_info_attachment

Attaches json dumps of installed dkms package information.

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches json dumps of installed dkms package information.
Plugin:

attachment
Requires:

package.name == ‘dkms’
dmesg_attachment

Attach a copy of dmesg or the current dmesg buffer to the test results.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a copy of /var/log/dmesg or the current dmesg buffer to the test results
User:

root
Plugin:

attachment
dmi_attachment

Attach a copy of /sys/class/dmi/id/*

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches info on DMI
Plugin:

attachment
Requires:

dmi_present.state == ‘supported’
dmidecode_attachment

Attach output of dmidecode

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches dmidecode output
User:

root
Plugin:

attachment
Requires:

executable.name == ‘dmidecode’ dmi_present.state == ‘supported’
dpkg_attachment

Attach dpkg -l output

Category ID:

info
Status:

Non-blocking
Purpose:

Attach system debian packages status
Plugin:

attachment
Requires:

executable.name == ‘dpkg’
efi_attachment

Attaches firmware version info

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the firmware version
User:

root
Plugin:

attachment
info/buildstamp

Attaches the buildstamp identifier for the OS

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the buildstamp identifier for the OS
Plugin:

attachment
info/disk_partitions

Attaches info about disk partitions

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches information about disk partitions
User:

root
Plugin:

attachment
info/hdparm_name.txt

Attaches info from hdparm about {name}

Unit type:

template
Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
From template:

info/hdparm_name.txt
Template resource:

device
Template filter:

device.category == ‘DISK’
info/kernel-config-iommu-flag

Check if the kernel is compiled with IOMMU support

Category ID:

info
Status:

Non-blocking
Purpose:

Checks the value of the CONFIG_INTEL_IOMMU_DEFAULT_ON flag in the kernel configuration
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”)
info/secure-boot-check

Check secure boot state

Category ID:

info
Status:

Non-blocking
Purpose:

Output whether secure boot is enabled or disabled
Plugin:

attachment
Requires:

package.name == ‘mokutil’ manifest.has_secure_boot == ‘True’
For details about the required manifest entry, see has_secure_boot.
info/systemd-analyze

System boot-up performance statistics

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
info/systemd-analyze-critical-chain

Print the tree of the time-critical chain of SystemD

Category ID:

info
Status:

Non-blocking
Purpose:

This job prints a tree of the time-critical chain of SystemD units.
Plugin:

shell
info/touchpad_driver

Gather touchpad name, driver name, and driver version information.

Category ID:

info
Status:

Non-blocking
Purpose:

Returns the name, driver name and driver version of any touchpad discovered on the system.
Plugin:

attachment
Requires:

device.category == ‘TOUCHPAD’
installer_debug.gz

Attach the installer’s debug log for diagnostic purposes.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the installer debug log if it exists.
User:

root
Plugin:

attachment
kernel_cmdline_attachment

Attach a copy of /proc/cmdline

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the kernel command line used to boot
Plugin:

attachment
lsmod_attachment

Attach a list of currently running kernel modules

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a list of the currently running kernel modules.
Plugin:

attachment
lspci_attachment

Attach a list of PCI devices

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches very verbose lspci output.
Plugin:

attachment
lspci_standard_config_attachment

Attach PCI configuration space hex dump

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a hex dump of the standard part of the PCI configuration space for all PCI devices.
Plugin:

attachment
lstopo_visual_attachment

Attach the output of lstopo command to present system topology.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the system topology as presented by the lstopo command
Plugin:

attachment
Requires:

executable.name == ‘lstopo’
lsusb_attachment

Attach output of lsusb

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a list of detected USB devices.
After-suspend:

True
User:

root
Plugin:

attachment
manifest

Hardware Manifest

Category ID:

info
Status:

Non-blocking
Purpose:

This job loads the hardware manifest and exposes it as a resource.
Plugin:

resource
meminfo_attachment

Attach copy of /proc/meminfo

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches info on system memory as seen in /proc/meminfo.
Plugin:

attachment
modinfo_attachment

Attach modinfo information

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches modinfo information for all currently loaded modules
Plugin:

attachment
modprobe_attachment

Attach the contents of /etc/modprobe.*

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of the various modprobe conf files.
User:

root
Plugin:

attachment
modules_attachment

Attach the contents of /etc/modules

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of the /etc/modules file.
Plugin:

attachment
sysctl_attachment

Attach sysctl configuration files.

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of various sysctl config files.
User:

root
Plugin:

attachment
sysfs_attachment

Attach detailed sysfs property output from udev

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches a report of sysfs attributes.
Plugin:

attachment
Requires:

cpuinfo.platform not in (“aarch64”)
udev_attachment

Attach dump of udev database

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches a dump of the udev database showing system hardware information.
Plugin:

attachment

Intel Integrated Sensor Hub Transport Protocol (ISHTP)

The following test units are covered in this category:

eclite/module-detect

Verifies that the ishtp_eclite module is loaded

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_eclite == ‘True’
For details about the required manifest entry, see has_eclite.
eclite/temperature-reading-from-thermal-acpitz

Read the temperature of acpitz to ensure Eclite is functional

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

For testing the feature of Eclite, one way is to monitor the temperature of acpitz thermal.
User:

root
Plugin:

shell
ishtp/device-detect

Verify that at least 1 device entry exists in /sys/bus/ishtp/devices

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
ishtp/module-detect

Verifies that the intel_ish_ipc module for ISHTP is loaded

Category ID:

intel-ishtp
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

manifest.has_ishtp == ‘True’
For details about the required manifest entry, see has_ishtp.

Intel Management Engine Interface (MEI)

The following test units are covered in this category:

mei/check-device

Detect if the Intel MEI device is available

Category ID:

mei
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”) manifest.has_mei == ‘True’
For details about the required manifest entry, see has_mei.
mei/check-module

Detect if the Intel MEI kernel module is loaded

Category ID:

mei
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
Requires:

cpuinfo.platform in (“i386”, “x86_64”) manifest.has_mei == ‘True’
For details about the required manifest entry, see has_mei.
mei/get-firmware-version

Retrieve MEI firmware version by MEI interface

Category ID:

mei
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell

LED tests

The following LEDS will be tested for functionality:

  • Cloud LED

  • Wireless LED

  • Bluetooth LED

  • WWAN LED

The following test units are covered in this category:

led-indicator/gpio-controller-leds-name

Check control of {name} LED indicator.

Unit type:

template
Category ID:

led
Status:

Non-blocking
Purpose:

Check that {name} LED turns on and off.
After-suspend:

True
From template:

led-indicator/gpio-controller-leds-name
Template resource:

led-indicator/gpio-controller-leds
led-indicator/gpio-leds-name

Check control of {name} LED indicator.

Unit type:

template
Category ID:

led
Status:

Non-blocking
Purpose:

Check that {name} LED turns on and off.
After-suspend:

True
From template:

led-indicator/gpio-leds-name
Template resource:

led-indicator/gpio-leds
led-indicator/sysfs-leds-manual

Check control of {name} LED.

Unit type:

template
Category ID:

led
Status:

Non-blocking
Purpose:

Check that {name} LED turns on and off.
After-suspend:

True
From template:

led-indicator/sysfs-leds-manual
Template resource:

led-indicator/sysfs-leds

Media Card tests

The following test units are covered in this category:

mediacard/sdxc-storage-manual

Test SDXC card insertion + read/write + removal.

Category ID:

mediacard
Status:

Non-blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Secure Digital Extended Capacity (SDXC) media card Then it performs a read and write test on the SDXC card. Finally, it checks that the system detects the removal of the SDXC card.
Steps:

  1. Commence the test and then insert an UNLOCKED SDXC card into the reader. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the SDXC card from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
Requires:

manifest.has_card_reader == ‘True’
For details about the required manifest entry, see has_card_reader.

Miscellaneous tests

The following test units are covered in this category:

install/apt-get-gets-updates

Ensure apt can access repositories and get updates without installing them, to aid in recovery from broken updates.

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Tests to see that apt can access repositories and get updates (does not install updates). This is done to confirm that you could recover from an incomplete or broken update.
User:

root
Plugin:

shell
Requires:

package.name == ‘apt’
miscellanea/chvt

Verify the system’s ability to switch between a virtual terminal and the X session.

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

This test will check that the system can switch to a virtual terminal and back to X
Steps:

  1. Click “Test” to switch to another virtual terminal and then back to X
Verification:

Did your screen change temporarily to a text console and then switch back to your current session?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

package.name == ‘kbd’
miscellanea/device_check

Device Check

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Device check
Steps:

  1. Commence the test 2. Compare items on System Manifest to the devices known to udev
Verification:

Do the devices reported by udev match the devices on the Manifest?
Plugin:

user-interact-verify
miscellanea/oops_results.log

Attach the FWTS oops results for submission.

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Attaches the FWTS oops results log to the submission
Plugin:

attachment
miscellanea/submission-resources

Check that data for a complete result are present

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

A meta-job that verifies the data necessary for a complete result submission are present. Failure indicates that the results are incomplete and may be rejected.
Plugin:

shell

Monitor tests

The following test units are covered in this category:

monitor/dvi-to-vga

Monitor works (DVI-to-VGA)

Category ID:

monitor
Status:

Non-blocking
Purpose:

Check output to display through VGA adaptor on DVI port
Steps:

  1. Connect display to VGA adaptor on DVI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_dvi == ‘True’
For details about the required manifest entry, see has_dvi.
monitor/hdmi-to-vga

Monitor works (HDMI-to-VGA)

Category ID:

monitor
Status:

Non-blocking
Purpose:

Check output to display through VGA adaptor on HDMI port
Steps:

  1. Connect display to VGA adaptor on HDMI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_hdmi == ‘True’
For details about the required manifest entry, see has_hdmi.
monitor/vga

Monitor works (VGA)

Category ID:

monitor
Status:

Non-blocking
Purpose:

Check output to display through VGA port
Steps:

  1. Connect display to VGA port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_vga == ‘True’
For details about the required manifest entry, see has_vga.

Non-device specific networking tests

The following test units are covered in this category:

ipv6_detect

Test if the kernel is IPv6 ready

Category ID:

networking
Status:

Non-blocking
Purpose:

Test if the running kernel supports IPv6.
Steps:

<missing steps>
Verification:

<missing verification>
After-suspend:

True
networking/predictable_names

Verify that all network interfaces have predictable names.

Category ID:

networking
Status:

Non-blocking
Purpose:

Verify that all network interfaces have predictable names.
Plugin:

shell
Requires:

environment.CHECKBOX_RUNNING_STRICT_SNAP != “1” or lsb.release >= ‘20’ environment.CHECKBOX_RUNNING_STRICT_SNAP != “1” or model_assertion.gadget != “pi” environment.CHECKBOX_RUNNING_STRICT_SNAP == “1” or lsb.release >= ‘18’

Power Management tests

Suspend/Resume

For x86 devices, a 30 cycle suspend/resume stress test is performed using the FWTS. The suspend mode (e.g. S3, S2Idle) used during the test will be the default for the system under test. The test is passed if all 30 cycles complete without failure. Any errors reported in the fwts log for the 30 cycle suspend/resume stress test are informational only and do not affect the outcome of the test, however, we do recommend examining and fixing any failures noted, as they indicate firmware non-compliance with standards.

In addition a single suspend is performed across which the following features and devices are tested:

  • CPU

  • Memory

  • Networking (Wifi, Ethernet)

  • Audio

  • Bluetooth

  • USB controllers

  • Input devices

  • Mediacards

The following test units are covered in this category:

power-management/fwts_wakealarm-log-attach

Attach and display fwts wakealarm test log.

Category ID:

power-management
Status:

Non-blocking
Purpose:

Attach log from fwts wakealarm test
Plugin:

attachment
watchdog/detect

Detect the presence of a Hardware Watchdog

Category ID:

power-management
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
Requires:

manifest.has_hardware_watchdog == ‘True’
For details about the required manifest entry, see has_hardware_watchdog.
watchdog/post-trigger-system-reset-auto

Post watchdog reset service check

Category ID:

power-management
Status:

Non-blocking
Purpose:

Check there are no failed services after the watchdog triggered
User:

root
Plugin:

shell
Requires:

manifest.has_hardware_watchdog == ‘True’
For details about the required manifest entry, see has_hardware_watchdog.
watchdog/systemd-config

Check if the hardware watchdog is properly configured

Category ID:

power-management
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
Requires:

manifest.has_hardware_watchdog == ‘True’
For details about the required manifest entry, see has_hardware_watchdog.
watchdog/trigger-system-reset-auto

Test that the watchdog module can trigger a system reset

Category ID:

power-management
Status:

Non-blocking
Purpose:

Ensure that the watchdog module can successfully initiate a system reset.
User:

root
Plugin:

shell

Quadrature Encoder Peripherals

The following test units are covered in this category:

qep/qep-device-driver-for-qep-device

Verify PCI Device {qep-device} is using the correct driver

Unit type:

template
Category ID:

qep
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Checks that the device exists in lspci and that the device driver associated is correct. Device ID should be 8086:{qep-device} and the driver is always intel_qep.
From template:

qep/qep-device-driver-for-qep-device
Template resource:

qep/qep-devices
qep/qep-device-node-for-qep-device

Verify device directory exists for {qep-device}

Unit type:

template
Category ID:

qep
Status:

Non-blocking
Purpose:

Detects if the device’s directory exists.
From template:

qep/qep-device-node-for-qep-device
Template resource:

qep/qep-devices

Real Time Clock (RTC)

The following test units are covered in this category:

rtc/rtc_alarm_rtc

Check that RTC alarm of {rtc} works

Unit type:

template
Category ID:

rtc_category
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

rtc/rtc_alarm_rtc
Template resource:

rtc_list
rtc/rtc_clock_rtc

Check that {rtc} clock is synchronized with system clock

Unit type:

template
Category ID:

rtc_category
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

rtc/rtc_clock_rtc
Template resource:

rtc_list
rtc/rtc_number

Check the number of RTC as expected.

Category ID:

rtc_category
Status:

Non-blocking
Purpose:

Check the number of RTC on platform as expected.
After-suspend:

True
Environment variable:

TOTAL_RTC_NUM
User:

root
Plugin:

shell

Snapd

The following test units are covered in this category:

snappy/snap-install

Test the snap install command is working

Category ID:

snapd
Status:

Non-blocking
Purpose:

The store should contain the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap. The test makes sure this can be downloaded and installed on the system.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-list

Test that the snap list command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

If snap list command is working then should at least find the ubuntu-core package.
Plugin:

shell
snappy/snap-refresh-automated

Test whether the snap refresh command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

The test will install the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap from the stable channel and then refreshes it to the edge channel and compares the revision before and after the refresh.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-remove

Test the snap remove command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

After having installed the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap, check it can be removed.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-reupdate-automated

Test the snap refresh command works after blacklisting.

Category ID:

snapd
Status:

Non-blocking
Purpose:

Checks that the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap can be refreshed after removal of the blacklisted revision.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-revert-automated

Test the snap revert command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

Checks if the edge channel {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap is reverted back to the one from stable.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-snaps-confinement

Test all the snaps’ confinement

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test all the snaps’ confinement, devmode, revision. Make sure the confinement is “strict”, devmode is “False”, and revision should not be sideloaded.
Plugin:

shell
snappy/test-store-config-store

Test that image is using the correct snappy store configuration.

Unit type:

template
Category ID:

snapd
Status:

Non-blocking
Purpose:

The image can be tied to using a particular store for the OEM. This tests the store for the image is as expected.
From template:

snappy/test-store-config-store
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.store not in (‘unknown’)
snappy/test-store-install-beta

Snappy install command - beta channel store

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test the snappy install command is able to install and remove snap in beta channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-store-install-edge

Snappy install command - edge channel store

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test the snappy install command is able to install a snap in the edge channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-system-confinement

Test if the system confinement is strict

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test if the system confinement is “strict” If not, list the missing features
Plugin:

shell

SocketCAN interface tests

Devices that support the SocketCAN standard are tested to ensure that the adapter is present and can be communicated with via CANBus configuration commands.

The following test units are covered in this category:

socketcan/send_packet_local_eff_virtual

Virtual CAN device support test (Local test with raw socket and EFF)

Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_eff_interface

CAN device support test for {interface} (Raw, Local, EFF)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket, this is only a local test as the broadcast packet is received on the same device.
After-suspend:

True
From template:

socketcan/send_packet_local_eff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_local_fd_virtual

Virtual CAN device support test (Raw, Local, FD)

Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket; this is only a local test as the broadcast packet is received on the same device.
After-suspend:

True
User:

root
Plugin:

shell
Requires:

manifest.has_socket_can_fd == ‘True’
For details about the required manifest entry, see has_socket_can_fd.
socketcan/send_packet_local_fd_interface

CAN device support test for {interface} (Raw, Local, FD)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
From template:

socketcan/send_packet_local_fd_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_local_sff_virtual

Virtual CAN device support test (Raw, Local)

Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_sff_interface

CAN device support test for {interface} (Raw, Local)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
From template:

socketcan/send_packet_local_sff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_eff_interface

CAN device support test {interface} (Raw, Remote, EFF)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket to a remote device. As a prerequisite the remote device should have can-echo-server installed so as to return the predicted packet.
After-suspend:

True
From template:

socketcan/send_packet_remote_eff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_fd_interface

CAN device support test {interface} (Raw, Remote, FD)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket to a remote device. As a prerequisite, the remote device should have can-echo-server installed so as to return the predicted packet.
After-suspend:

True
From template:

socketcan/send_packet_remote_fd_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_sff_interface

CAN device support test {interface} (Raw, Remote)

Unit type:

template
Category ID:

socketcan
Status:

Non-blocking
Purpose:

Test a CAN device by sending packets using a raw socket to a remote device. As a prerequisite, the remote device should have can-echo-server installed so as to return the predicted packet.
After-suspend:

True
From template:

socketcan/send_packet_remote_sff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’

Suspend tests

The following test units are covered in this category:

suspend/any_suspend_failure

Tests if any device in this machine reports a suspend failure

Category ID:

suspend
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

If the DUT suspended successfully, /sys/power/suspend_stats/fail* and fail* in /sys/kernel/debug/suspend_stats will be zero. If a device failed /sys/power/suspend_stats/last_failed_dev and last_failed_dev in /sys/kernel/debug/suspend_stats will show the failed device
User:

root
Plugin:

shell
suspend/oops_results_after_suspend.log

Attach FWTS oops results log post-suspend.

Category ID:

suspend
Status:

Non-blocking
Purpose:

Attaches the FWTS oops results log to the submission after suspend
Plugin:

attachment
suspend/suspend-single-log-attach

Attaches the log from the single suspend/resume test

Category ID:

suspend
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

attachment
suspend/suspend-time-check

Ensure time to suspend/resume is under threshold

Category ID:

suspend
Status:

Non-blocking
Purpose:

Checks the sleep times to ensure that a machine suspends and resumes within a given threshold
Plugin:

shell
suspend/xrandr_screens_after_suspend_primary

Attach screenshots from suspend/cycle resolutions after suspend test to results.

Category ID:

suspend
Status:

Non-blocking
Purpose:

This attaches screenshots from the suspend/cycle_resolutions_after_suspend test to the results submission.
Plugin:

attachment

TPM 2.0 (Trusted Platform Module)

The following test units are covered in this category:

tpm2/fwts-event-log-dump

Dump the contents of the TPM Event Log

Category ID:

tpm2
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

The information in the TPM Event Log can be useful in debugging problems with TPM command support and adherance to standards. This can be of particular help when checking whether a device can support Ubuntu Core Full Disk Encryption.
User:

root
Plugin:

attachment
Requires:

manifest.has_tpm2_chip == ‘True’ executable.name == ‘fwts’
For details about the required manifest entry, see has_tpm2_chip.

USB tests

The following test units are covered in this category:

usb-c-otg/g_ether

Check DUT can be detected as USB ethernet device

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Steps:

  1. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT. 2. Press Enter to configure the IP address for the USB OTG ethernet interface on the DUT. (IP will be 192.168.9.1/24) 3. On the host, configure the IP address for the USB OTG ethernet interface. It should be in the same subnet as the DUT. (e.g., 192.168.9.2/24) 4. Try to ping between the host and DUT.
Verification:

The host and DUT can ping each other.
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_ether-cleanup

Cleanup USB OTG ethernet interface setup after ethernet device test

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_mass_storage

Check DUT can be detected as a mass storage device

Category ID:

usb
Status:

Non-blocking
Purpose:

Check that after connecting the device under test (DUT) to another device (host), the DUT can be detected as a mass storage device by the host.
Steps:

  1. Press Enter to setup a 16 MB FAT32 image on the device. 2. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT.
Verification:

The host detects and mounts the mass storage device. It has read and write permissions on it.
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_mass_storage-cleanup

Cleanup mass storage setup after mass storage device test

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_serial

Check if USB OTG can work as a serial port.

Category ID:

usb
Status:

Non-blocking
Purpose:

Check if USB OTG can work as a serial port.
Steps:

  1. Press enter to probe USB OTG g_serial 2. Connect a USB cable from an Ubuntu host to the USB OTG (it’s the USB Type C connector) port on DUT. 3. On the host, check if ttyACM0 has been probed $ ls /dev/ttyACM0 4. On the host, listen to the ttyACM0 port as the receiving side. $ sudo su $ cat /dev/ttyACM0 5. On DUT, send a string via /dev/ttyGS0 as the sending side. $ sudo su $ echo 123 > /dev/ttyGS0 6. Check if the string “123” was received on the host side. 7. Check the string sending from the host to DUT by repeating steps 4-6, but swap the send and receive sides.
Verification:

Does string send and receive function correctly?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c-otg/g_serial-cleanup

Cleanup USB OTG serial interface setup after serial device test

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
Requires:

manifest.has_usbc_otg == ‘True’
For details about the required manifest entry, see has_usbc_otg.
usb-c/c-to-a-adapter/hid

USB HID work on USB Type-C port using a “USB Type-C to Type-A” adapter

Category ID:

usb
Status:

Non-blocking
Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port using a “USB Type-C to Type-A” adapter
Steps:

  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port using a “USB Type-C to Type-A” adapter 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text
Verification:

Did the device work as expected?
After-suspend:

True
Plugin:

manual
Requires:

manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.
usb-c/c-to-a-adapter/storage-manual

Test USB 3 storage device insertion + read/write + removal using a “Type-C to Type-A” adapter.

Category ID:

usb
Status:

Non-blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3 storage device in a USB Type-C connector using a “Type-C to Type-A” adapter. Then it performs a read and write test on the device. Finally, it checks that the system correctly detects the removal of the USB 3 storage. NOTE: Make sure the USB 3 storage device has a partition before starting the test.
Steps:

  1. Commence and then insert a USB 3 storage device to a USB Type-C port using a “Type-C to Type-A” adapter. (Note: this test will time-out after 30 seconds.) 2. Do not remove the device after this test. 3. Wait for the read/write operations to complete. 4. Press Enter to start the removal test. 5. Remove the USB 3 storage from the reader. (Note: this test will time-out after 30 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
Requires:

usb.usb3 == ‘supported’ manifest.has_usbc_data == ‘True’
For details about the required manifest entry, see has_usbc_data.

Wi-Fi access point

The following test units are covered in this category:

wireless/nmcli_wifi_ap_a_interface

Create 802.11a Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

wireless/nmcli_wifi_ap_a_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/nmcli_wifi_ap_bg_interface

Create 802.11b/g Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Create an 802.11b/g Wi-Fi access point on a specified interface using NetworkManager command-line tools.
After-suspend:

True
From template:

wireless/nmcli_wifi_ap_bg_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_b_no_sta_interface_auto

Create open 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11b Access Point without any STA connection on {interface} by configuring the system using wifi-ap snap and then checking status of the interface using iw command.
After-suspend:

True
From template:

wireless/wifi_ap_open_b_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_b_no_sta_interface_manual

Create open 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11b Access Point without any STA connection
After-suspend:

True
From template:

wireless/wifi_ap_open_b_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_g_no_sta_interface_auto

Create an open 802.11g Wi-Fi AP on {interface} with no STA connected.

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11g Access Point without any Station (STA) connection on {interface} by configuring the system using the wifi-ap snap and then checking the status of the interface using the iw command.
After-suspend:

True
From template:

wireless/wifi_ap_open_g_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_g_no_sta_interface_manual

Create open 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11g Access Point without any STA connection
After-suspend:

True
From template:

wireless/wifi_ap_open_g_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_setup_wizard_interface_auto

Create Access Point on {interface} using wifi-ap.setup-wizard

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create a WPA2 802.11g Access Point using wifi-ap.setup-wizard command on {interface}.
After-suspend:

True
From template:

wireless/wifi_ap_setup_wizard_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_no_sta_interface_auto

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11b Access Point without any STA connection on {interface} by configuring the system using wifi-ap snap and then checking the status of the interface using iw command.
After-suspend:

True
From template:

wireless/wifi_ap_wpa_b_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_no_sta_interface_manual

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create a WPA2 802.11b Access Point without any STA connected
After-suspend:

True
From template:

wireless/wifi_ap_wpa_b_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_with_sta_interface_auto

Create a WPA2 802.11b Wi-Fi Access Point on {interface} with active STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

wireless/wifi_ap_wpa_b_with_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_no_sta_interface_auto

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create an open 802.11g Access Point without any Station (STA) connection on {interface} by configuring the system using the wifi-ap snap and then checking the status of the interface using the iw command.
After-suspend:

True
From template:

wireless/wifi_ap_wpa_g_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_no_sta_interface_manual

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

Check that the system can create a WPA2 802.11g Access Point without any STA connection
After-suspend:

True
From template:

wireless/wifi_ap_wpa_g_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_with_sta_interface_auto

Create WPA2 802.11g Wi-Fi Access Point on {interface} with active STA

Unit type:

template
Category ID:

wifi_ap
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
From template:

wireless/wifi_ap_wpa_g_with_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless networking tests

The following test units are covered in this category:

wireless/check_iwlwifi_microcode_crash_interface

Check there have been no iwlwifi crashes

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
Purpose:

Ensure no crashes have occurred in the iwlwifi microcode.
After-suspend:

True
From template:

wireless/check_iwlwifi_microcode_crash_interface
Template resource:

device
Template filter:

device.driver == ‘iwlwifi’
wireless/wifi7_mlo_interface

Test Wi-Fi 7 Multi-Link-Operation (MLO) on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
Purpose:

Check if the DUT supports Wi-Fi 7 features like Multi-Link-Operation
After-suspend:

True
From template:

wireless/wifi7_mlo_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless Wide Area Network

The following test units are covered in this category:

wwan/3gpp-scan-manufacturer-model-hw_id-auto

Scan for available 3GPP networks with the {model} modem

Unit type:

template
Category ID:

wwan
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Scan for available 3GPP networks with the target modem
After-suspend:

True
From template:

wwan/3gpp-scan-manufacturer-model-hw_id-auto
Template resource:

wwan_resource

Manifest Entries

The following manifest entries are required for certification:

_ignore_disconnected_ethernet_interfaces

Ignore disconnected Ethernet interfaces

Value type:

bool
gpio_loopback

GPIO Loopback Connector

Value type:

bool
Prompt:

Does this device have the following?:
has_apste_support

Disk with Autonomous Power State Transition (APST) support

Value type:

bool
has_audio_capture

Audio capture: Machine can record sound. (For example, a desktop PC probably requires a microphone connected to it)

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_audio_loopback_connector

Audio Loopback Connector

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_audio_playback

Audio playback: Machine can emit sound. (For example, a desktop PC probably requires speakers connected to it)

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_bt_adapter

A Bluetooth Module

Value type:

bool
has_bt_obex_support

A Bluetooth Module with OBject EXchange (OBEX) Support

Value type:

bool
has_bt_smart

A Bluetooth Module with Smart (4.0 or later) Support

Value type:

bool
has_card_reader

Media Card Reader

Value type:

bool
has_dp

DisplayPort

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_dvi

DVI

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_eclite

Has support for Intel ISHTP eclite controller Driver (ECLITE)

Value type:

bool
has_edac_module

An Error Detection And Correction (EDAC) Module

Value type:

bool
has_eeprom

Does device supported EEPROM?

Value type:

bool
has_ethernet_adapter

An Ethernet Port

Value type:

bool
has_ethernet_wake_on_lan_support

Wake-on-LAN support through Ethernet port

Value type:

bool
has_hardware_watchdog

A Hardware Watchdog Timer

Value type:

bool
has_hdmi

HDMI

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_headset

Headset (Headphones + Microphone) port

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_i2c

An I²C bus

Value type:

bool
has_internal_microphone

Internal microphone

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_internal_speakers

Internal speakers

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_ishtp

Has support for Intel Integrated Sensor Hub Tranport Protocol (ISHTP)

Value type:

bool
has_led_gpio_sysfs

LEDs controlled via GPIO or sysfs

Value type:

bool
Prompt:

Does this machine have the following LED indicators?
has_line_in

Line in (Microphone) port

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_line_out

Line out (Headphones) port

Value type:

bool
Prompt:

Does this machine have the following audio features?
has_md_raid

Software RAID

Value type:

bool
has_mei

Has support for Intel Management Engine Interface (MEI)

Value type:

bool
has_qep

Has support for Quadrature Encoder Peripherals

Value type:

bool
has_secure_boot

Secure Boot support

Value type:

bool
has_sim_card

A working SIM card inserted

Value type:

bool
has_socket_can_fd

CAN FD capable devices

Value type:

bool
has_thunderbolt

Thunderbolt Support

Value type:

bool
has_thunderbolt3

Thunderbolt 3 Support

Value type:

bool
has_touchscreen

Touchscreen

Value type:

bool
has_touchscreen_multitouch

Touchscreen Multi-touch Support

Value type:

bool
has_tpm2_chip

TPM 2.0 Support

Value type:

bool
has_usb_dwc3_controller

A DesignWare Core SuperSpeed USB 3.0 Controller (DWC3)

Value type:

bool
has_usb_storage

USB Storage Device Connected

Value type:

bool
has_usbc_data

USB Type-C Data (e.g. HID, Drives, Ethernet)

Value type:

bool
has_usbc_otg

Does the platform support USB-C OTG?

Value type:

bool
has_usbc_video

USB Type-C Video Adapter (USB-C to HDMI/DP/…)

Value type:

bool
has_va_api

Has support for hardware video acceleration (VA API)

Value type:

bool
has_vga

VGA

Value type:

bool
Prompt:

Does this machine have the following graphics ports?
has_wifi_mlo_environment

Does the test environment have a WiFi 7 router with tri-band (2.4, 5, 6) multi-link-operation (MLO) enabled and are this router’s ssid and password specified in the environment variables MLO_SSID/MLO_PSK?

Value type:

bool
has_wlan_adapter

A Wi-Fi Module

Value type:

bool
has_wwan_module

A WWAN Module

Value type:

bool
socket_can_echo_server_running

A SocketCAN Echo Server

Value type:

bool
Prompt:

Is the device connected to the following?:

Appendix A. FWTS tests

As part of the certification process, we run a series of firmware tests that are part of the Canonical Firmware Test Suite. In general, any HIGH or CRITICAL error found in the fwts log can cause potential errors in the system and should be looked at by OEMs/ODMs.

Category

Test Item

Description

Information

acpidump

Check ACPI table acpidump.

Information

version

Gather kernel system information.

ACPI

acpitables

ACPI table settings confidence checks.

ACPI

apicinstance

Check for single instance of APIC/MADT table.

ACPI

hpet_check

High Precision Event Timer configuration test.

ACPI

mcfg

MCFG PCI Express* memory mapped config space.

ACPI

method

ACPI DSDT Method Semantic Tests.

CPU

mpcheck

Check Multi Processor tables.

CPU

msr

CPU MSR consistency check.

CPU

mtrr

MTRR validation.

System

apicedge

APIC Edge/Level Check.

System

klog

Scan kernel log for errors and warnings.