Internet of Things Certified Hardware Coverage for Ubuntu Core 22 / Ubuntu 22.04

Introduction

This document lists the coverage for certification of Internet of Things (IoT) devices with Ubuntu images. IoT devices can be certified with the following image types:

  • Ubuntu Core 22

  • Ubuntu Server 22.04 LTS

  • Ubuntu Desktop 22.04 LTS

The guide applies to devices submitted to Canonical through one of the following programmes:

  • IoT Devices Enablement Programme with Certification

  • IoT ODM Partner Programme

For each test job, one of the following certification statuses is specified:

Blocking

Features that are required for certification. If any of the blocking tests fails, the certification will fail.

Non-blocking

Features that are tested but not mandatory for certification. Failure in non-blocking tests will not prevent certification. However, a note will be added to the certificate to inform potential customers or users.

Note

Only categories of hardware are tested and not specific types of hardware. For example, tests are run to verify USB controllers work, but the type of peripheral(s) used during those tests are not specified.

Coverage is flexible based on customer requirements (for example, if a device’s use cases don’t require LEDs, then LEDs can be non-blocking)

Full test descriptions can be found in Canonical certification site for partners:

https://certification.canonical.com

client-cert-iot-ubuntucore-22

Note

The certification tests presented in this document are validated by Checkbox version 4.2.0.dev76.

Blocking

Advanced Configuration and Power Interface

The following test units are covered in this category:

acpi/oem_osi

Test ACPI OEM _OSI strings

Category ID:

acpi
Status:

Blocking
Purpose:

This checks if the deprecated OEM _OSI strings are still used by checking the ACPI DSDT and SSDT tables
User:

root
Plugin:

shell

Audio tests

The following test units are covered in this category:

audio/alsa-loopback-automated

Captured sound matches played one (automated)

Category ID:

audio
Status:

Blocking
Purpose:

Check if sound that is ‘hearable’ by capture device. This test is intended mostly for devices without internal speakers/microphones, but with a line-in and a line-out ports to which a loopback cable can be connected.
After-suspend:

True
Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH, LD_LIBRARY_PATH
User:

root
Plugin:

shell
audio/alsa-playback

Playback works

Category ID:

audio
Status:

Blocking
Purpose:

Check if sound is played through ALSA on the default output
Steps:

  1. Make sure speakers or headphones are connected to the device 2. Commence the test
Verification:

Did you hear the sound?
After-suspend:

True
Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH
User:

root
Plugin:

user-interact-verify
audio/detect-capture-devices

Check that at least one audio capture device exists

Category ID:

audio
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
audio/detect-playback-devices

Check that at least one audio playback device exists

Category ID:

audio
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell

Bluetooth - BlueZ Self Tests

The following test units are covered in this category:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test

BlueZ-{bluez-internal-bnep-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Blocking
After-suspend:

True
From template:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test
Template resource:

bluez-internal-bnep-tests
bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test

BlueZ-{bluez-internal-hci-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Blocking
Description:

Runs a specific test from the hci test suite
After-suspend:

True
From template:

bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test
Template resource:

bluez-internal-hci-tests
bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test

BlueZ-{bluez-internal-rfcomm-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Blocking
After-suspend:

True
From template:

bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test
Template resource:

bluez-internal-rfcomm-tests
bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test

BlueZ-{bluez-internal-uc-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Blocking
After-suspend:

True
From template:

bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test
Template resource:

bluez-internal-uc-tests

Bluetooth tests

The following test units are covered in this category:

bluetooth/bluetooth_obex_send

Bluetooth OBEX send

Category ID:

bluetooth
Status:

Blocking
Purpose:

This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable
After-suspend:

True
Environment variable:

BTDEVADDR, PLAINBOX_PROVIDER_DATA
Plugin:

shell
bluetooth/bluez-controller-detect

Check bluez lists a controller if rfkill detects one

Category ID:

bluetooth
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
bluetooth/detect

Make sure at least one bluetooth device is detected

Category ID:

bluetooth
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
bluetooth/keyboard-manual

Bluetooth keyboard manual test

Category ID:

bluetooth
Status:

Blocking
Purpose:

Check Bluetooth input device works
Steps:

  1. Run the following command to start bluetoothctl console: sudo bluetoothctl ***Following steps are run in bluetoothctl console*** 2. Power on the device: power on 3. Register agent for keyboard: agent KeyboardOnly default-agent 4. Put controller in pairable mode: pairable on 5. Scan nearby Bluetooth device: scan on 6. Stop Scanning after Bluetooth keyboard is found: scan off 7. Pair Bluetooth pair [MAC address of Bluetooth keyboard] 8. Enter PIN Code on Bluetooth keyboard if needed 9. Trust the Bluetooth keyboard trust [MAC address of Bluetooth keyboard] 10. Connect to Bluetooth keyboard: connect [MAC address of Bluetooth keyboard] 11. Quit bluetoothctl console quit 12. Use Bluetooth keyboard to input **For headless, please check the input outside the Bluetooth console by using: $ sudo cat /dev/tty1
Verification:

Confirm characters from Bluetooth input device are displayed correctly
After-suspend:

True
Plugin:

manual
bluetooth/zapper-a2dp

Check if the system can connect to a Bluetooth speaker using A2DP profile

Category ID:

bluetooth
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

ZAPPER_HOST
Plugin:

shell
bluetooth4/beacon_eddystone_url_interface

Test system can get beacon EddyStone URL advertisements on the {interface} adapter

Unit type:

template
Category ID:

bluetooth
Status:

Blocking
Description:

<missing description>
After-suspend:

True
From template:

bluetooth4/beacon_eddystone_url_interface
Template resource:

device
Template filter:

device.category == ‘BLUETOOTH’

Camera tests

The following test units are covered in this category:

camera/multiple-resolution-images-rpi-attachment_name

Attach an image from the multiple resolution images test on rpi

Unit type:

template
Category ID:

camera
Status:

Blocking
Description:

This test will attach one of the images used for the multiple resolution images test.
From template:

camera/multiple-resolution-images-rpi-attachment_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images-rpi_name

Webcam multiple resolution capture test for Pi Camera

Unit type:

template
Category ID:

camera
Status:

Blocking
From template:

camera/multiple-resolution-images-rpi_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images_name

Webcam multiple resolution capture test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Blocking
After-suspend:

True
From template:

camera/multiple-resolution-images_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’
camera/roundtrip-qrcode_name

Test video output and camera {{ name }} by displaying and reading a QR code

Unit type:

template
Category ID:

camera
Status:

Blocking
From template:

camera/roundtrip-qrcode_name
Template resource:

device
Template filter:

device.category in (‘CAPTURE’, ‘MMAL’) and device.name != ‘’

CPU tests

The following test units are covered in this category:

cpu/arm64_vfp_support_platform

Validate that the Floating Point Unit is running on {platform} device

Unit type:

template
Category ID:

cpu
Status:

Blocking
Description:

Validate that the Floating Point Unit is running on {platform} device.
From template:

cpu/arm64_vfp_support_platform
Template resource:

cpuinfo
Template filter:

cpuinfo.platform == ‘aarch64’
cpu/armhf_vfp_support_platform

Validate that the Vector Floating Point Unit is running on {platform} device

Unit type:

template
Category ID:

cpu
Status:

Blocking
Description:

Validate that the Vector Floating Point Unit is running on {platform} device.
From template:

cpu/armhf_vfp_support_platform
Template resource:

cpuinfo
Template filter:

cpuinfo.platform == ‘armv7l’
cpu/clocktest

Tests the CPU for clock jitter

Category ID:

cpu
Status:

Blocking
Purpose:

Runs a test for clock jitter on SMP machines.
After-suspend:

True
Plugin:

shell
cpu/cstates

Run C-States tests

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Uses the Firmware Test Suite (fwts) to test the power saving states of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
cpu/cstates_results.log

Attach C-States test log

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission.
Plugin:

attachment
cpu/maxfreq_test

Test that the CPU can run at its max frequency

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.
Environment variable:

LD_LIBRARY_PATH, SNAP
User:

root
Plugin:

shell
cpu/maxfreq_test-log-attach

Attach CPU max frequency log

Category ID:

cpu
Status:

Blocking
Purpose:

Attaches the log generated by cpu/maxfreq_test to the results submission.
Plugin:

attachment
cpu/offlining_test

Test offlining of each CPU core

Category ID:

cpu
Status:

Blocking
Purpose:

Attempts to offline each core in a multicore system.
After-suspend:

True
User:

root
Plugin:

shell
cpu/scaling_test

Test the CPU scaling capabilities

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use Firmware Test Suite (fwts cpufreq) to test the scaling capabilities of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
cpu/scaling_test-log-attach

Attach CPU scaling capabilities log

Category ID:

cpu
Status:

Blocking
Purpose:

Attaches the log generated by cpu/scaling_test to the results submission.
Plugin:

attachment
cpu/topology

Check CPU topology for accuracy between proc and sysfs

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.
After-suspend:

True
Plugin:

shell

Disk tests

The following test units are covered in this category:

disk/check-software-raid

Validate the configuration of software RAID devices are expected

Category ID:

disk
Status:

Blocking
Purpose:

<missing purpose>
Description:

Examine the system to detect Software RAID devices are created and the RAID mode are expected the SOFTWARE_RAID_LEVEL variable is needed for this tests. e.g. SOFTWARE_RAID_LEVEL=”raid0 raid1 raid5”
Environment variable:

SOFTWARE_RAID_LEVEL
User:

root
Plugin:

shell
disk/detect

Gathers information about each disk detected

Category ID:

disk
Status:

Blocking
Purpose:

Uses lsblk to gather information about each disk detected on the system under test.
After-suspend:

True
Plugin:

shell
disk/read_performance_name

Disk performance test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
After-suspend:

True
Environment variable:

DISK_MDADM_READ_PERF, DISK_NVME_READ_PERF, DISK_READ_PERF
From template:

disk/read_performance_name
Template resource:

device
Template filter:

device.category == ‘DISK’
disk/stats_name

Disk statistics for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
After-suspend:

True
From template:

disk/stats_name
Template resource:

device
Template filter:

device.category == ‘DISK’ and device.name != ‘’
disk/storage_device_name

Disk I/O stress test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
After-suspend:

True
From template:

disk/storage_device_name
Template resource:

device
Template filter:

device.category == ‘DISK’
thunderbolt3/insert

Storage insert detection on Thunderbolt 3 port

Category ID:

disk
Status:

Blocking
Purpose:

This test will check if the insertion of a Thunderbolt 3 HDD could be detected
Steps:

  1. Click ‘Test’ to begin the test. This test will timeout and fail if the insertion has not been detected within 40 seconds. 2. Plug a Thunderbolt HDD into an available Thunderbolt 3 port, if it’s not mounted automatically, please click the HDD icon to mount it.
Verification:

The verification of this test is automated. Do not change the automatically selected result
After-suspend:

True
User:

root
Plugin:

user-interact
thunderbolt3/remove

Storage removal detection on Thunderbolt 3 port

Category ID:

disk
Status:

Blocking
Purpose:

This test will check if the system can detect the removal of a Thunderbolt 3 HDD
Steps:

  1. Click ‘Test’ to begin the test. This test will timeout and fail if the removal has not been detected within 20 seconds. 2. Remove the previously attached Thunderbolt HDD from the Thunderbolt port.
Verification:

The verification of this test is automated. Do not change the automatically selected result
After-suspend:

True
User:

root
Plugin:

user-interact
thunderbolt3/storage-test

Storage test on Thunderbolt 3

Category ID:

disk
Status:

Blocking
Purpose:

This is an automated test which performs read/write operations on an attached Thunderbolt HDD
After-suspend:

True
User:

root
Plugin:

shell

Docker containers

The following test units are covered in this category:

docker/build-single_arch

Test docker build with a single container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/build-single_arch
Template resource:

docker_resource
docker/commit_arch

Test docker commit a change to a single container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/commit_arch
Template resource:

docker_resource
docker/compose-and-basic_arch

Test docker compose and basic command

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/compose-and-basic_arch
Template resource:

docker_resource
docker/compose-restart_arch

Test compose a container with restart policy applied

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/compose-restart_arch
Template resource:

docker_resource
docker/compose-single_arch

Test docker compose with a single container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/compose-single_arch
Template resource:

docker_resource
docker/copy_arch

Test copy a file bwtween a container and local filesystem

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/copy_arch
Template resource:

docker_resource
docker/deploy-registry_arch

Deploy a registry server and run it on localhost

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/deploy-registry_arch
Template resource:

docker_resource
docker/diff_arch

Test changes to files in Ubuntu container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/diff_arch
Template resource:

docker_resource
docker/export-and-import_arch

Test docker import and export a docker container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/export-and-import_arch
Template resource:

docker_resource
docker/info

Display system-wide information about docker

Category ID:

docker
Status:

Blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
User:

root
docker/inspect_arch

Test query low-level information on a docker object

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/inspect_arch
Template resource:

docker_resource
docker/interative_arch

Test an interactive shell in Ubuntu container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/interative_arch
Template resource:

docker_resource
docker/kill_arch

Test killing containers

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/kill_arch
Template resource:

docker_resource
docker/restart-always_arch

Test container restart policy with always applied

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/restart-always_arch
Template resource:

docker_resource
docker/restart-on-failure_arch

Test container restart policy with on_failure applied

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/restart-on-failure_arch
Template resource:

docker_resource
docker/run_arch

Download and run ubuntu container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/run_arch
Template resource:

docker_resource
docker/save-and-load_arch

Test docker save and load a docker image

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/save-and-load_arch
Template resource:

docker_resource
docker/start-stop_arch

Start and stop a single container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/start-stop_arch
Template resource:

docker_resource
docker/update_arch

Test update configuration of one container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/update_arch
Template resource:

docker_resource
docker/version

Display docker version information

Category ID:

docker
Status:

Blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
User:

root

Ethernet Device tests

The following test units are covered in this category:

ethernet/detect

Detect if at least one ethernet device is detected

Category ID:

ethernet
Status:

Blocking
Purpose:

Test to detect and return information about available network controllers on the system under test.
After-suspend:

True
Plugin:

shell
ethernet/hotplug-interface

Ensure hotplugging works on port {{ interface }}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
After-suspend:

True
From template:

ethernet/hotplug-interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/ping_interface

Can ping another machine over Ethernet port {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
After-suspend:

True
From template:

ethernet/ping_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/wol_S3_interface

Wake on LAN (WOL) test from S3 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
From template:

ethernet/wol_S3_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’ and device.interface != ‘UNKNOWN’
ethernet/wol_S4_interface

Wake on LAN (WOL) test from S4 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
From template:

ethernet/wol_S4_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’
ethernet/wol_S5_interface

Wake on LAN (WOL) test from S5 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
From template:

ethernet/wol_S5_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’

Firmware tests

The following test units are covered in this category:

firmware/fwts_desktop_diagnosis

Run FWTS QA-concerned desktop-specific diagnosis tests.

Category ID:

firmware
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (fwts) QA-concerned desktop-specific diagnosis tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
firmware/fwts_desktop_diagnosis_results.log.gz

Attach FWTS desktop diagnosis log to submission

Category ID:

firmware
Status:

Blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission
Plugin:

attachment

Gathers information about the DUT

The following test units are covered in this category:

connections

Collect information about connections

Category ID:

information_gathering
Status:

Blocking
Purpose:

<missing purpose>
Description:

Generates a list of plug and slot connections on the device
Plugin:

resource
rtc

Creates resource info for RTC

Category ID:

information_gathering
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

resource
serial_assertion

Collect serial assertions on the device

Category ID:

information_gathering
Status:

Blocking
Purpose:

<missing purpose>
Description:

Queries the snapd REST API for serial assertions present on the device.
Plugin:

resource
sleep

Create resource info for supported sleep states

Category ID:

information_gathering
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

resource
snap

Collect information about installed snap packages

Category ID:

information_gathering
Status:

Blocking
Purpose:

<missing purpose>
Description:

Generates a list of snap packages
Plugin:

resource

General Purpose I/O

The following test units are covered in this category:

gpio/gpiomem_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via /dev/gpiomem

Unit type:

template
Category ID:

gpio
Status:

Blocking
After-suspend:

True
From template:

gpio/gpiomem_loopback_pairs_model
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)
gpio/sysfs_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template
Category ID:

gpio
Status:

Blocking
Description:

<missing description>
After-suspend:

True
From template:

gpio/sysfs_loopback_pairs_model
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)
gpio/sysfs_loopback_pairs_vendor_product

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template
Category ID:

gpio
Status:

Blocking
Description:

<missing description>
After-suspend:

True
From template:

gpio/sysfs_loopback_pairs_vendor_product
Template resource:

dmi
Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’, ‘UPN-EHL01’)

Informational tests

The following test units are covered in this category:

info/systemd-analyze

System boot-up performance statistics

Category ID:

info
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
info/systemd-analyze-critical-chain

Print the tree of the time-critical chain of SystemD

Category ID:

info
Status:

Blocking
Purpose:

This job prints a tree of the time-critical chain of SystemD units.
Plugin:

shell
lspci_attachment

Attach a list of PCI devices

Category ID:

info
Status:

Blocking
Purpose:

Attaches very verbose lspci output.
Plugin:

attachment
lsusb_attachment

Attach output of lsusb

Category ID:

info
Status:

Blocking
Purpose:

Attaches a list of detected USB devices.
After-suspend:

True
User:

root
Plugin:

attachment
net_if_management_attachment

Collect logging from the net_if_management job

Category ID:

info
Status:

Blocking
Purpose:

<missing purpose>
Description:

Allows logging of debug info and errors by the associated resource job
Plugin:

attachment
parts_meta_info_attachment

Attaches an information about all parts that constituted this snap

Category ID:

info
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

CHECKBOX_RUNTIME, SNAP
User:

root
Plugin:

attachment

Input Devices tests

The following test units are covered in this category:

input/zapper-keyboard

Check if the system recognizes keyboard events from Zapper

Category ID:

input
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

ZAPPER_HOST
User:

root
Plugin:

shell

I²C (Inter-Integrated Circuit)

The following test units are covered in this category:

i2c/i2c-bus-detect

Check presence of an I²C bus

Category ID:

i2c
Status:

Blocking
Purpose:

If an expected number of I²C buses is provided, the job will verify the detected number is correct. If the expected number of buses is not provided, the job will pass if at least one I²C bus is detected.
Environment variable:

I2C_BUS_NUMBER
User:

root
Plugin:

shell
i2c/i2c-device-detect

Check if any I²C devices can be detected

Category ID:

i2c
Status:

Blocking
Purpose:

The test will pass if there’s at least one I²C device detected on any I²C bus.
User:

root
Plugin:

shell

Kernel snap tests

The following test units are covered in this category:

kernel-snap/booted-kernel-matches-current-name

The booted kernel image matches the image in the current kernel snap

Unit type:

template
Category ID:

kernel-snap
Status:

Blocking
From template:

kernel-snap/booted-kernel-matches-current-name
Template resource:

bootloader
Template filter:

bootloader.booted_kernel_path != ‘unknown’

LED tests

The following test units are covered in this category:

led/fn

Test the Fn key LED functionality by activating/deactivating the Fn keys locking.

Category ID:

led
Status:

Blocking
Purpose:

This test will test the Fn key LED.
Steps:

  1. Press the Fn+Fn Lock key to activate/deactivate Fn keys locking. 2. The Fn key LED should be switched on/off every time the key is pressed.
Verification:

Did the Fn key LED light as expected?
After-suspend:

True
Plugin:

manual
led/power

Power LED behavior when powered

Category ID:

led
Status:

Blocking
Purpose:

Check power led is on when system is powered on
Steps:

  1. Check power led when system is powered on
Verification:

Power led is on when system is powered on
After-suspend:

True
Plugin:

manual
led/serial

Serial ports LED behavior

Category ID:

led
Status:

Blocking
Purpose:

Check serial ports LED behavior is correct
Steps:

  1. Start the test to send data to all serial ports (/dev/ttyS*)
Verification:

All serial ports LED are on for a few seconds (3-4s)
User:

root
Plugin:

user-interact-verify
led/sysfs_led_brightness_off_vendor_product

Ensure the leds_aaeon driver properly sets all LEDs to off or minimum brightness by running a test.

Unit type:

template
Category ID:

led
Status:

Blocking
After-suspend:

True
From template:

led/sysfs_led_brightness_off_vendor_product
Template resource:

dmi
Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’)
led/sysfs_led_brightness_on_vendor_product

Verify the functionality of the leds_aaeon driver by ensuring all external LEDs achieve maximum brightness.

Unit type:

template
Category ID:

led
Status:

Blocking
After-suspend:

True
From template:

led/sysfs_led_brightness_on_vendor_product
Template resource:

dmi
Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’)
led/wireless

Verify the WLAN/Bluetooth LED functionality by toggling wireless connections.

Category ID:

led
Status:

Blocking
Purpose:

Wireless (WLAN + Bluetooth) LED verification
Steps:

  1. Make sure WLAN connection is established and Bluetooth is enabled. 2. WLAN/Bluetooth LED should light 3. Switch WLAN and Bluetooth off from a hardware switch (if present) 4. Switch them back on 5. Switch WLAN and Bluetooth off from the panel applet 6. Switch them back on
Verification:

Did the WLAN/Bluetooth LED light as expected?
Plugin:

manual

Location Service

The following test units are covered in this category:

location/status

Queries the status of a service instance

Category ID:

location
Status:

Blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
After-suspend:

True

Media Card tests

The following test units are covered in this category:

mediacard/cf-insert

Verify the system’s media card reader can detect a Compact Flash card insertion.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Compact Flash (CF) media card
Steps:

  1. Commence the test and then insert a CF card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/cf-remove

Ensure the system detects CF card removal from the card reader correctly.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a CF card from the system’s card reader.
Steps:

  1. Commence the test and then remove the CF card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/cf-storage

Automate testing for reading and writing to the CF card after mediacard/cf-insert test.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/cf-insert test is run. It tests reading and writing to the CF card.
User:

root
Plugin:

shell
mediacard/mmc-insert

Test the system’s media card reader for MMC insertion detection.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Multimedia Card (MMC) media
Steps:

  1. Commence the test and then insert an MMC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/mmc-remove

Test the detection of MMC card removal from the system’s card reader.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of the MMC card from the system’s card reader.
Steps:

  1. Commence the test and then remove the MMC card from the reader. (Note: this test will time out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/mmc-storage

Test automated execution for reading and writing to the MMC card.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/mmc-insert test is run. It tests reading and writing to the MMC card.
User:

root
Plugin:

shell
mediacard/ms-insert

Verify the detection of Memory Stick (MS) card insertion by the system’s media card reader.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Memory Stick (MS) media card
Steps:

  1. Commence the test and then insert a MS card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/ms-remove

Test if the system detects the removal of an MS card correctly.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of an MS card from the system’s card reader.
Steps:

  1. Commence the test and then remove the MS card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/ms-storage

Automated test for reading and writing to the MS card after mediacard/ms-insert test.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/ms-insert test is run. It tests reading and writing to the MS card.
User:

root
Plugin:

shell
mediacard/msp-insert

Verify the system’s media card reader can detect a Memory Stick Pro (MSP) card insertion.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Memory Stick Pro (MSP) media card
Steps:

  1. Commence the test and then insert a MSP card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/msp-remove

Ensure MSP card removal is correctly detected by the system.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a MSP card from the system’s card reader.
Steps:

  1. Commence the test and remove the MSP card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/msp-storage

Automated test for reading and writing to the MSP card after mediacard/msp-insert test.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/msp-insert test is run. It tests reading and writing to the MSP card.
User:

root
Plugin:

shell
mediacard/sd-insert

Test that insertion of an SD card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of an UNLOCKED Secure Digital (SD) media card
Steps:

  1. Commence the test and then insert an UNLOCKED SD card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/sd-remove

Test that removal of an SD card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of an SD card from the system’s card reader.
Steps:

  1. Commence the test and then remove the SD card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/sd-storage

Test reading & writing to an SD Card

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/sd-insert test is run. It tests reading and writing to the SD card.
User:

root
Plugin:

shell
mediacard/sdhc-insert

Test that insertion of an SDHC card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a UNLOCKED Secure Digital High-Capacity (SDHC) media card
Steps:

  1. Commence the test and then insert an UNLOCKED SDHC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
mediacard/sdhc-remove

Test that removal of an SDHC card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of an SDHC card from the system’s card reader.
Steps:

  1. Commence the test and then remove the SDHC card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
mediacard/sdhc-storage

Test reading & writing to a SDHC Card

Category ID:

mediacard
Status:

Blocking
Purpose:

<missing purpose>
Description:

This test is automated and executes after the mediacard/sdhc-insert test is run. It tests reading and writing to the SDHC card.
After-suspend:

True
User:

root
Plugin:

shell
mediacard/sdxc-insert

Test that insertion of an SDXC card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Secure Digital Extended Capacity (SDXC) media card
Steps:

  1. Commence the test and then insert an UNLOCKED SDXC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/sdxc-remove

Test that removal of an SDXC card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a SDXC card from the system’s card reader.
Steps:

  1. Commence the test and then remove the SDXC card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/sdxc-storage

Test reading & writing to an SDXC Card

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/sdxc-insert test is run. It tests reading and writing to the SDXC card.
User:

root
Plugin:

shell
mediacard/xd-insert

Ensure the system’s media card reader detects the insertion of an xD media card.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of an Extreme Digital (xD) media card.
Steps:

  1. Commence the test and then insert an xD card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/xd-remove

Automated test for detecting the removal of a xD card from the system’s card reader.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a xD card from the system’s card reader.
Steps:

  1. Commence the test and then remove the xD card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/xd-storage

Automated test to verify reading and writing functionality of the xD card after mediacard/xd-insert test.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/xd-insert test is run. It tests reading and writing to the xD card.
User:

root
Plugin:

shell

Memory tests

The following test units are covered in this category:

memory/info

Check the amount of memory reported by meminfo against DMI

Category ID:

memory
Status:

Blocking
Purpose:

This test checks the amount of memory which is reported in meminfo against the size of the memory modules detected by DMI.
User:

root
Plugin:

shell

Miscellaneous tests

The following test units are covered in this category:

miscellanea/device_check

Device Check

Category ID:

miscellanea
Status:

Blocking
Purpose:

Device check
Steps:

  1. Commence the test 2. Compare items on System Manifest to the devices known to udev
Verification:

Do the devices reported by udev match the devices on the Manifest?
Plugin:

user-interact-verify
miscellanea/submission-resources

Check that data for a complete result are present

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

A meta-job that verifies the data necessary for a complete result submission are present. Failure indicates that the results are incomplete and may be rejected.
Plugin:

shell

Monitor tests

The following test units are covered in this category:

monitor/displayport_hotplug

Can hotplug monitor (DisplayPort)

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the DisplayPort port and the ability to do hotplugging.
Steps:

Skip this test if your system does not have a DisplayPort port. 1. If a display is already connected, unplug it. 2. (Re-)Connect a display to the DisplayPort port on your system
Verification:

Was the interface displayed correctly on the screen?
After-suspend:

True
Plugin:

manual
monitor/dvi

Monitor works (DVI)

Category ID:

monitor
Status:

Blocking
Purpose:

Check output to display through DVI port
Steps:

  1. Connect display to DVI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
monitor/dvi-to-vga

Monitor works (DVI-to-VGA)

Category ID:

monitor
Status:

Blocking
Purpose:

Check output to display through VGA adaptor on DVI port
Steps:

  1. Connect display to VGA adaptor on DVI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
monitor/hdmi

Monitor works (HDMI)

Category ID:

monitor
Status:

Blocking
Purpose:

Check output to display through HDMI port
Steps:

  1. Connect display to HDMI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
monitor/hdmi-to-vga

Monitor works (HDMI-to-VGA)

Category ID:

monitor
Status:

Blocking
Purpose:

Check output to display through VGA adaptor on HDMI port
Steps:

  1. Connect display to VGA adaptor on HDMI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
monitor/vga

Monitor works (VGA)

Category ID:

monitor
Status:

Blocking
Purpose:

Check output to display through VGA port
Steps:

  1. Connect display to VGA port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
monitor/zapper-edid

Check if the system automatically changes the resolution based on EDID

Category ID:

monitor
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

ZAPPER_HOST
Plugin:

shell

Non-device specific networking tests

The following test units are covered in this category:

ipv6_detect

Test if the kernel is IPv6 ready

Category ID:

networking
Status:

Blocking
Purpose:

Test if the running kernel supports IPv6.
Steps:

<missing steps>
Verification:

<missing verification>
After-suspend:

True
networking/info_device__index___interface

Network Information of device {__index__} ({interface})

Unit type:

template
Category ID:

networking
Status:

Blocking
After-suspend:

True
From template:

networking/info_device__index___interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’
networking/predictable_names

Verify that all network interfaces have predictable names.

Category ID:

networking
Status:

Blocking
Purpose:

Verify that all network interfaces have predictable names.
Plugin:

shell

Power Management tests

The following test units are covered in this category:

power-management/cold-reboot

Cold reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This test powers off the system and then powers it on using RTC
Environment variable:

COLD_REBOOT_DELAY, RTC_DEVICE_FILE
User:

root
Plugin:

shell
power-management/post-cold-reboot

Post cold reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the cold reboot
Plugin:

shell
power-management/post-warm-reboot

Post warm reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the warm reboot
Plugin:

shell
power-management/warm-reboot

Warm reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This tests reboots the system using the reboot command
User:

root
Plugin:

shell
watchdog/detect

Detect the presence of a Hardware Watchdog

Category ID:

power-management
Status:

Blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
watchdog/post-trigger-system-reset-auto

Post watchdog reset service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the watchdog triggered
Plugin:

shell
watchdog/systemd-config

Check if the hardware watchdog is properly configured

Category ID:

power-management
Status:

Blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
watchdog/trigger-system-reset-auto

Test that the watchdog module can trigger a system reset

Category ID:

power-management
Status:

Blocking
Purpose:

Ensure that the watchdog module can successfully initiate a system reset.
User:

root
Plugin:

shell

Real Time Clock (RTC)

The following test units are covered in this category:

rtc/battery

RTC battery tracks the time and ensures the system can wake up from power off state.

Category ID:

rtc_category
Status:

Blocking
Purpose:

RTC battery backup power can send system wakeup events.
Steps:

  1. Start the test to power off the system (wakeup scheduled in 30s).
Verification:

RTC can wake up the system successfully. environ: RTC_DEVICE_FILE
User:

root
Plugin:

user-interact

Serial Port

The following test units are covered in this category:

serial/loopback-dev

Serial loopback test of {dev}

Unit type:

template
Category ID:

serial
Status:

Blocking
After-suspend:

True
From template:

serial/loopback-dev
Template resource:

serial_ports_static
serial/rs232-console

Serial debugging console is enabled and operational

Category ID:

serial
Status:

Blocking
Purpose:

Check user can log into system through serial port from another machine
Steps:

  1. Connect USB to DB9 null modem adapter cable to the serial port of the test machine. 2. Connect the cable to a USB port of another Ubuntu machine (client). 3. Install screen on the client (if not done in Prerequisite). 4. Execute the following command on the client: sudo screen /dev/ttyUSB0 5. Start the getty service on the test machine: sudo systemctl start getty@[rs232 device, e.g., /dev/ttyS0].service 6. Log into the test machine from the terminal on the client.
Verification:

  1. The output to the client is fine after the getty service is started. 2. Log into the test machine from the terminal on the client successfully.
After-suspend:

True
Plugin:

manual

Snapd

The following test units are covered in this category:

snappy/os-refresh

Refresh the system using the snap tool

Category ID:

snapd
Status:

Blocking
Purpose:

Check “core” can be refreshed by snap refresh
Steps:

  1. Check version number snap list core 2. Update snap refresh core –edge 3. Reboot the system and log in sudo reboot 4. Check version number snap list core
Verification:

Check core version is updated using the edge channel
Plugin:

manual
snappy/os-revert

Rollback system update using the snap tool

Category ID:

snapd
Status:

Blocking
Purpose:

Check core can be reverted by snap revert
Steps:

  1. Check version number (Note the version number) snap list core 2. Revert snap revert core 3. Reboot the system and log in sudo reboot 4. Check version number snap list core 5. Remove reverted version (and associated data) to make system clean snap remove core –revision=<edge_revision>
Verification:

Check core version at step 4 is back to its stable version
Plugin:

manual
snappy/snap-install

Test the snap install command is working

Category ID:

snapd
Status:

Blocking
Purpose:

The store should contain the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap. The test makes sure this can be downloaded and installed on the system.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-list

Test that the snap list command is working.

Category ID:

snapd
Status:

Blocking
Purpose:

If snap list command is working then should at least find the ubuntu-core package.
Plugin:

shell
snappy/snap-refresh-automated

Test whether the snap refresh command is working.

Category ID:

snapd
Status:

Blocking
Purpose:

The test will install the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap from the stable channel and then refreshes it to the edge channel and compares the revision before and after the refresh.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-remove

Test the snap remove command is working.

Category ID:

snapd
Status:

Blocking
Purpose:

After having installed the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap, check it can be removed.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-reupdate-automated

Test the snap refresh command works after blacklisting.

Category ID:

snapd
Status:

Blocking
Purpose:

Checks that the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap can be refreshed after removal of the blacklisted revision.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-revert-automated

Test the snap revert command is working.

Category ID:

snapd
Status:

Blocking
Purpose:

Checks if the edge channel {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap is reverted back to the one from stable.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-snaps-confinement

Test all the snaps’ confinement

Category ID:

snapd
Status:

Blocking
Purpose:

Test all the snaps’ confinement, devmode, revision. Make sure the confinement is “strict”, devmode is “False”, and revision should not be sideloaded.
Plugin:

shell
snappy/test-store-config-store

Test that image is using the correct snappy store configuration.

Unit type:

template
Category ID:

snapd
Status:

Blocking
Environment variable:

STORE_ID
From template:

snappy/test-store-config-store
Template resource:

com.canonical.certification::model_assertion
snappy/test-store-install-beta

Snappy install command - beta channel store

Category ID:

snapd
Status:

Blocking
Purpose:

Test the snappy install command is able to install and remove snap in beta channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-store-install-edge

Snappy install command - edge channel store

Category ID:

snapd
Status:

Blocking
Purpose:

Test the snappy install command is able to install a snap in the edge channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-system-confinement

Test if the system confinement is strict

Category ID:

snapd
Status:

Blocking
Purpose:

Test if the system confinement is “strict” If not, list the missing features
Plugin:

shell

SocketCAN interface tests

The following test units are covered in this category:

socketcan/send_packet_local_eff_virtual

Virtual CAN device support test (Local test with raw socket and EFF)

Category ID:

socketcan
Status:

Blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_eff_interface

CAN device support test for {interface} (Raw, Local, EFF)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_local_eff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_local_fd_virtual

Virtual CAN device support test (Raw, Local, FD)

Category ID:

socketcan
Status:

Blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket; this is only a local test as the broadcast packet is received on the same device.
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_fd_interface

CAN device support test for {interface} (Raw, Local, FD)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_local_fd_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_local_sff_virtual

Virtual CAN device support test (Raw, Local)

Category ID:

socketcan
Status:

Blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_sff_interface

CAN device support test for {interface} (Raw, Local)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_local_sff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_eff_interface

CAN device support test {interface} (Raw, Remote, EFF)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_remote_eff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_fd_interface

CAN device support test {interface} (Raw, Remote, FD)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_remote_fd_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_sff_interface

CAN device support test {interface} (Raw, Remote)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_remote_sff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’

TPM 2.0 (Trusted Platform Module)

The following test units are covered in this category:

clevis-encrypt-tpm2/detect-ecc-capabilities

Ensure the TPM has required capabilities for clevis ECC test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/detect-rsa-capabilities

Ensure the TPM has required capabilities for clevis RSA test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/ecc

clevis encrypt/decrypt key ecc

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/precheck

clevis encrypt/decrypt precheck

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
clevis-encrypt-tpm2/rsa

clevis encrypt/decrypt key rsa

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
tpm2/fwts-event-log-dump

Dump the contents of the TPM Event Log

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

The information in the TPM Event Log can be useful in debugging problems with TPM command support and adherance to standards. This can be of particular help when checking whether a device can support Ubuntu Core Full Disk Encryption.
User:

root
Plugin:

attachment

Ubuntu Core OS feature tests

The following test units are covered in this category:

ubuntucore/os-fail-boot-description

Automatically rollback after failed boot after upgrade

Unit type:

template
Category ID:

ubuntucore
Status:

Blocking
From template:

ubuntucore/os-fail-boot-description
Template resource:

lsb
Template filter:

lsb.distributor_id == ‘Ubuntu Core’
ubuntucore/os-recovery-mode

Reboot into recovery mode and log into the system using prior credentials.

Category ID:

ubuntucore
Status:

Blocking
Purpose:

Check if the system will reboot to recovery mode successfully
Steps:

  1. Reboot the system to recovery mode $ sudo snap reboot –recover 2. The system should respond to the command and reboot itself 3. Wait until the system completes the reboot process 4. Check if the system is running in recovery mode $ cat /proc/cmdline 5. Reboot the system back to normal run mode $ sudo snap reboot 6. Check again if the system is running in run mode $ cat /proc/cmdline
Verification:

Check if kernel cmdline when the system is in recovery mode includes: ‘snapd_recovery_mode=recover’ Check if kernel cmdline when the system is in normal run mode includes: ‘snapd_recovery_mode=run’
Plugin:

manual
ubuntucore/os-reinstall-mode

Reboot into reinstall mode and trigger a factory reset on the device.

Category ID:

ubuntucore
Status:

Blocking
Purpose:

Check if the system will reboot to reinstall mode and reinitialize the device with a fresh factory reset.
Steps:

WARNING: ALL EXISTING DATA ON THIS DEVICE WILL BE WIPED!! 1. Check the current serial-assertion device-key $ ls /var/lib/snapd/save/device/private-keys-v1 2. Clear TPM first if this device has enabled secure boot & FDE For x86-based platforms: $ sudo su $ echo 5 > /sys/class/tpm/tpm0/ppi/request For ARM-based platforms: There is no generic command for ARM-based platforms, please refer to the device user manual. 3. Reboot the system to reinstall mode $ sudo snap reboot –install 4. The system should respond to the command and reboot itself. 5. Wait until the system completes the installation and initialization process. 6. Check the serial-assertion device-key after installation completes. $ ls /var/lib/snapd/save/device/private-keys-v1
Verification:

Check if a new serial-assertion device-key got generated after reinstallation completes.
Plugin:

manual
ubuntucore/sshd

SSH is enabled and operational

Category ID:

ubuntucore
Status:

Blocking
Purpose:

Check if a user can access the system through SSH from another machine
Steps:

  1. Execute the following command on another machine in the same network: $ ssh [user id]@[ip address of the testing system] 2. Enter the password to log in
Verification:

Can log into the system through SSH from another machine
Plugin:

manual

USB tests

The following test units are covered in this category:

usb-c-otg/g_ether

Check DUT can be detected as USB ethernet device

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Steps:

  1. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT. 2. Press Enter to configure the IP address for the USB OTG ethernet interface on the DUT. (IP will be 192.168.9.1/24) 3. On the host, configure the IP address for the USB OTG ethernet interface. It should be in the same subnet as the DUT. (e.g., 192.168.9.2/24) 4. Try to ping between the host and DUT.
Verification:

The host and DUT can ping each other.
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

user-interact-verify
usb-c-otg/g_ether-cleanup

Cleanup USB OTG ethernet interface setup after ethernet device test

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
usb-c-otg/g_mass_storage

Check DUT can be detected as a mass storage device

Category ID:

usb
Status:

Blocking
Purpose:

Check that after connecting the device under test (DUT) to another device (host), the DUT can be detected as a mass storage device by the host.
Steps:

  1. Press Enter to setup a 16 MB FAT32 image on the device. 2. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT.
Verification:

The host detects and mounts the mass storage device. It has read and write permissions on it.
After-suspend:

True
User:

root
Plugin:

user-interact-verify
usb-c-otg/g_mass_storage-cleanup

Cleanup mass storage setup after mass storage device test

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
usb-c-otg/g_serial

Check if USB OTG can work as a serial port.

Category ID:

usb
Status:

Blocking
Purpose:

Check if USB OTG can work as a serial port.
Steps:

  1. Press enter to probe USB OTG g_serial 2. Connect a USB cable from an Ubuntu host to the USB OTG (it’s the USB Type C connector) port on DUT. 3. On the host, check if ttyACM0 has been probed $ ls /dev/ttyACM0 4. On the host, listen to the ttyACM0 port as the receiving side. $ sudo su $ cat /dev/ttyACM0 5. On DUT, send a string via /dev/ttyGS0 as the sending side. $ sudo su $ echo 123 > /dev/ttyGS0 6. Check if the string “123” was received on the host side. 7. Check the string sending from the host to DUT by repeating steps 4-6, but swap the send and receive sides.
Verification:

Does string send and receive function correctly?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
usb-c-otg/g_serial-cleanup

Cleanup USB OTG serial interface setup after serial device test

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
usb-c/c-to-a-adapter/hid

USB HID work on USB Type-C port using a “USB Type-C to Type-A” adapter

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port using a “USB Type-C to Type-A” adapter
Steps:

  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port using a “USB Type-C to Type-A” adapter 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text
Verification:

Did the device work as expected?
After-suspend:

True
Plugin:

manual
usb-c/c-to-a-adapter/insert

Verify that the system detects a USB 3 storage device using a “Type-C to Type-A” adapter.

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3 storage device in a USB Type-C connector using a “Type-C to Type-A” adapter. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test 2. Connect a USB 3 storage device to a USB Type-C port using a “Type-C to Type-A” adapter. 3. Do not unplug the device after the test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/c-to-a-adapter/remove

Check the detection of a USB 3 storage device removal, connected via a “Type-C to Type-A” adapter.

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a USB 3 storage device connected to a USB Type-C port using a “USB Type-C to Type-A” adapter.
Steps:

  1. Begin the test 2. Disconnect a USB 3 storage device from a USB Type-C port using a “Type-C to Type-A” adapter
Verification:

The verification of this test is automated. Do not modify the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/c-to-a-adapter/storage-automated

Automate the execution process post usb-c/c-to-a-adapter/insert test.

Category ID:

usb
Status:

Blocking
Purpose:

This test is automated and executes after the usb-c/c-to-a-adapter/insert test is run.
After-suspend:

True
User:

root
Plugin:

shell
usb-c/hid

USB HID work on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port
Steps:

  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text
Verification:

Did the device work as expected?
After-suspend:

True
Plugin:

manual
usb-c/insert

USB 3.0 storage device insertion detected on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3.0 storage device in a USB Type-C connector. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test. 2. Connect a USB 3.0 storage device to a USB Type-C port. 3. Do not unplug the device after the test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/remove

USB 3.0 storage device removal detected on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a USB 3.0 storage device connected to a USB Type-C port.
Steps:

  1. Commence the test 2. Connect a USB 3.0 storage device to a USB Type-C port 3. Disconnect the USB 3.0 storage device from the USB Type-C port
Verification:

The system should automatically detect the removal of the USB 3.0 storage device. The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/storage-automated

USB 3.0 storage device read & write works on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test is automated and executes after the usb-c/insert test is run.
After-suspend:

True
User:

root
Plugin:

shell
usb/hid

USB keyboard works

Category ID:

usb
Status:

Blocking
Purpose:

Check USB input device works
Steps:

  1. Connect USB keyboard 2. Input something with USB keyboard
Verification:

What was just input is displayed correctly
After-suspend:

True
Plugin:

manual
usb/insert

USB 2.0 storage device insertion detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect USB 2.0 storage when inserted. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Press continue 2. Wait until the message “INSERT NOW” is printed on the screen 3. Connect USB 2.0 storage device
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb/remove

USB 2.0 storage device removal detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect removal of a USB 2.0 storage device
Steps:

  1. Press continue 2. Wait until the message “REMOVE NOW” is printed on the screen 3. Disconnect USB 2.0 storage device
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb/storage-automated

USB 2.0 storage device read & write works

Category ID:

usb
Status:

Blocking
Purpose:

Check system can read/write to USB 2.0 storage correctly
After-suspend:

True
User:

root
Plugin:

shell
usb/storage-detect

Detect storage partitions on a device on the USB bus

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
usb/zapper-usb-insert-port_alias

Zapper-driven USB insertion test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb/zapper-usb-insert-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’
usb/zapper-usb-remove-port_alias

Zapper-driven USB removal test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb/zapper-usb-remove-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’
usb3/insert

USB 3.0 storage device insertion detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect insertion of a USB 3.0 storage device. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Press continue. 2. Wait until the message “INSERT NOW” is displayed on the screen. 3. Connect USB 3.0 storage device.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb3/remove

USB 3.0 storage device removal detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect removal of a USB 3.0 storage device
Steps:

  1. Press continue 2. Wait until the message “REMOVE NOW” is printed on the screen 3. Disconnect USB 3.0 storage device
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb3/storage-automated

USB 3.0 storage device read & write works

Category ID:

usb
Status:

Blocking
Purpose:

Check system can read/write to USB 3.0 storage devices correctly
After-suspend:

True
User:

root
Plugin:

shell
usb3/zapper-usb-insert-port_alias

Zapper-driven USB 3.0 insertion test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb3/zapper-usb-insert-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’
usb3/zapper-usb-remove-port_alias

Zapper-driven USB 3.0 removal test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb3/zapper-usb-remove-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’

Wi-Fi access point

The following test units are covered in this category:

wireless/nmcli_wifi_ap_a_interface

Create 802.11a Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
Description:

<missing description>
After-suspend:

True
From template:

wireless/nmcli_wifi_ap_a_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/nmcli_wifi_ap_bg_interface

Create 802.11b/g Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
From template:

wireless/nmcli_wifi_ap_bg_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_b_no_sta_interface_auto

Create open 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
Environment variable:

WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_open_b_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_b_no_sta_interface_manual

Create open 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
From template:

wireless/wifi_ap_open_b_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_g_no_sta_interface_auto

Create an open 802.11g Wi-Fi AP on {interface} with no STA connected.

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
Environment variable:

WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_open_g_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_g_no_sta_interface_manual

Create open 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
From template:

wireless/wifi_ap_open_g_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_setup_wizard_interface_auto

Create Access Point on {interface} using wifi-ap.setup-wizard

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
Environment variable:

WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_setup_wizard_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_no_sta_interface_auto

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
Environment variable:

WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_wpa_b_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_no_sta_interface_manual

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
From template:

wireless/wifi_ap_wpa_b_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_with_sta_interface_auto

Create a WPA2 802.11b Wi-Fi Access Point on {interface} with active STA

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
Description:

<missing description>
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, OPEN_BG_SSID, WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_wpa_b_with_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_no_sta_interface_auto

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
Environment variable:

WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_wpa_g_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_no_sta_interface_manual

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
From template:

wireless/wifi_ap_wpa_g_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_with_sta_interface_auto

Create WPA2 802.11g Wi-Fi Access Point on {interface} with active STA

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
Description:

<missing description>
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, OPEN_BG_SSID, WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_wpa_g_with_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless networking tests

The following test units are covered in this category:

wireless/check_iwlwifi_microcode_crash_interface

Check there have been no iwlwifi crashes

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/check_iwlwifi_microcode_crash_interface
Template resource:

device
Template filter:

device.driver == ‘iwlwifi’
wireless/detect

Detect if at least one Wireless LAN device is detected

Category ID:

wireless
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
wireless/wireless_connection_open_ac_nm_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ac_np_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_AC_SSID
From template:

wireless/wireless_connection_open_ac_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_nm_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_np_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_AX_SSID
From template:

wireless/wireless_connection_open_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_nm_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_np_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_BE_SSID
From template:

wireless/wireless_connection_open_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_nm_interface

Connect to an unencrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_np_interface

Connect to unencrypted 802.11b/g Wi-Fi network on {{ interface }} using netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_BG_SSID
From template:

wireless/wireless_connection_open_bg_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_nm_interface

Connect to an unencrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_np_interface

Connect to unencrypted 802.11n Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_N_SSID
From template:

wireless/wireless_connection_open_n_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_nm_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_np_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA3_AX_PSK, WPA3_AX_SSID
From template:

wireless/wireless_connection_wpa3_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_nm_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_np_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA3_BE_PSK, WPA3_BE_SSID
From template:

wireless/wireless_connection_wpa3_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_nm_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_np_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_AC_PSK, WPA_AC_SSID
From template:

wireless/wireless_connection_wpa_ac_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_nm_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_np_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_AX_PSK, WPA_AX_SSID
From template:

wireless/wireless_connection_wpa_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_nm_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_np_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_BE_PSK, WPA_BE_SSID
From template:

wireless/wireless_connection_wpa_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_nm_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_np_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_BG_PSK, WPA_BG_SSID
From template:

wireless/wireless_connection_wpa_bg_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_nm_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_np_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }} using netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_N_PSK, WPA_N_SSID
From template:

wireless/wireless_connection_wpa_n_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_scanning_interface

Test system can discover Wi-Fi networks on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_scanning_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless Wide Area Network

The following test units are covered in this category:

wwan/3gpp-scan-manufacturer-model-hw_id-auto

Scan for available 3GPP networks with the modem

Unit type:

template
Category ID:

wwan
Status:

Blocking
Description:

Scan for available 3GPP networks with the target modem
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH
From template:

wwan/3gpp-scan-manufacturer-model-hw_id-auto
Template resource:

wwan_resource
wwan/check-sim-present-manual

Check if a SIM card is present in a slot connected to the modem

Category ID:

wwan
Status:

Blocking
Purpose:

Check if a SIM card is present in a slot connected to the modem
Steps:

  1. Open another terminal on SUT (or press Ctrl+Z to suspend Checkbox) 2. Run sudo mmcli -m 0 | grep SIM 3. Run fg to jump back to Checkbox (if you’re running in the same terminal)
Verification:

Check the output. Was the SIM information printed?
After-suspend:

True
Plugin:

manual
wwan/check-sim-present-manufacturer-model-hw_id-auto

Check if a SIM card is present in a slot connected to the modem

Unit type:

template
Category ID:

wwan
Status:

Blocking
Description:

Check if a SIM card is present in a slot connected to the modem
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH
From template:

wwan/check-sim-present-manufacturer-model-hw_id-auto
Template resource:

wwan_resource
wwan/detect

Identify if WWAN module is missing

Category ID:

wwan
Status:

Blocking
Purpose:

Tests that there is a WWAN module present and indicates that testing of it should follow.
After-suspend:

True
User:

root
Plugin:

shell
wwan/detect-manual

Check if WWAN module is available

Category ID:

wwan
Status:

Blocking
Purpose:

Check if WWAN module is available and ready to be used.
Steps:

  1. Open another terminal on SUT (or press ctrl+z to suspend Checkbox) 2. Run sudo mmcli -L command. 3. Run fg to jump back to checkbox (if you’re running in the same terminal)
Verification:

Check the output. Was the modem listed?
After-suspend:

True
Plugin:

manual
wwan/gsm-connection-interrupted-manual

Verify a GSM broadband connection can be reconnected after the signal is lost

Category ID:

wwan
Status:

Blocking
Purpose:

Ensure that the data connection can be revived after losing signal
Steps:

  1. Open another terminal on SUT (or press ctrl+z to suspend Checkbox). 2. Run sudo nmcli c add type gsm ifname {{ __checkbox_env__.get(“WWAN_CONTROL_IF”, “<device name>”) }} con-name GSM apn {{ __checkbox_env__.get(“WWAN_APN”, “<your.carrier.com>”) }} {%- if “WWAN_CONTROL_IF” not in __checkbox_env__ %} Replacing <your.carrier.com> with the appropriate APN name, and <device name> with the appropriate device (device can be found by running sudo nmcli d). {%- endif %} {%- if “WWAN_NET_IF” not in __checkbox_env__ %} 3. Ping Google (Run: ping -I <ifname> 8.8.8.8 -c 5). {%- else %} 3. Ping Google (Run: ping -I {{ __checkbox_env__.get(“WWAN_NET_IF”) }} 8.8.8.8 -c 5). {%- endif %} 4. Place the system in a Faraday bag. 5. Ping Google (and verify that the connection no longer works). 6. Remove the Faraday bag. {%- if “WWAN_NET_IF” not in __checkbox_env__ %} 7. Ping Google (Run: ping -I <ifname> 8.8.8.8 -c 5). {%- else %} 7. Ping Google (Run: ping -I {{ __checkbox_env__.get(“WWAN_NET_IF”) }} 8.8.8.8 -c 5). {%- endif %} 8. Disconnect from the network (Run: sudo nmcli r wwan off). 9. Run fg to jump back to Checkbox (if you’re running in the same terminal).
Verification:

Was the connection revived after removing the Faraday bag?
After-suspend:

True
Plugin:

manual
wwan/gsm-connection-manufacturer-model-hw_id-auto

Verify a GSM broadband modem can create a data connection

Unit type:

template
Category ID:

wwan
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, WWAN_APN, WWAN_CONTROL_IF, WWAN_NET_IF, WWAN_SETUPTIME
From template:

wwan/gsm-connection-manufacturer-model-hw_id-auto
Template resource:

wwan_resource

Non-blocking

Informational tests

The following test units are covered in this category:

manifest

Hardware Manifest

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

This job loads the hardware manifest and exposes it as a resource.
Plugin:

resource

client-cert-iot-server-22-04

Note

The certification tests presented in this document are validated by Checkbox version 4.2.0.dev76.

Blocking

Advanced Configuration and Power Interface

The following test units are covered in this category:

acpi/oem_osi

Test ACPI OEM _OSI strings

Category ID:

acpi
Status:

Blocking
Purpose:

This checks if the deprecated OEM _OSI strings are still used by checking the ACPI DSDT and SSDT tables
User:

root
Plugin:

shell

Audio tests

Output needs to be undistorted between 0%-100%. Output lines tested:

  • Internal speakers

  • 3.5mm headphones

  • HDMI audio output

  • DisplayPort audio output

Input needs to be recorded undistorted between 0%-100%. Input lines tested:

  • Internal microphone

  • 3.5mm microphone

Plug detection: when a new audio line input or output is plugged in the system, it needs to be recognized.

The following test units are covered in this category:

audio/alsa-loopback-automated

Captured sound matches played one (automated)

Category ID:

audio
Status:

Blocking
Purpose:

Check if sound that is ‘hearable’ by capture device. This test is intended mostly for devices without internal speakers/microphones, but with a line-in and a line-out ports to which a loopback cable can be connected.
After-suspend:

True
Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH, LD_LIBRARY_PATH
User:

root
Plugin:

shell
audio/alsa-playback

Playback works

Category ID:

audio
Status:

Blocking
Purpose:

Check if sound is played through ALSA on the default output
Steps:

  1. Make sure speakers or headphones are connected to the device 2. Commence the test
Verification:

Did you hear the sound?
After-suspend:

True
Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH
User:

root
Plugin:

user-interact-verify
audio/detect-capture-devices

Check that at least one audio capture device exists

Category ID:

audio
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
audio/detect-playback-devices

Check that at least one audio playback device exists

Category ID:

audio
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell

Bluetooth - BlueZ Self Tests

The following test units are covered in this category:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test

BlueZ-{bluez-internal-bnep-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Blocking
After-suspend:

True
From template:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test
Template resource:

bluez-internal-bnep-tests
bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test

BlueZ-{bluez-internal-hci-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Blocking
Description:

Runs a specific test from the hci test suite
After-suspend:

True
From template:

bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test
Template resource:

bluez-internal-hci-tests
bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test

BlueZ-{bluez-internal-rfcomm-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Blocking
After-suspend:

True
From template:

bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test
Template resource:

bluez-internal-rfcomm-tests
bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test

BlueZ-{bluez-internal-uc-test}

Unit type:

template
Category ID:

bluetooth_bluez5_selftests
Status:

Blocking
After-suspend:

True
From template:

bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test
Template resource:

bluez-internal-uc-tests

Bluetooth tests

Bluetooth LE (Smart and Smart Ready) is tested for device scanning and pairing. Apart from pairing, several profiles are specifically tested and required:

  • Eddystone Beacon

  • HID Over GATT Profile (HOGP), Low-Energy keyboard or mouse with basic functionality

The following test units are covered in this category:

bluetooth/bluetooth_obex_send

Bluetooth OBEX send

Category ID:

bluetooth
Status:

Blocking
Purpose:

This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable
After-suspend:

True
Environment variable:

BTDEVADDR, PLAINBOX_PROVIDER_DATA
Plugin:

shell
bluetooth/bluez-controller-detect

Check bluez lists a controller if rfkill detects one

Category ID:

bluetooth
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
bluetooth/detect

Make sure at least one bluetooth device is detected

Category ID:

bluetooth
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
bluetooth/keyboard-manual

Bluetooth keyboard manual test

Category ID:

bluetooth
Status:

Blocking
Purpose:

Check Bluetooth input device works
Steps:

  1. Run the following command to start bluetoothctl console: sudo bluetoothctl ***Following steps are run in bluetoothctl console*** 2. Power on the device: power on 3. Register agent for keyboard: agent KeyboardOnly default-agent 4. Put controller in pairable mode: pairable on 5. Scan nearby Bluetooth device: scan on 6. Stop Scanning after Bluetooth keyboard is found: scan off 7. Pair Bluetooth pair [MAC address of Bluetooth keyboard] 8. Enter PIN Code on Bluetooth keyboard if needed 9. Trust the Bluetooth keyboard trust [MAC address of Bluetooth keyboard] 10. Connect to Bluetooth keyboard: connect [MAC address of Bluetooth keyboard] 11. Quit bluetoothctl console quit 12. Use Bluetooth keyboard to input **For headless, please check the input outside the Bluetooth console by using: $ sudo cat /dev/tty1
Verification:

Confirm characters from Bluetooth input device are displayed correctly
After-suspend:

True
Plugin:

manual
bluetooth/zapper-a2dp

Check if the system can connect to a Bluetooth speaker using A2DP profile

Category ID:

bluetooth
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

ZAPPER_HOST
Plugin:

shell
bluetooth4/beacon_eddystone_url_interface

Test system can get beacon EddyStone URL advertisements on the {interface} adapter

Unit type:

template
Category ID:

bluetooth
Status:

Blocking
Description:

<missing description>
After-suspend:

True
From template:

bluetooth4/beacon_eddystone_url_interface
Template resource:

device
Template filter:

device.category == ‘BLUETOOTH’

Camera tests

The following test units are covered in this category:

camera/multiple-resolution-images-rpi-attachment_name

Attach an image from the multiple resolution images test on rpi

Unit type:

template
Category ID:

camera
Status:

Blocking
Description:

This test will attach one of the images used for the multiple resolution images test.
From template:

camera/multiple-resolution-images-rpi-attachment_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images-rpi_name

Webcam multiple resolution capture test for Pi Camera

Unit type:

template
Category ID:

camera
Status:

Blocking
From template:

camera/multiple-resolution-images-rpi_name
Template resource:

device
Template filter:

device.category == ‘MMAL’ and device.name != ‘’
camera/multiple-resolution-images_name

Webcam multiple resolution capture test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Blocking
After-suspend:

True
From template:

camera/multiple-resolution-images_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’
camera/roundtrip-qrcode_name

Test video output and camera {{ name }} by displaying and reading a QR code

Unit type:

template
Category ID:

camera
Status:

Blocking
From template:

camera/roundtrip-qrcode_name
Template resource:

device
Template filter:

device.category in (‘CAPTURE’, ‘MMAL’) and device.name != ‘’

CPU tests

x86_64 and ARM processors are tested to ensure proper functionality. We will test specific features as:

  • CPU’s performance states (frequency up and down in runtime)

  • CPU’s sleep states (cpu on and off in runtime)

  • Running CPU at its maximum frequency

We will also include a general stress test performed for 120 minutes to verify that the system can handle a sustained high load for a period of time. This test uses the tool “stress-ng” available in the Universe repositories.

For Intel CPU’s, the IPDT (Intel Processor Diagnostic Tool) test suite will be run. The diagnostic checks for brand identification, verifies the processor operating frequency, tests specific processor features, and performs a stress test on the processor.

The following test units are covered in this category:

cpu/arm64_vfp_support_platform

Validate that the Floating Point Unit is running on {platform} device

Unit type:

template
Category ID:

cpu
Status:

Blocking
Description:

Validate that the Floating Point Unit is running on {platform} device.
From template:

cpu/arm64_vfp_support_platform
Template resource:

cpuinfo
Template filter:

cpuinfo.platform == ‘aarch64’
cpu/armhf_vfp_support_platform

Validate that the Vector Floating Point Unit is running on {platform} device

Unit type:

template
Category ID:

cpu
Status:

Blocking
Description:

Validate that the Vector Floating Point Unit is running on {platform} device.
From template:

cpu/armhf_vfp_support_platform
Template resource:

cpuinfo
Template filter:

cpuinfo.platform == ‘armv7l’
cpu/clocktest

Tests the CPU for clock jitter

Category ID:

cpu
Status:

Blocking
Purpose:

Runs a test for clock jitter on SMP machines.
After-suspend:

True
Plugin:

shell
cpu/cstates

Run C-States tests

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Uses the Firmware Test Suite (fwts) to test the power saving states of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
cpu/cstates_results.log

Attach C-States test log

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission.
Plugin:

attachment
cpu/maxfreq_test

Test that the CPU can run at its max frequency

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.
Environment variable:

LD_LIBRARY_PATH, SNAP
User:

root
Plugin:

shell
cpu/maxfreq_test-log-attach

Attach CPU max frequency log

Category ID:

cpu
Status:

Blocking
Purpose:

Attaches the log generated by cpu/maxfreq_test to the results submission.
Plugin:

attachment
cpu/offlining_test

Test offlining of each CPU core

Category ID:

cpu
Status:

Blocking
Purpose:

Attempts to offline each core in a multicore system.
After-suspend:

True
User:

root
Plugin:

shell
cpu/scaling_test

Test the CPU scaling capabilities

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use Firmware Test Suite (fwts cpufreq) to test the scaling capabilities of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
cpu/scaling_test-log-attach

Attach CPU scaling capabilities log

Category ID:

cpu
Status:

Blocking
Purpose:

Attaches the log generated by cpu/scaling_test to the results submission.
Plugin:

attachment
cpu/topology

Check CPU topology for accuracy between proc and sysfs

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.
After-suspend:

True
Plugin:

shell

Disk tests

The following test units are covered in this category:

disk/check-software-raid

Validate the configuration of software RAID devices are expected

Category ID:

disk
Status:

Blocking
Purpose:

<missing purpose>
Description:

Examine the system to detect Software RAID devices are created and the RAID mode are expected the SOFTWARE_RAID_LEVEL variable is needed for this tests. e.g. SOFTWARE_RAID_LEVEL=”raid0 raid1 raid5”
Environment variable:

SOFTWARE_RAID_LEVEL
User:

root
Plugin:

shell
disk/detect

Gathers information about each disk detected

Category ID:

disk
Status:

Blocking
Purpose:

Uses lsblk to gather information about each disk detected on the system under test.
After-suspend:

True
Plugin:

shell
disk/read_performance_name

Disk performance test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
After-suspend:

True
Environment variable:

DISK_MDADM_READ_PERF, DISK_NVME_READ_PERF, DISK_READ_PERF
From template:

disk/read_performance_name
Template resource:

device
Template filter:

device.category == ‘DISK’
disk/stats_name

Disk statistics for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
After-suspend:

True
From template:

disk/stats_name
Template resource:

device
Template filter:

device.category == ‘DISK’ and device.name != ‘’
disk/storage_device_name

Disk I/O stress test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
After-suspend:

True
From template:

disk/storage_device_name
Template resource:

device
Template filter:

device.category == ‘DISK’
thunderbolt3/insert

Storage insert detection on Thunderbolt 3 port

Category ID:

disk
Status:

Blocking
Purpose:

This test will check if the insertion of a Thunderbolt 3 HDD could be detected
Steps:

  1. Click ‘Test’ to begin the test. This test will timeout and fail if the insertion has not been detected within 40 seconds. 2. Plug a Thunderbolt HDD into an available Thunderbolt 3 port, if it’s not mounted automatically, please click the HDD icon to mount it.
Verification:

The verification of this test is automated. Do not change the automatically selected result
After-suspend:

True
User:

root
Plugin:

user-interact
thunderbolt3/remove

Storage removal detection on Thunderbolt 3 port

Category ID:

disk
Status:

Blocking
Purpose:

This test will check if the system can detect the removal of a Thunderbolt 3 HDD
Steps:

  1. Click ‘Test’ to begin the test. This test will timeout and fail if the removal has not been detected within 20 seconds. 2. Remove the previously attached Thunderbolt HDD from the Thunderbolt port.
Verification:

The verification of this test is automated. Do not change the automatically selected result
After-suspend:

True
User:

root
Plugin:

user-interact
thunderbolt3/storage-test

Storage test on Thunderbolt 3

Category ID:

disk
Status:

Blocking
Purpose:

This is an automated test which performs read/write operations on an attached Thunderbolt HDD
After-suspend:

True
User:

root
Plugin:

shell

Docker containers

The following test units are covered in this category:

docker/build-single_arch

Test docker build with a single container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/build-single_arch
Template resource:

docker_resource
docker/commit_arch

Test docker commit a change to a single container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/commit_arch
Template resource:

docker_resource
docker/compose-and-basic_arch

Test docker compose and basic command

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/compose-and-basic_arch
Template resource:

docker_resource
docker/compose-restart_arch

Test compose a container with restart policy applied

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/compose-restart_arch
Template resource:

docker_resource
docker/compose-single_arch

Test docker compose with a single container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/compose-single_arch
Template resource:

docker_resource
docker/copy_arch

Test copy a file bwtween a container and local filesystem

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/copy_arch
Template resource:

docker_resource
docker/deploy-registry_arch

Deploy a registry server and run it on localhost

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/deploy-registry_arch
Template resource:

docker_resource
docker/diff_arch

Test changes to files in Ubuntu container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/diff_arch
Template resource:

docker_resource
docker/export-and-import_arch

Test docker import and export a docker container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/export-and-import_arch
Template resource:

docker_resource
docker/info

Display system-wide information about docker

Category ID:

docker
Status:

Blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
User:

root
docker/inspect_arch

Test query low-level information on a docker object

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/inspect_arch
Template resource:

docker_resource
docker/interative_arch

Test an interactive shell in Ubuntu container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/interative_arch
Template resource:

docker_resource
docker/kill_arch

Test killing containers

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/kill_arch
Template resource:

docker_resource
docker/restart-always_arch

Test container restart policy with always applied

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/restart-always_arch
Template resource:

docker_resource
docker/restart-on-failure_arch

Test container restart policy with on_failure applied

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/restart-on-failure_arch
Template resource:

docker_resource
docker/run_arch

Download and run ubuntu container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/run_arch
Template resource:

docker_resource
docker/save-and-load_arch

Test docker save and load a docker image

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/save-and-load_arch
Template resource:

docker_resource
docker/start-stop_arch

Start and stop a single container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/start-stop_arch
Template resource:

docker_resource
docker/update_arch

Test update configuration of one container

Unit type:

template
Category ID:

docker
Status:

Blocking
Description:

<missing description>
From template:

docker/update_arch
Template resource:

docker_resource
docker/version

Display docker version information

Category ID:

docker
Status:

Blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
User:

root

Ethernet Device tests

Connections are tested for functionality, but not for performance.

The following test units are covered in this category:

ethernet/detect

Detect if at least one ethernet device is detected

Category ID:

ethernet
Status:

Blocking
Purpose:

Test to detect and return information about available network controllers on the system under test.
After-suspend:

True
Plugin:

shell
ethernet/hotplug-interface

Ensure hotplugging works on port {{ interface }}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
After-suspend:

True
From template:

ethernet/hotplug-interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/ping_interface

Can ping another machine over Ethernet port {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
After-suspend:

True
From template:

ethernet/ping_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/wol_S3_interface

Wake on LAN (WOL) test from S3 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
From template:

ethernet/wol_S3_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’ and device.interface != ‘UNKNOWN’
ethernet/wol_S4_interface

Wake on LAN (WOL) test from S4 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
From template:

ethernet/wol_S4_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’
ethernet/wol_S5_interface

Wake on LAN (WOL) test from S5 - {interface}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
From template:

ethernet/wol_S5_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’

Firmware tests

The Ubuntu image must be installed using the factory default bootloader firmware (for example BIOS, UEFI or uboot as applicable) and with the default options (including SecureBoot, if that’s the default setting). Firmware needs to be compliant with Canonical Firmware Test Suite (FWTS).

It is recommended that after running Canonical fwts with the list of tests defined in the Appendix A, ideally, no CRITICAL or HIGH failures should be reported, but those are not automatically certification blockers.

The following test units are covered in this category:

firmware/fwts_desktop_diagnosis

Run FWTS QA-concerned desktop-specific diagnosis tests.

Category ID:

firmware
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (fwts) QA-concerned desktop-specific diagnosis tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
firmware/fwts_desktop_diagnosis_results.log.gz

Attach FWTS desktop diagnosis log to submission

Category ID:

firmware
Status:

Blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission
Plugin:

attachment

Gathers information about the DUT

The following test units are covered in this category:

connections

Collect information about connections

Category ID:

information_gathering
Status:

Blocking
Purpose:

<missing purpose>
Description:

Generates a list of plug and slot connections on the device
Plugin:

resource
rtc

Creates resource info for RTC

Category ID:

information_gathering
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

resource
serial_assertion

Collect serial assertions on the device

Category ID:

information_gathering
Status:

Blocking
Purpose:

<missing purpose>
Description:

Queries the snapd REST API for serial assertions present on the device.
Plugin:

resource
sleep

Create resource info for supported sleep states

Category ID:

information_gathering
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

resource
snap

Collect information about installed snap packages

Category ID:

information_gathering
Status:

Blocking
Purpose:

<missing purpose>
Description:

Generates a list of snap packages
Plugin:

resource

General Purpose I/O

We test the functionality of individual GPIO lines when the associated controller driver in the kernel implements a GPIO Sysfs Interface via the gpiolib implementers framework. In such cases, the GPIO system may be tested in two ways:

  • Direct:

    • GPIO controllers are exposed through sysfs

    • GPIO lines are accessible by the user

  • Indirect:

    • Communication with device connected via GPIO

The following test units are covered in this category:

gpio/gpiomem_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via /dev/gpiomem

Unit type:

template
Category ID:

gpio
Status:

Blocking
After-suspend:

True
From template:

gpio/gpiomem_loopback_pairs_model
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)
gpio/sysfs_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template
Category ID:

gpio
Status:

Blocking
Description:

<missing description>
After-suspend:

True
From template:

gpio/sysfs_loopback_pairs_model
Template resource:

com.canonical.certification::model_assertion
Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)
gpio/sysfs_loopback_pairs_vendor_product

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template
Category ID:

gpio
Status:

Blocking
Description:

<missing description>
After-suspend:

True
From template:

gpio/sysfs_loopback_pairs_vendor_product
Template resource:

dmi
Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’, ‘UPN-EHL01’)

Informational tests

The following test units are covered in this category:

info/systemd-analyze

System boot-up performance statistics

Category ID:

info
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
info/systemd-analyze-critical-chain

Print the tree of the time-critical chain of SystemD

Category ID:

info
Status:

Blocking
Purpose:

This job prints a tree of the time-critical chain of SystemD units.
Plugin:

shell
lspci_attachment

Attach a list of PCI devices

Category ID:

info
Status:

Blocking
Purpose:

Attaches very verbose lspci output.
Plugin:

attachment
lsusb_attachment

Attach output of lsusb

Category ID:

info
Status:

Blocking
Purpose:

Attaches a list of detected USB devices.
After-suspend:

True
User:

root
Plugin:

attachment
net_if_management_attachment

Collect logging from the net_if_management job

Category ID:

info
Status:

Blocking
Purpose:

<missing purpose>
Description:

Allows logging of debug info and errors by the associated resource job
Plugin:

attachment
parts_meta_info_attachment

Attaches an information about all parts that constituted this snap

Category ID:

info
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

CHECKBOX_RUNTIME, SNAP
User:

root
Plugin:

attachment

Input Devices tests

The following test units are covered in this category:

input/zapper-keyboard

Check if the system recognizes keyboard events from Zapper

Category ID:

input
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

ZAPPER_HOST
User:

root
Plugin:

shell

I²C (Inter-Integrated Circuit)

All devices attached to the I2C bus must be detectable. This includes:

  • Temperature sensors

  • Humidity sensors

  • Accelerometers

The following test units are covered in this category:

i2c/i2c-bus-detect

Check presence of an I²C bus

Category ID:

i2c
Status:

Blocking
Purpose:

If an expected number of I²C buses is provided, the job will verify the detected number is correct. If the expected number of buses is not provided, the job will pass if at least one I²C bus is detected.
Environment variable:

I2C_BUS_NUMBER
User:

root
Plugin:

shell
i2c/i2c-device-detect

Check if any I²C devices can be detected

Category ID:

i2c
Status:

Blocking
Purpose:

The test will pass if there’s at least one I²C device detected on any I²C bus.
User:

root
Plugin:

shell

Kernel snap tests

The following test units are covered in this category:

kernel-snap/booted-kernel-matches-current-name

The booted kernel image matches the image in the current kernel snap

Unit type:

template
Category ID:

kernel-snap
Status:

Blocking
From template:

kernel-snap/booted-kernel-matches-current-name
Template resource:

bootloader
Template filter:

bootloader.booted_kernel_path != ‘unknown’

LED tests

When LEDs exist, they will be tested by following some basic expectations here. The actual behavior may vary depending on the hardware design. To ensure that the behavior is working as expected, please be sure to test against specifications obtained from OEM, as each OEM may have different defined behavior for LEDs. The following LEDs are tested:

  • Power

  • Serial Port LEDs (indicating activity)

The following test units are covered in this category:

led/fn

Test the Fn key LED functionality by activating/deactivating the Fn keys locking.

Category ID:

led
Status:

Blocking
Purpose:

This test will test the Fn key LED.
Steps:

  1. Press the Fn+Fn Lock key to activate/deactivate Fn keys locking. 2. The Fn key LED should be switched on/off every time the key is pressed.
Verification:

Did the Fn key LED light as expected?
After-suspend:

True
Plugin:

manual
led/power

Power LED behavior when powered

Category ID:

led
Status:

Blocking
Purpose:

Check power led is on when system is powered on
Steps:

  1. Check power led when system is powered on
Verification:

Power led is on when system is powered on
After-suspend:

True
Plugin:

manual
led/serial

Serial ports LED behavior

Category ID:

led
Status:

Blocking
Purpose:

Check serial ports LED behavior is correct
Steps:

  1. Start the test to send data to all serial ports (/dev/ttyS*)
Verification:

All serial ports LED are on for a few seconds (3-4s)
User:

root
Plugin:

user-interact-verify
led/sysfs_led_brightness_off_vendor_product

Ensure the leds_aaeon driver properly sets all LEDs to off or minimum brightness by running a test.

Unit type:

template
Category ID:

led
Status:

Blocking
After-suspend:

True
From template:

led/sysfs_led_brightness_off_vendor_product
Template resource:

dmi
Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’)
led/sysfs_led_brightness_on_vendor_product

Verify the functionality of the leds_aaeon driver by ensuring all external LEDs achieve maximum brightness.

Unit type:

template
Category ID:

led
Status:

Blocking
After-suspend:

True
From template:

led/sysfs_led_brightness_on_vendor_product
Template resource:

dmi
Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’)
led/wireless

Verify the WLAN/Bluetooth LED functionality by toggling wireless connections.

Category ID:

led
Status:

Blocking
Purpose:

Wireless (WLAN + Bluetooth) LED verification
Steps:

  1. Make sure WLAN connection is established and Bluetooth is enabled. 2. WLAN/Bluetooth LED should light 3. Switch WLAN and Bluetooth off from a hardware switch (if present) 4. Switch them back on 5. Switch WLAN and Bluetooth off from the panel applet 6. Switch them back on
Verification:

Did the WLAN/Bluetooth LED light as expected?
Plugin:

manual

Location Service

The following test units are covered in this category:

location/status

Queries the status of a service instance

Category ID:

location
Status:

Blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
After-suspend:

True

Media Card tests

Media Card readers are tested for read and write for the following type of cards:

  • CF

  • MMC

  • MS

  • MSP

  • SD

  • SDHC

  • SDXC

  • XD

The following test units are covered in this category:

mediacard/cf-insert

Verify the system’s media card reader can detect a Compact Flash card insertion.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Compact Flash (CF) media card
Steps:

  1. Commence the test and then insert a CF card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/cf-remove

Ensure the system detects CF card removal from the card reader correctly.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a CF card from the system’s card reader.
Steps:

  1. Commence the test and then remove the CF card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/cf-storage

Automate testing for reading and writing to the CF card after mediacard/cf-insert test.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/cf-insert test is run. It tests reading and writing to the CF card.
User:

root
Plugin:

shell
mediacard/mmc-insert

Test the system’s media card reader for MMC insertion detection.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Multimedia Card (MMC) media
Steps:

  1. Commence the test and then insert an MMC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/mmc-remove

Test the detection of MMC card removal from the system’s card reader.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of the MMC card from the system’s card reader.
Steps:

  1. Commence the test and then remove the MMC card from the reader. (Note: this test will time out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/mmc-storage

Test automated execution for reading and writing to the MMC card.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/mmc-insert test is run. It tests reading and writing to the MMC card.
User:

root
Plugin:

shell
mediacard/ms-insert

Verify the detection of Memory Stick (MS) card insertion by the system’s media card reader.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Memory Stick (MS) media card
Steps:

  1. Commence the test and then insert a MS card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/ms-remove

Test if the system detects the removal of an MS card correctly.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of an MS card from the system’s card reader.
Steps:

  1. Commence the test and then remove the MS card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/ms-storage

Automated test for reading and writing to the MS card after mediacard/ms-insert test.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/ms-insert test is run. It tests reading and writing to the MS card.
User:

root
Plugin:

shell
mediacard/msp-insert

Verify the system’s media card reader can detect a Memory Stick Pro (MSP) card insertion.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Memory Stick Pro (MSP) media card
Steps:

  1. Commence the test and then insert a MSP card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/msp-remove

Ensure MSP card removal is correctly detected by the system.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a MSP card from the system’s card reader.
Steps:

  1. Commence the test and remove the MSP card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/msp-storage

Automated test for reading and writing to the MSP card after mediacard/msp-insert test.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/msp-insert test is run. It tests reading and writing to the MSP card.
User:

root
Plugin:

shell
mediacard/sd-insert

Test that insertion of an SD card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of an UNLOCKED Secure Digital (SD) media card
Steps:

  1. Commence the test and then insert an UNLOCKED SD card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/sd-remove

Test that removal of an SD card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of an SD card from the system’s card reader.
Steps:

  1. Commence the test and then remove the SD card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/sd-storage

Test reading & writing to an SD Card

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/sd-insert test is run. It tests reading and writing to the SD card.
User:

root
Plugin:

shell
mediacard/sdhc-insert

Test that insertion of an SDHC card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a UNLOCKED Secure Digital High-Capacity (SDHC) media card
Steps:

  1. Commence the test and then insert an UNLOCKED SDHC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
mediacard/sdhc-remove

Test that removal of an SDHC card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of an SDHC card from the system’s card reader.
Steps:

  1. Commence the test and then remove the SDHC card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
mediacard/sdhc-storage

Test reading & writing to a SDHC Card

Category ID:

mediacard
Status:

Blocking
Purpose:

<missing purpose>
Description:

This test is automated and executes after the mediacard/sdhc-insert test is run. It tests reading and writing to the SDHC card.
After-suspend:

True
User:

root
Plugin:

shell
mediacard/sdxc-insert

Test that insertion of an SDXC card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a Secure Digital Extended Capacity (SDXC) media card
Steps:

  1. Commence the test and then insert an UNLOCKED SDXC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/sdxc-remove

Test that removal of an SDXC card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a SDXC card from the system’s card reader.
Steps:

  1. Commence the test and then remove the SDXC card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/sdxc-storage

Test reading & writing to an SDXC Card

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/sdxc-insert test is run. It tests reading and writing to the SDXC card.
User:

root
Plugin:

shell
mediacard/xd-insert

Ensure the system’s media card reader detects the insertion of an xD media card.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of an Extreme Digital (xD) media card.
Steps:

  1. Commence the test and then insert an xD card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/xd-remove

Automated test for detecting the removal of a xD card from the system’s card reader.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a xD card from the system’s card reader.
Steps:

  1. Commence the test and then remove the xD card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
User:

root
Plugin:

user-interact
mediacard/xd-storage

Automated test to verify reading and writing functionality of the xD card after mediacard/xd-insert test.

Category ID:

mediacard
Status:

Blocking
Purpose:

This test is automated and executes after the mediacard/xd-insert test is run. It tests reading and writing to the xD card.
User:

root
Plugin:

shell

Memory tests

Proper detection of the amount of memory installed is required (the amount of memory installed is the memory seen by the OS).

The following test units are covered in this category:

memory/info

Check the amount of memory reported by meminfo against DMI

Category ID:

memory
Status:

Blocking
Purpose:

This test checks the amount of memory which is reported in meminfo against the size of the memory modules detected by DMI.
User:

root
Plugin:

shell

Miscellaneous tests

The following test units are covered in this category:

miscellanea/device_check

Device Check

Category ID:

miscellanea
Status:

Blocking
Purpose:

Device check
Steps:

  1. Commence the test 2. Compare items on System Manifest to the devices known to udev
Verification:

Do the devices reported by udev match the devices on the Manifest?
Plugin:

user-interact-verify
miscellanea/submission-resources

Check that data for a complete result are present

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

A meta-job that verifies the data necessary for a complete result submission are present. Failure indicates that the results are incomplete and may be rejected.
Plugin:

shell

Monitor tests

Each of the available external video ports (supported ports are HDMI, DisplayPort, DVI) are tested one by one. Output to the display must work i.e. a console is presented.

The following test units are covered in this category:

monitor/displayport_hotplug

Can hotplug monitor (DisplayPort)

Category ID:

monitor
Status:

Blocking
Purpose:

This test will check the DisplayPort port and the ability to do hotplugging.
Steps:

Skip this test if your system does not have a DisplayPort port. 1. If a display is already connected, unplug it. 2. (Re-)Connect a display to the DisplayPort port on your system
Verification:

Was the interface displayed correctly on the screen?
After-suspend:

True
Plugin:

manual
monitor/dvi

Monitor works (DVI)

Category ID:

monitor
Status:

Blocking
Purpose:

Check output to display through DVI port
Steps:

  1. Connect display to DVI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
monitor/dvi-to-vga

Monitor works (DVI-to-VGA)

Category ID:

monitor
Status:

Blocking
Purpose:

Check output to display through VGA adaptor on DVI port
Steps:

  1. Connect display to VGA adaptor on DVI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
monitor/hdmi

Monitor works (HDMI)

Category ID:

monitor
Status:

Blocking
Purpose:

Check output to display through HDMI port
Steps:

  1. Connect display to HDMI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
monitor/hdmi-to-vga

Monitor works (HDMI-to-VGA)

Category ID:

monitor
Status:

Blocking
Purpose:

Check output to display through VGA adaptor on HDMI port
Steps:

  1. Connect display to VGA adaptor on HDMI port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
monitor/vga

Monitor works (VGA)

Category ID:

monitor
Status:

Blocking
Purpose:

Check output to display through VGA port
Steps:

  1. Connect display to VGA port 2. Check the display
Verification:

Output to display works
After-suspend:

True
Plugin:

manual
monitor/zapper-edid

Check if the system automatically changes the resolution based on EDID

Category ID:

monitor
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Environment variable:

ZAPPER_HOST
Plugin:

shell

Non-device specific networking tests

The following test units are covered in this category:

ipv6_detect

Test if the kernel is IPv6 ready

Category ID:

networking
Status:

Blocking
Purpose:

Test if the running kernel supports IPv6.
Steps:

<missing steps>
Verification:

<missing verification>
After-suspend:

True
networking/info_device__index___interface

Network Information of device {__index__} ({interface})

Unit type:

template
Category ID:

networking
Status:

Blocking
After-suspend:

True
From template:

networking/info_device__index___interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’
networking/predictable_names

Verify that all network interfaces have predictable names.

Category ID:

networking
Status:

Blocking
Purpose:

Verify that all network interfaces have predictable names.
Plugin:

shell

Power Management tests

Warm reboot is tested such that the system must be able to perform the reboot command and services must be restarted such that systemctl does not identify a failed state.

Cold reboot is performed where an RTC is available (see next section). The wakealarm is used to reboot the system after a period of rest and services must be restarted such that systemctl does not identify a failed state.

The following test units are covered in this category:

power-management/cold-reboot

Cold reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This test powers off the system and then powers it on using RTC
Environment variable:

COLD_REBOOT_DELAY, RTC_DEVICE_FILE
User:

root
Plugin:

shell
power-management/post-cold-reboot

Post cold reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the cold reboot
Plugin:

shell
power-management/post-warm-reboot

Post warm reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the warm reboot
Plugin:

shell
power-management/warm-reboot

Warm reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This tests reboots the system using the reboot command
User:

root
Plugin:

shell
watchdog/detect

Detect the presence of a Hardware Watchdog

Category ID:

power-management
Status:

Blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
watchdog/post-trigger-system-reset-auto

Post watchdog reset service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the watchdog triggered
Plugin:

shell
watchdog/systemd-config

Check if the hardware watchdog is properly configured

Category ID:

power-management
Status:

Blocking
Purpose:

<missing purpose>
Steps:

<missing steps>
Verification:

<missing verification>
Description:

<missing description>
watchdog/trigger-system-reset-auto

Test that the watchdog module can trigger a system reset

Category ID:

power-management
Status:

Blocking
Purpose:

Ensure that the watchdog module can successfully initiate a system reset.
User:

root
Plugin:

shell

Real Time Clock (RTC)

The following test units are covered in this category:

rtc/battery

RTC battery tracks the time and ensures the system can wake up from power off state.

Category ID:

rtc_category
Status:

Blocking
Purpose:

RTC battery backup power can send system wakeup events.
Steps:

  1. Start the test to power off the system (wakeup scheduled in 30s).
Verification:

RTC can wake up the system successfully. environ: RTC_DEVICE_FILE
User:

root
Plugin:

user-interact

Serial Port

Tests are carried out on ports that provide access via the Linux tty layer. The exact tests performed depend on the physical characteristics of the driver/receiver hardware. The possible tests include:

  • Ensure expected number of devices are available

  • Looped tests:

  • RS232 Ports: perform loopback test to ensure RX/TX

  • RS422/485 Ports: connect together to ensure RX/TX

  • Machine to Machine tests: confirm that a connection can be made to another

PC device and RX/TX is operational

The following test units are covered in this category:

serial/loopback-dev

Serial loopback test of {dev}

Unit type:

template
Category ID:

serial
Status:

Blocking
After-suspend:

True
From template:

serial/loopback-dev
Template resource:

serial_ports_static
serial/rs232-console

Serial debugging console is enabled and operational

Category ID:

serial
Status:

Blocking
Purpose:

Check user can log into system through serial port from another machine
Steps:

  1. Connect USB to DB9 null modem adapter cable to the serial port of the test machine. 2. Connect the cable to a USB port of another Ubuntu machine (client). 3. Install screen on the client (if not done in Prerequisite). 4. Execute the following command on the client: sudo screen /dev/ttyUSB0 5. Start the getty service on the test machine: sudo systemctl start getty@[rs232 device, e.g., /dev/ttyS0].service 6. Log into the test machine from the terminal on the client.
Verification:

  1. The output to the client is fine after the getty service is started. 2. Log into the test machine from the terminal on the client successfully.
After-suspend:

True
Plugin:

manual

Snapd

The following test units are covered in this category:

snappy/snap-install

Test the snap install command is working

Category ID:

snapd
Status:

Blocking
Purpose:

The store should contain the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap. The test makes sure this can be downloaded and installed on the system.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-list

Test that the snap list command is working.

Category ID:

snapd
Status:

Blocking
Purpose:

If snap list command is working then should at least find the ubuntu-core package.
Plugin:

shell
snappy/snap-refresh-automated

Test whether the snap refresh command is working.

Category ID:

snapd
Status:

Blocking
Purpose:

The test will install the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap from the stable channel and then refreshes it to the edge channel and compares the revision before and after the refresh.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-remove

Test the snap remove command is working.

Category ID:

snapd
Status:

Blocking
Purpose:

After having installed the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap, check it can be removed.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-reupdate-automated

Test the snap refresh command works after blacklisting.

Category ID:

snapd
Status:

Blocking
Purpose:

Checks that the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap can be refreshed after removal of the blacklisted revision.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-revert-automated

Test the snap revert command is working.

Category ID:

snapd
Status:

Blocking
Purpose:

Checks if the edge channel {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap is reverted back to the one from stable.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-snaps-confinement

Test all the snaps’ confinement

Category ID:

snapd
Status:

Blocking
Purpose:

Test all the snaps’ confinement, devmode, revision. Make sure the confinement is “strict”, devmode is “False”, and revision should not be sideloaded.
Plugin:

shell
snappy/test-store-config-store

Test that image is using the correct snappy store configuration.

Unit type:

template
Category ID:

snapd
Status:

Blocking
Environment variable:

STORE_ID
From template:

snappy/test-store-config-store
Template resource:

com.canonical.certification::model_assertion
snappy/test-store-install-beta

Snappy install command - beta channel store

Category ID:

snapd
Status:

Blocking
Purpose:

Test the snappy install command is able to install and remove snap in beta channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-store-install-edge

Snappy install command - edge channel store

Category ID:

snapd
Status:

Blocking
Purpose:

Test the snappy install command is able to install a snap in the edge channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-system-confinement

Test if the system confinement is strict

Category ID:

snapd
Status:

Blocking
Purpose:

Test if the system confinement is “strict” If not, list the missing features
Plugin:

shell

SocketCAN interface tests

The following test units are covered in this category:

socketcan/send_packet_local_eff_virtual

Virtual CAN device support test (Local test with raw socket and EFF)

Category ID:

socketcan
Status:

Blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_eff_interface

CAN device support test for {interface} (Raw, Local, EFF)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_local_eff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_local_fd_virtual

Virtual CAN device support test (Raw, Local, FD)

Category ID:

socketcan
Status:

Blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket; this is only a local test as the broadcast packet is received on the same device.
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_fd_interface

CAN device support test for {interface} (Raw, Local, FD)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_local_fd_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_local_sff_virtual

Virtual CAN device support test (Raw, Local)

Category ID:

socketcan
Status:

Blocking
Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device
After-suspend:

True
User:

root
Plugin:

shell
socketcan/send_packet_local_sff_interface

CAN device support test for {interface} (Raw, Local)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_local_sff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_eff_interface

CAN device support test {interface} (Raw, Remote, EFF)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_remote_eff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_fd_interface

CAN device support test {interface} (Raw, Remote, FD)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_remote_fd_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’
socketcan/send_packet_remote_sff_interface

CAN device support test {interface} (Raw, Remote)

Unit type:

template
Category ID:

socketcan
Status:

Blocking
After-suspend:

True
From template:

socketcan/send_packet_remote_sff_interface
Template resource:

device
Template filter:

device.category == ‘SOCKETCAN’

TPM 2.0 (Trusted Platform Module)

On Intel and AMD x86 platforms that include TPM 2.0 compliant modules, it is required that all commands necessary to support Ubuntu’s Full Disk Encryption functionality are supported.

The following test units are covered in this category:

clevis-encrypt-tpm2/detect-ecc-capabilities

Ensure the TPM has required capabilities for clevis ECC test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/detect-rsa-capabilities

Ensure the TPM has required capabilities for clevis RSA test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/ecc

clevis encrypt/decrypt key ecc

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/precheck

clevis encrypt/decrypt precheck

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
clevis-encrypt-tpm2/rsa

clevis encrypt/decrypt key rsa

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
tpm2/fwts-event-log-dump

Dump the contents of the TPM Event Log

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

The information in the TPM Event Log can be useful in debugging problems with TPM command support and adherance to standards. This can be of particular help when checking whether a device can support Ubuntu Core Full Disk Encryption.
User:

root
Plugin:

attachment

Ubuntu Core OS feature tests

The following test units are covered in this category:

ubuntucore/sshd

SSH is enabled and operational

Category ID:

ubuntucore
Status:

Blocking
Purpose:

Check if a user can access the system through SSH from another machine
Steps:

  1. Execute the following command on another machine in the same network: $ ssh [user id]@[ip address of the testing system] 2. Enter the password to log in
Verification:

Can log into the system through SSH from another machine
Plugin:

manual

USB tests

USB 2.0

USB storage devices must work on all available USB ports. USB Human Interface Devices (HID), specifically keyboard or mouse, should be working properly on any USB port.

USB 3.0

USB storage devices must work on all available USB ports. USB Human Interface Devices (HID), specifically keyboard or mouse, should be working properly on any USB port.

USB Type C (USB 3.1)

USB Type C (USB 3.1) supports various types of devices (e.g. Video, Power) through the use of adapters or peripherals. The following adapters/peripherals should work:

  • Storage devices

  • Keyboard or mouse (basic functionality)

  • When DisplayPort over USB Type-C is advertised:

  • Display hot plugging and the following display are required to work:

    mirrored, extended, internal only, external only.

  • Audio output needs to be undistorted over this port.

The following test units are covered in this category:

usb-c-otg/g_ether

Check DUT can be detected as USB ethernet device

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Steps:

  1. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT. 2. Press Enter to configure the IP address for the USB OTG ethernet interface on the DUT. (IP will be 192.168.9.1/24) 3. On the host, configure the IP address for the USB OTG ethernet interface. It should be in the same subnet as the DUT. (e.g., 192.168.9.2/24) 4. Try to ping between the host and DUT.
Verification:

The host and DUT can ping each other.
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

user-interact-verify
usb-c-otg/g_ether-cleanup

Cleanup USB OTG ethernet interface setup after ethernet device test

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
usb-c-otg/g_mass_storage

Check DUT can be detected as a mass storage device

Category ID:

usb
Status:

Blocking
Purpose:

Check that after connecting the device under test (DUT) to another device (host), the DUT can be detected as a mass storage device by the host.
Steps:

  1. Press Enter to setup a 16 MB FAT32 image on the device. 2. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT.
Verification:

The host detects and mounts the mass storage device. It has read and write permissions on it.
After-suspend:

True
User:

root
Plugin:

user-interact-verify
usb-c-otg/g_mass_storage-cleanup

Cleanup mass storage setup after mass storage device test

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
usb-c-otg/g_serial

Check if USB OTG can work as a serial port.

Category ID:

usb
Status:

Blocking
Purpose:

Check if USB OTG can work as a serial port.
Steps:

  1. Press enter to probe USB OTG g_serial 2. Connect a USB cable from an Ubuntu host to the USB OTG (it’s the USB Type C connector) port on DUT. 3. On the host, check if ttyACM0 has been probed $ ls /dev/ttyACM0 4. On the host, listen to the ttyACM0 port as the receiving side. $ sudo su $ cat /dev/ttyACM0 5. On DUT, send a string via /dev/ttyGS0 as the sending side. $ sudo su $ echo 123 > /dev/ttyGS0 6. Check if the string “123” was received on the host side. 7. Check the string sending from the host to DUT by repeating steps 4-6, but swap the send and receive sides.
Verification:

Does string send and receive function correctly?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
usb-c-otg/g_serial-cleanup

Cleanup USB OTG serial interface setup after serial device test

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
usb-c/c-to-a-adapter/hid

USB HID work on USB Type-C port using a “USB Type-C to Type-A” adapter

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port using a “USB Type-C to Type-A” adapter
Steps:

  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port using a “USB Type-C to Type-A” adapter 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text
Verification:

Did the device work as expected?
After-suspend:

True
Plugin:

manual
usb-c/c-to-a-adapter/insert

Verify that the system detects a USB 3 storage device using a “Type-C to Type-A” adapter.

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3 storage device in a USB Type-C connector using a “Type-C to Type-A” adapter. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test 2. Connect a USB 3 storage device to a USB Type-C port using a “Type-C to Type-A” adapter. 3. Do not unplug the device after the test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/c-to-a-adapter/remove

Check the detection of a USB 3 storage device removal, connected via a “Type-C to Type-A” adapter.

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a USB 3 storage device connected to a USB Type-C port using a “USB Type-C to Type-A” adapter.
Steps:

  1. Begin the test 2. Disconnect a USB 3 storage device from a USB Type-C port using a “Type-C to Type-A” adapter
Verification:

The verification of this test is automated. Do not modify the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/c-to-a-adapter/storage-automated

Automate the execution process post usb-c/c-to-a-adapter/insert test.

Category ID:

usb
Status:

Blocking
Purpose:

This test is automated and executes after the usb-c/c-to-a-adapter/insert test is run.
After-suspend:

True
User:

root
Plugin:

shell
usb-c/hid

USB HID work on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port
Steps:

  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text
Verification:

Did the device work as expected?
After-suspend:

True
Plugin:

manual
usb-c/insert

USB 3.0 storage device insertion detected on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3.0 storage device in a USB Type-C connector. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test. 2. Connect a USB 3.0 storage device to a USB Type-C port. 3. Do not unplug the device after the test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/remove

USB 3.0 storage device removal detected on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a USB 3.0 storage device connected to a USB Type-C port.
Steps:

  1. Commence the test 2. Connect a USB 3.0 storage device to a USB Type-C port 3. Disconnect the USB 3.0 storage device from the USB Type-C port
Verification:

The system should automatically detect the removal of the USB 3.0 storage device. The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/storage-automated

USB 3.0 storage device read & write works on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test is automated and executes after the usb-c/insert test is run.
After-suspend:

True
User:

root
Plugin:

shell
usb/hid

USB keyboard works

Category ID:

usb
Status:

Blocking
Purpose:

Check USB input device works
Steps:

  1. Connect USB keyboard 2. Input something with USB keyboard
Verification:

What was just input is displayed correctly
After-suspend:

True
Plugin:

manual
usb/insert

USB 2.0 storage device insertion detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect USB 2.0 storage when inserted. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Press continue 2. Wait until the message “INSERT NOW” is printed on the screen 3. Connect USB 2.0 storage device
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb/remove

USB 2.0 storage device removal detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect removal of a USB 2.0 storage device
Steps:

  1. Press continue 2. Wait until the message “REMOVE NOW” is printed on the screen 3. Disconnect USB 2.0 storage device
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb/storage-automated

USB 2.0 storage device read & write works

Category ID:

usb
Status:

Blocking
Purpose:

Check system can read/write to USB 2.0 storage correctly
After-suspend:

True
User:

root
Plugin:

shell
usb/storage-detect

Detect storage partitions on a device on the USB bus

Category ID:

usb
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
usb/zapper-usb-insert-port_alias

Zapper-driven USB insertion test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb/zapper-usb-insert-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’
usb/zapper-usb-remove-port_alias

Zapper-driven USB removal test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb/zapper-usb-remove-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’
usb3/insert

USB 3.0 storage device insertion detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect insertion of a USB 3.0 storage device. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Press continue. 2. Wait until the message “INSERT NOW” is displayed on the screen. 3. Connect USB 3.0 storage device.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb3/remove

USB 3.0 storage device removal detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect removal of a USB 3.0 storage device
Steps:

  1. Press continue 2. Wait until the message “REMOVE NOW” is printed on the screen 3. Disconnect USB 3.0 storage device
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb3/storage-automated

USB 3.0 storage device read & write works

Category ID:

usb
Status:

Blocking
Purpose:

Check system can read/write to USB 3.0 storage devices correctly
After-suspend:

True
User:

root
Plugin:

shell
usb3/zapper-usb-insert-port_alias

Zapper-driven USB 3.0 insertion test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb3/zapper-usb-insert-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’
usb3/zapper-usb-remove-port_alias

Zapper-driven USB 3.0 removal test on port {{ port_alias }}

Unit type:

template
Category ID:

usb
Status:

Blocking
Description:

<missing description>
Environment variable:

ZAPPER_HOST
From template:

usb3/zapper-usb-remove-port_alias
Template resource:

zapper_capabilities
Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’

Wi-Fi access point

The following test units are covered in this category:

wireless/nmcli_wifi_ap_a_interface

Create 802.11a Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
Description:

<missing description>
After-suspend:

True
From template:

wireless/nmcli_wifi_ap_a_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/nmcli_wifi_ap_bg_interface

Create 802.11b/g Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
From template:

wireless/nmcli_wifi_ap_bg_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_b_no_sta_interface_auto

Create open 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
Environment variable:

WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_open_b_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_b_no_sta_interface_manual

Create open 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
From template:

wireless/wifi_ap_open_b_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_g_no_sta_interface_auto

Create an open 802.11g Wi-Fi AP on {interface} with no STA connected.

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
Environment variable:

WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_open_g_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_open_g_no_sta_interface_manual

Create open 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
From template:

wireless/wifi_ap_open_g_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_setup_wizard_interface_auto

Create Access Point on {interface} using wifi-ap.setup-wizard

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
Environment variable:

WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_setup_wizard_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_no_sta_interface_auto

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
Environment variable:

WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_wpa_b_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_no_sta_interface_manual

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
From template:

wireless/wifi_ap_wpa_b_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_b_with_sta_interface_auto

Create a WPA2 802.11b Wi-Fi Access Point on {interface} with active STA

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
Description:

<missing description>
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, OPEN_BG_SSID, WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_wpa_b_with_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_no_sta_interface_auto

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
Environment variable:

WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_wpa_g_no_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_no_sta_interface_manual

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
After-suspend:

True
From template:

wireless/wifi_ap_wpa_g_no_sta_interface_manual
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wifi_ap_wpa_g_with_sta_interface_auto

Create WPA2 802.11g Wi-Fi Access Point on {interface} with active STA

Unit type:

template
Category ID:

wifi_ap
Status:

Blocking
Description:

<missing description>
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, OPEN_BG_SSID, WIFI_AP_SETUPTIME
From template:

wireless/wifi_ap_wpa_g_with_sta_interface_auto
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless networking tests

Wi-Fi interfaces are tested for connection to access points configured for 802.11 b/g/n/ac/ax protocols.

The following test units are covered in this category:

wireless/check_iwlwifi_microcode_crash_interface

Check there have been no iwlwifi crashes

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/check_iwlwifi_microcode_crash_interface
Template resource:

device
Template filter:

device.driver == ‘iwlwifi’
wireless/detect

Detect if at least one Wireless LAN device is detected

Category ID:

wireless
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
wireless/wireless_connection_open_ac_nm_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ac_np_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_AC_SSID
From template:

wireless/wireless_connection_open_ac_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_nm_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_np_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_AX_SSID
From template:

wireless/wireless_connection_open_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_nm_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_np_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_BE_SSID
From template:

wireless/wireless_connection_open_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_nm_interface

Connect to an unencrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_np_interface

Connect to unencrypted 802.11b/g Wi-Fi network on {{ interface }} using netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_BG_SSID
From template:

wireless/wireless_connection_open_bg_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_nm_interface

Connect to an unencrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_np_interface

Connect to unencrypted 802.11n Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_N_SSID
From template:

wireless/wireless_connection_open_n_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_nm_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_np_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA3_AX_PSK, WPA3_AX_SSID
From template:

wireless/wireless_connection_wpa3_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_nm_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_np_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA3_BE_PSK, WPA3_BE_SSID
From template:

wireless/wireless_connection_wpa3_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_nm_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_np_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_AC_PSK, WPA_AC_SSID
From template:

wireless/wireless_connection_wpa_ac_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_nm_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_np_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_AX_PSK, WPA_AX_SSID
From template:

wireless/wireless_connection_wpa_ax_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_nm_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_np_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_BE_PSK, WPA_BE_SSID
From template:

wireless/wireless_connection_wpa_be_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_nm_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_np_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }} - netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_BG_PSK, WPA_BG_SSID
From template:

wireless/wireless_connection_wpa_bg_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_nm_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_np_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }} using netplan

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_N_PSK, WPA_N_SSID
From template:

wireless/wireless_connection_wpa_n_np_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_scanning_interface

Test system can discover Wi-Fi networks on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_scanning_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless Wide Area Network

WWAN interfaces are tested for connection to 3G/4G/LTE services.

The following test units are covered in this category:

wwan/3gpp-scan-manufacturer-model-hw_id-auto

Scan for available 3GPP networks with the modem

Unit type:

template
Category ID:

wwan
Status:

Blocking
Description:

Scan for available 3GPP networks with the target modem
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH
From template:

wwan/3gpp-scan-manufacturer-model-hw_id-auto
Template resource:

wwan_resource
wwan/check-sim-present-manual

Check if a SIM card is present in a slot connected to the modem

Category ID:

wwan
Status:

Blocking
Purpose:

Check if a SIM card is present in a slot connected to the modem
Steps:

  1. Open another terminal on SUT (or press Ctrl+Z to suspend Checkbox) 2. Run sudo mmcli -m 0 | grep SIM 3. Run fg to jump back to Checkbox (if you’re running in the same terminal)
Verification:

Check the output. Was the SIM information printed?
After-suspend:

True
Plugin:

manual
wwan/check-sim-present-manufacturer-model-hw_id-auto

Check if a SIM card is present in a slot connected to the modem

Unit type:

template
Category ID:

wwan
Status:

Blocking
Description:

Check if a SIM card is present in a slot connected to the modem
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH
From template:

wwan/check-sim-present-manufacturer-model-hw_id-auto
Template resource:

wwan_resource
wwan/detect

Identify if WWAN module is missing

Category ID:

wwan
Status:

Blocking
Purpose:

Tests that there is a WWAN module present and indicates that testing of it should follow.
After-suspend:

True
User:

root
Plugin:

shell
wwan/detect-manual

Check if WWAN module is available

Category ID:

wwan
Status:

Blocking
Purpose:

Check if WWAN module is available and ready to be used.
Steps:

  1. Open another terminal on SUT (or press ctrl+z to suspend Checkbox) 2. Run sudo mmcli -L command. 3. Run fg to jump back to checkbox (if you’re running in the same terminal)
Verification:

Check the output. Was the modem listed?
After-suspend:

True
Plugin:

manual
wwan/gsm-connection-interrupted-manual

Verify a GSM broadband connection can be reconnected after the signal is lost

Category ID:

wwan
Status:

Blocking
Purpose:

Ensure that the data connection can be revived after losing signal
Steps:

  1. Open another terminal on SUT (or press ctrl+z to suspend Checkbox). 2. Run sudo nmcli c add type gsm ifname {{ __checkbox_env__.get(“WWAN_CONTROL_IF”, “<device name>”) }} con-name GSM apn {{ __checkbox_env__.get(“WWAN_APN”, “<your.carrier.com>”) }} {%- if “WWAN_CONTROL_IF” not in __checkbox_env__ %} Replacing <your.carrier.com> with the appropriate APN name, and <device name> with the appropriate device (device can be found by running sudo nmcli d). {%- endif %} {%- if “WWAN_NET_IF” not in __checkbox_env__ %} 3. Ping Google (Run: ping -I <ifname> 8.8.8.8 -c 5). {%- else %} 3. Ping Google (Run: ping -I {{ __checkbox_env__.get(“WWAN_NET_IF”) }} 8.8.8.8 -c 5). {%- endif %} 4. Place the system in a Faraday bag. 5. Ping Google (and verify that the connection no longer works). 6. Remove the Faraday bag. {%- if “WWAN_NET_IF” not in __checkbox_env__ %} 7. Ping Google (Run: ping -I <ifname> 8.8.8.8 -c 5). {%- else %} 7. Ping Google (Run: ping -I {{ __checkbox_env__.get(“WWAN_NET_IF”) }} 8.8.8.8 -c 5). {%- endif %} 8. Disconnect from the network (Run: sudo nmcli r wwan off). 9. Run fg to jump back to Checkbox (if you’re running in the same terminal).
Verification:

Was the connection revived after removing the Faraday bag?
After-suspend:

True
Plugin:

manual
wwan/gsm-connection-manufacturer-model-hw_id-auto

Verify a GSM broadband modem can create a data connection

Unit type:

template
Category ID:

wwan
Status:

Blocking
After-suspend:

True
Environment variable:

LD_LIBRARY_PATH, WWAN_APN, WWAN_CONTROL_IF, WWAN_NET_IF, WWAN_SETUPTIME
From template:

wwan/gsm-connection-manufacturer-model-hw_id-auto
Template resource:

wwan_resource

Non-blocking

Informational tests

The following test units are covered in this category:

manifest

Hardware Manifest

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

This job loads the hardware manifest and exposes it as a resource.
Plugin:

resource

Appendix A. FWTS tests

As part of the certification process, we run a series of firmware tests that are part of the Canonical Firmware Test Suite. In general, any HIGH or CRITICAL error found in the fwts log can cause potential errors in the system and should be looked at by OEMs/ODMs.

Category

Test Item

Description

Information

acpidump

Check ACPI table acpidump.

Information

version

Gather kernel system information.

ACPI

acpitables

ACPI table settings confidence checks.

ACPI

apicinstance

Check for single instance of APIC/MADT table.

ACPI

hpet_check

High Precision Event Timer configuration test.

ACPI

mcfg

MCFG PCI Express* memory mapped config space.

ACPI

method

ACPI DSDT Method Semantic Tests.

CPU

mpcheck

Check Multi Processor tables.

CPU

msr

CPU MSR consistency check.

CPU

mtrr

MTRR validation.

System

apicedge

APIC Edge/Level Check.

System

klog

Scan kernel log for errors and warnings.