Internet of Things Certified Hardware Coverage for Ubuntu Core 24 / Ubuntu 24.04

Introduction

This document lists the coverage for certification of Internet of Things (IoT) devices with Ubuntu images. IoT devices can be certified with the following image types:

  • Ubuntu Core 24

  • Ubuntu Server 24.04 LTS

  • Ubuntu Desktop 24.04 LTS

The guide applies to devices submitted to Canonical through one of the following programmes:

  • IoT Devices Enablement Programme with Certification

  • IoT ODM Partner Programme

For each test job, one of the following certification statuses is specified:

Blocking

Features that are required for certification. If any of the blocking tests fails, the certification will fail.

Non-blocking

Features that are tested but not mandatory for certification. Failure in non-blocking tests will not prevent certification. However, a note will be added to the certificate to inform potential customers or users.

Note

Only categories of hardware are tested and not specific types of hardware. For example, tests are run to verify USB controllers work, but the type of peripheral(s) used during those tests are not specified.

Coverage is flexible based on customer requirements (for example, if a device’s use cases don’t require LEDs, then LEDs can be non-blocking)

Full test descriptions can be found in Canonical certification site for partners:

https://certification.canonical.com

client-cert-iot-ubuntucore-24

Note

The certification tests presented in this document are validated by Checkbox version 4.2.0.dev76.

Blocking

Advanced Configuration and Power Interface

The following test units are covered in this category:

acpi/oem_osi

Test ACPI OEM _OSI strings

Category ID:

acpi

Status:

Blocking

Purpose:

This checks if the deprecated OEM _OSI strings are still used by checking the ACPI DSDT and SSDT tables

User:

root

Plugin:

shell

Audio tests

The following test units are covered in this category:

audio/alsa-loopback-automated

Captured sound matches played one (automated)

Category ID:

audio

Status:

Blocking

Purpose:

Check if sound that is ‘hearable’ by capture device. This test is intended mostly for devices without internal speakers/microphones, but with a line-in and a line-out ports to which a loopback cable can be connected.

After-suspend:

True

Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH, LD_LIBRARY_PATH

User:

root

Plugin:

shell

audio/alsa-playback

Playback works

Category ID:

audio

Status:

Blocking

Purpose:

Check if sound is played through ALSA on the default output

Steps:
  1. Make sure speakers or headphones are connected to the device 2. Commence the test

Verification:

Did you hear the sound?

After-suspend:

True

Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH

User:

root

Plugin:

user-interact-verify

audio/detect-capture-devices

Check that at least one audio capture device exists

Category ID:

audio

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

audio/detect-playback-devices

Check that at least one audio playback device exists

Category ID:

audio

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

Bluetooth - BlueZ Self Tests

The following test units are covered in this category:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test

BlueZ-{bluez-internal-bnep-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

After-suspend:

True

From template:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test

Template resource:

bluez-internal-bnep-tests

bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test

BlueZ-{bluez-internal-hci-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

Description:

Runs a specific test from the hci test suite

After-suspend:

True

From template:

bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test

Template resource:

bluez-internal-hci-tests

bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test

BlueZ-{bluez-internal-rfcomm-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

After-suspend:

True

From template:

bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test

Template resource:

bluez-internal-rfcomm-tests

bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test

BlueZ-{bluez-internal-uc-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

After-suspend:

True

From template:

bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test

Template resource:

bluez-internal-uc-tests

Bluetooth tests

The following test units are covered in this category:

bluetooth/audio-a2dp

Verify Bluetooth device’s High Fidelity Playback (A2DP) capability.

Category ID:

bluetooth

Status:

Blocking

Purpose:

This test will check the High Fidelity Playback (A2DP) capability of your Bluetooth device, to see if you can hear audio from it.

Steps:
  1. Enable and pair the Bluetooth headset 2. Click “Test” to play a brief tone on your Bluetooth device, if it failed to set the Mode to A2DP, please select the device and change it manually in the “Sound Settings”

Verification:

Did you hear the tone?

After-suspend:

True

Plugin:

user-interact-verify

bluetooth/audio_record_playback

Verify Bluetooth HSP/HFP profile capability for recording and playback.

Category ID:

bluetooth

Status:

Blocking

Purpose:

This test will check the Headset Head Unit (HSP/HFP) capability of your Bluetooth device, to check if you can record sounds.

Steps:
  1. Enable and pair the bluetooth headset. 2. Click “Test”, then speak into your Bluetooth microphone. 3. After a few seconds, your speech will be played back to you.

Verification:

Did you hear your speech played back?

After-suspend:

True

Plugin:

user-interact-verify

bluetooth/bluetooth_obex_send

Bluetooth OBEX send

Category ID:

bluetooth

Status:

Blocking

Purpose:

This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable

After-suspend:

True

Environment variable:

BTDEVADDR, PLAINBOX_PROVIDER_DATA

Plugin:

shell

bluetooth/bluez-controller-detect

Check bluez lists a controller if rfkill detects one

Category ID:

bluetooth

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

bluetooth/detect

Make sure at least one bluetooth device is detected

Category ID:

bluetooth

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

bluetooth/detect-output

Store Bluetooth device information for reports.

Category ID:

bluetooth

Status:

Blocking

Purpose:

Automated test to store Bluetooth device information in the Checkbox report

After-suspend:

True

Plugin:

shell

bluetooth/zapper-a2dp

Check if the system can connect to a Bluetooth speaker using A2DP profile

Category ID:

bluetooth

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

ZAPPER_HOST

Plugin:

shell

bluetooth4/HOGP-keyboard

Verify HOGP keyboard functionality with Bluetooth Smart.

Category ID:

bluetooth

Status:

Blocking

Purpose:

This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart keyboard.

Steps:
  1. Enable a Bluetooth Smart keyboard, and put it into pairing mode. 2. Commence the test to do the auto-pairing, you will be asked to select the targeting keyboard from the list. 3. After it’s paired and connected, enter some text with your keyboard.

Verification:

Did the Bluetooth Smart keyboard work as expected?

After-suspend:

True

Plugin:

user-interact-verify

bluetooth4/beacon_eddystone_url_interface

Test system can get beacon EddyStone URL advertisements on the {interface} adapter

Unit type:

template

Category ID:

bluetooth

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

bluetooth4/beacon_eddystone_url_interface

Template resource:

device

Template filter:

device.category == ‘BLUETOOTH’

Camera tests

The following test units are covered in this category:

camera/multiple-resolution-images-rpi-attachment_name

Attach an image from the multiple resolution images test on rpi

Unit type:

template

Category ID:

camera

Status:

Blocking

Description:

This test will attach one of the images used for the multiple resolution images test.

From template:

camera/multiple-resolution-images-rpi-attachment_name

Template resource:

device

Template filter:

device.category == ‘MMAL’ and device.name != ‘’

camera/multiple-resolution-images-rpi_name

Webcam multiple resolution capture test for Pi Camera

Unit type:

template

Category ID:

camera

Status:

Blocking

From template:

camera/multiple-resolution-images-rpi_name

Template resource:

device

Template filter:

device.category == ‘MMAL’ and device.name != ‘’

camera/multiple-resolution-images_name

Webcam multiple resolution capture test for {product_slug}

Unit type:

template

Category ID:

camera

Status:

Blocking

After-suspend:

True

From template:

camera/multiple-resolution-images_name

Template resource:

device

Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

camera/roundtrip-qrcode_name

Test video output and camera {{ name }} by displaying and reading a QR code

Unit type:

template

Category ID:

camera

Status:

Blocking

From template:

camera/roundtrip-qrcode_name

Template resource:

device

Template filter:

device.category in (‘CAPTURE’, ‘MMAL’) and device.name != ‘’

CPU tests

The following test units are covered in this category:

cpu/arm64_vfp_support_platform

Validate that the Floating Point Unit is running on {platform} device

Unit type:

template

Category ID:

cpu

Status:

Blocking

Description:

Validate that the Floating Point Unit is running on {platform} device.

From template:

cpu/arm64_vfp_support_platform

Template resource:

cpuinfo

Template filter:

cpuinfo.platform == ‘aarch64’

cpu/armhf_vfp_support_platform

Validate that the Vector Floating Point Unit is running on {platform} device

Unit type:

template

Category ID:

cpu

Status:

Blocking

Description:

Validate that the Vector Floating Point Unit is running on {platform} device.

From template:

cpu/armhf_vfp_support_platform

Template resource:

cpuinfo

Template filter:

cpuinfo.platform == ‘armv7l’

cpu/clocktest

Tests the CPU for clock jitter

Category ID:

cpu

Status:

Blocking

Purpose:

Runs a test for clock jitter on SMP machines.

After-suspend:

True

Plugin:

shell

cpu/cstates

Run C-States tests

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Uses the Firmware Test Suite (fwts) to test the power saving states of the CPU.

Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP

User:

root

Plugin:

shell

cpu/cstates_results.log

Attach C-States test log

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Attaches the FWTS desktop diagnosis results log to the submission.

Plugin:

attachment

cpu/maxfreq_test

Test that the CPU can run at its max frequency

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.

Environment variable:

LD_LIBRARY_PATH, SNAP

User:

root

Plugin:

shell

cpu/maxfreq_test-log-attach

Attach CPU max frequency log

Category ID:

cpu

Status:

Blocking

Purpose:

Attaches the log generated by cpu/maxfreq_test to the results submission.

Plugin:

attachment

cpu/offlining_test

Test offlining of each CPU core

Category ID:

cpu

Status:

Blocking

Purpose:

Attempts to offline each core in a multicore system.

After-suspend:

True

User:

root

Plugin:

shell

cpu/scaling_test

Test the CPU scaling capabilities

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Use Firmware Test Suite (fwts cpufreq) to test the scaling capabilities of the CPU.

Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP

User:

root

Plugin:

shell

cpu/scaling_test-log-attach

Attach CPU scaling capabilities log

Category ID:

cpu

Status:

Blocking

Purpose:

Attaches the log generated by cpu/scaling_test to the results submission.

Plugin:

attachment

cpu/topology

Check CPU topology for accuracy between proc and sysfs

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.

After-suspend:

True

Plugin:

shell

Disk tests

The following test units are covered in this category:

disk/check-software-raid

Validate the configuration of software RAID devices are expected

Category ID:

disk

Status:

Blocking

Purpose:

<missing purpose>

Description:

Examine the system to detect Software RAID devices are created and the RAID mode are expected the SOFTWARE_RAID_LEVEL variable is needed for this tests. e.g. SOFTWARE_RAID_LEVEL=”raid0 raid1 raid5”

Environment variable:

SOFTWARE_RAID_LEVEL

User:

root

Plugin:

shell

disk/detect

Gathers information about each disk detected

Category ID:

disk

Status:

Blocking

Purpose:

Uses lsblk to gather information about each disk detected on the system under test.

After-suspend:

True

Plugin:

shell

disk/encryption/detect

Test that Full Disk Encryption is in use

Category ID:

disk

Status:

Blocking

Purpose:

Examine the system to detect if one of the standard full disk encryption implementations is in use

User:

root

Plugin:

shell

disk/read_performance_name

Disk performance test for {product_slug}

Unit type:

template

Category ID:

disk

Status:

Blocking

After-suspend:

True

Environment variable:

DISK_MDADM_READ_PERF, DISK_NVME_READ_PERF, DISK_READ_PERF

From template:

disk/read_performance_name

Template resource:

device

Template filter:

device.category == ‘DISK’

disk/stats_name

Disk statistics for {product_slug}

Unit type:

template

Category ID:

disk

Status:

Blocking

After-suspend:

True

From template:

disk/stats_name

Template resource:

device

Template filter:

device.category == ‘DISK’ and device.name != ‘’

disk/storage_device_name

Disk I/O stress test for {product_slug}

Unit type:

template

Category ID:

disk

Status:

Blocking

After-suspend:

True

From template:

disk/storage_device_name

Template resource:

device

Template filter:

device.category == ‘DISK’

thunderbolt3/insert

Storage insert detection on Thunderbolt 3 port

Category ID:

disk

Status:

Blocking

Purpose:

This test will check if the insertion of a Thunderbolt 3 HDD could be detected

Steps:
  1. Click ‘Test’ to begin the test. This test will timeout and fail if the insertion has not been detected within 40 seconds. 2. Plug a Thunderbolt HDD into an available Thunderbolt 3 port, if it’s not mounted automatically, please click the HDD icon to mount it.

Verification:

The verification of this test is automated. Do not change the automatically selected result

After-suspend:

True

User:

root

Plugin:

user-interact

thunderbolt3/remove

Storage removal detection on Thunderbolt 3 port

Category ID:

disk

Status:

Blocking

Purpose:

This test will check if the system can detect the removal of a Thunderbolt 3 HDD

Steps:
  1. Click ‘Test’ to begin the test. This test will timeout and fail if the removal has not been detected within 20 seconds. 2. Remove the previously attached Thunderbolt HDD from the Thunderbolt port.

Verification:

The verification of this test is automated. Do not change the automatically selected result

After-suspend:

True

User:

root

Plugin:

user-interact

thunderbolt3/storage-test

Storage test on Thunderbolt 3

Category ID:

disk

Status:

Blocking

Purpose:

This is an automated test which performs read/write operations on an attached Thunderbolt HDD

After-suspend:

True

User:

root

Plugin:

shell

Docker containers

The following test units are covered in this category:

docker/build-single_arch

Test docker build with a single container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/build-single_arch

Template resource:

docker_resource

docker/commit_arch

Test docker commit a change to a single container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/commit_arch

Template resource:

docker_resource

docker/compose-and-basic_arch

Test docker compose and basic command

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/compose-and-basic_arch

Template resource:

docker_resource

docker/compose-restart_arch

Test compose a container with restart policy applied

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/compose-restart_arch

Template resource:

docker_resource

docker/compose-single_arch

Test docker compose with a single container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/compose-single_arch

Template resource:

docker_resource

docker/copy_arch

Test copy a file bwtween a container and local filesystem

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/copy_arch

Template resource:

docker_resource

docker/deploy-registry_arch

Deploy a registry server and run it on localhost

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/deploy-registry_arch

Template resource:

docker_resource

docker/diff_arch

Test changes to files in Ubuntu container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/diff_arch

Template resource:

docker_resource

docker/export-and-import_arch

Test docker import and export a docker container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/export-and-import_arch

Template resource:

docker_resource

docker/info

Display system-wide information about docker

Category ID:

docker

Status:

Blocking

Purpose:

<missing purpose>

Steps:

<missing steps>

Verification:

<missing verification>

Description:

<missing description>

User:

root

docker/inspect_arch

Test query low-level information on a docker object

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/inspect_arch

Template resource:

docker_resource

docker/interative_arch

Test an interactive shell in Ubuntu container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/interative_arch

Template resource:

docker_resource

docker/kill_arch

Test killing containers

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/kill_arch

Template resource:

docker_resource

docker/restart-always_arch

Test container restart policy with always applied

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/restart-always_arch

Template resource:

docker_resource

docker/restart-on-failure_arch

Test container restart policy with on_failure applied

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/restart-on-failure_arch

Template resource:

docker_resource

docker/run_arch

Download and run ubuntu container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/run_arch

Template resource:

docker_resource

docker/save-and-load_arch

Test docker save and load a docker image

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/save-and-load_arch

Template resource:

docker_resource

docker/start-stop_arch

Start and stop a single container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/start-stop_arch

Template resource:

docker_resource

docker/update_arch

Test update configuration of one container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/update_arch

Template resource:

docker_resource

docker/version

Display docker version information

Category ID:

docker

Status:

Blocking

Purpose:

<missing purpose>

Steps:

<missing steps>

Verification:

<missing verification>

Description:

<missing description>

User:

root

Eeprom

The following test units are covered in this category:

eeprom/read-write

Test EEPROM read and write functions for the system.

Category ID:

eeprom

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

User:

root

Plugin:

shell

Error Detection And Correction (EDAC) Memory Controllers

The following test units are covered in this category:

edac/default-report

Attach the default report from EDAC util

Category ID:

edac

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

attachment

edac/detect

Detect if the EDAC drivers are loaded and Memory Controllers are found

Category ID:

edac

Status:

Blocking

Purpose:

<missing purpose>

Steps:

<missing steps>

Verification:

<missing verification>

Description:

<missing description>

Ethernet Device tests

The following test units are covered in this category:

ethernet/detect

Detect if at least one ethernet device is detected

Category ID:

ethernet

Status:

Blocking

Purpose:

Test to detect and return information about available network controllers on the system under test.

After-suspend:

True

Plugin:

shell

ethernet/hotplug-interface

Ensure hotplugging works on port {{ interface }}

Unit type:

template

Category ID:

ethernet

Status:

Blocking

After-suspend:

True

From template:

ethernet/hotplug-interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’

ethernet/ping_interface

Can ping another machine over Ethernet port {interface}

Unit type:

template

Category ID:

ethernet

Status:

Blocking

After-suspend:

True

From template:

ethernet/ping_interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’

ethernet/wol_S3_interface

Wake on LAN (WOL) test from S3 - {interface}

Unit type:

template

Category ID:

ethernet

Status:

Blocking

From template:

ethernet/wol_S3_interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’ and device.interface != ‘UNKNOWN’

ethernet/wol_S4_interface

Wake on LAN (WOL) test from S4 - {interface}

Unit type:

template

Category ID:

ethernet

Status:

Blocking

From template:

ethernet/wol_S4_interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’

ethernet/wol_S5_interface

Wake on LAN (WOL) test from S5 - {interface}

Unit type:

template

Category ID:

ethernet

Status:

Blocking

From template:

ethernet/wol_S5_interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’

Firmware tests

The following test units are covered in this category:

firmware/fwts_desktop_diagnosis

Run FWTS QA-concerned desktop-specific diagnosis tests.

Category ID:

firmware

Status:

Blocking

Purpose:

<missing purpose>

Description:

Run Firmware Test Suite (fwts) QA-concerned desktop-specific diagnosis tests.

Environment variable:

PLAINBOX_SESSION_SHARE

User:

root

Plugin:

shell

firmware/fwts_desktop_diagnosis_results.log.gz

Attach FWTS desktop diagnosis log to submission

Category ID:

firmware

Status:

Blocking

Purpose:

<missing purpose>

Description:

Attaches the FWTS desktop diagnosis results log to the submission

Plugin:

attachment

Gathers information about the DUT

The following test units are covered in this category:

connections

Collect information about connections

Category ID:

information_gathering

Status:

Blocking

Purpose:

<missing purpose>

Description:

Generates a list of plug and slot connections on the device

Plugin:

resource

rtc

Creates resource info for RTC

Category ID:

information_gathering

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

resource

serial_assertion

Collect serial assertions on the device

Category ID:

information_gathering

Status:

Blocking

Purpose:

<missing purpose>

Description:

Queries the snapd REST API for serial assertions present on the device.

Plugin:

resource

sleep

Create resource info for supported sleep states

Category ID:

information_gathering

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

resource

snap

Collect information about installed snap packages

Category ID:

information_gathering

Status:

Blocking

Purpose:

<missing purpose>

Description:

Generates a list of snap packages

Plugin:

resource

General Purpose I/O

The following test units are covered in this category:

gpio/gpiomem_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via /dev/gpiomem

Unit type:

template

Category ID:

gpio

Status:

Blocking

After-suspend:

True

From template:

gpio/gpiomem_loopback_pairs_model

Template resource:

com.canonical.certification::model_assertion

Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)

gpio/sysfs_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template

Category ID:

gpio

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

gpio/sysfs_loopback_pairs_model

Template resource:

com.canonical.certification::model_assertion

Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)

gpio/sysfs_loopback_pairs_vendor_product

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template

Category ID:

gpio

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

gpio/sysfs_loopback_pairs_vendor_product

Template resource:

dmi

Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’, ‘UPN-EHL01’)

Image verification tests

The following test units are covered in this category:

image/model-grade

Check that the model grade is correctly set

Category ID:

image

Status:

Blocking

Purpose:

Images with the ‘dangerous’ grade (the lowest of all available grades) results in certain security measures being relaxed. Images that require strict security-related implementations must have the model grade set to non-dangerous grades - either the highest grade of ‘secured’, or a grade passed to Checkbox for checking via the MODEL_GRADE configuration variable.

Plugin:

shell

Informational tests

The following test units are covered in this category:

info/systemd-analyze

System boot-up performance statistics

Category ID:

info

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

info/systemd-analyze-critical-chain

Print the tree of the time-critical chain of SystemD

Category ID:

info

Status:

Blocking

Purpose:

This job prints a tree of the time-critical chain of SystemD units.

Plugin:

shell

lspci_attachment

Attach a list of PCI devices

Category ID:

info

Status:

Blocking

Purpose:

Attaches very verbose lspci output.

Plugin:

attachment

lsusb_attachment

Attach output of lsusb

Category ID:

info

Status:

Blocking

Purpose:

Attaches a list of detected USB devices.

After-suspend:

True

User:

root

Plugin:

attachment

net_if_management_attachment

Collect logging from the net_if_management job

Category ID:

info

Status:

Blocking

Purpose:

<missing purpose>

Description:

Allows logging of debug info and errors by the associated resource job

Plugin:

attachment

parts_meta_info_attachment

Attaches an information about all parts that constituted this snap

Category ID:

info

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

CHECKBOX_RUNTIME, SNAP

User:

root

Plugin:

attachment

Input Devices tests

The following test units are covered in this category:

input/zapper-keyboard

Check if the system recognizes keyboard events from Zapper

Category ID:

input

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

ZAPPER_HOST

User:

root

Plugin:

shell

Intel Integrated Sensor Hub Transport Protocol (ISHTP)

The following test units are covered in this category:

eclite/module-detect

Verifies that the ishtp_eclite module is loaded

Category ID:

intel-ishtp

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

eclite/temperature-reading-from-thermal-acpitz

Read the temperature of acpitz to ensure Eclite is functional

Category ID:

intel-ishtp

Status:

Blocking

Purpose:

For testing the feature of Eclite, one way is to monitor the temperature of acpitz thermal.

User:

root

Plugin:

shell

ishtp/device-detect

Verify that at least 1 device entry exists in /sys/bus/ishtp/devices

Category ID:

intel-ishtp

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

ishtp/module-detect

Verifies that the intel_ish_ipc module for ISHTP is loaded

Category ID:

intel-ishtp

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

Intel Management Engine Interface (MEI)

The following test units are covered in this category:

mei/check-device

Detect if the Intel MEI device is available

Category ID:

mei

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

mei/check-module

Detect if the Intel MEI kernel module is loaded

Category ID:

mei

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

mei/get-firmware-version

Retrieve MEI firmware version by MEI interface

Category ID:

mei

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

I²C (Inter-Integrated Circuit)

The following test units are covered in this category:

i2c/i2c-bus-detect

Check presence of an I²C bus

Category ID:

i2c

Status:

Blocking

Purpose:

If an expected number of I²C buses is provided, the job will verify the detected number is correct. If the expected number of buses is not provided, the job will pass if at least one I²C bus is detected.

Environment variable:

I2C_BUS_NUMBER

User:

root

Plugin:

shell

i2c/i2c-device-detect

Check if any I²C devices can be detected

Category ID:

i2c

Status:

Blocking

Purpose:

The test will pass if there’s at least one I²C device detected on any I²C bus.

User:

root

Plugin:

shell

Kernel snap tests

The following test units are covered in this category:

kernel-snap/booted-kernel-matches-current-name

The booted kernel image matches the image in the current kernel snap

Unit type:

template

Category ID:

kernel-snap

Status:

Blocking

From template:

kernel-snap/booted-kernel-matches-current-name

Template resource:

bootloader

Template filter:

bootloader.booted_kernel_path != ‘unknown’

LED tests

The following test units are covered in this category:

led-indicator/gpio-controller-leds-name

Check control of {name} LED indicator.

Unit type:

template

Category ID:

led

Status:

Blocking

After-suspend:

True

From template:

led-indicator/gpio-controller-leds-name

Template resource:

led-indicator/gpio-controller-leds

led-indicator/gpio-leds-name

Check control of {name} LED indicator.

Unit type:

template

Category ID:

led

Status:

Blocking

After-suspend:

True

From template:

led-indicator/gpio-leds-name

Template resource:

led-indicator/gpio-leds

led-indicator/sysfs-leds-manual

Check control of {name} LED.

Unit type:

template

Category ID:

led

Status:

Blocking

After-suspend:

True

From template:

led-indicator/sysfs-leds-manual

Template resource:

led-indicator/sysfs-leds

led/fn

Test the Fn key LED functionality by activating/deactivating the Fn keys locking.

Category ID:

led

Status:

Blocking

Purpose:

This test will test the Fn key LED.

Steps:
  1. Press the Fn+Fn Lock key to activate/deactivate Fn keys locking. 2. The Fn key LED should be switched on/off every time the key is pressed.

Verification:

Did the Fn key LED light as expected?

After-suspend:

True

Plugin:

manual

led/power

Power LED behavior when powered

Category ID:

led

Status:

Blocking

Purpose:

Check power led is on when system is powered on

Steps:
  1. Check power led when system is powered on

Verification:

Power led is on when system is powered on

After-suspend:

True

Plugin:

manual

led/serial

Serial ports LED behavior

Category ID:

led

Status:

Blocking

Purpose:

Check serial ports LED behavior is correct

Steps:
  1. Start the test to send data to all serial ports (/dev/ttyS*)

Verification:

All serial ports LED are on for a few seconds (3-4s)

User:

root

Plugin:

user-interact-verify

led/sysfs_led_brightness_off_vendor_product

Ensure the leds_aaeon driver properly sets all LEDs to off or minimum brightness by running a test.

Unit type:

template

Category ID:

led

Status:

Blocking

After-suspend:

True

From template:

led/sysfs_led_brightness_off_vendor_product

Template resource:

dmi

Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’)

led/sysfs_led_brightness_on_vendor_product

Verify the functionality of the leds_aaeon driver by ensuring all external LEDs achieve maximum brightness.

Unit type:

template

Category ID:

led

Status:

Blocking

After-suspend:

True

From template:

led/sysfs_led_brightness_on_vendor_product

Template resource:

dmi

Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’)

led/wireless

Verify the WLAN/Bluetooth LED functionality by toggling wireless connections.

Category ID:

led

Status:

Blocking

Purpose:

Wireless (WLAN + Bluetooth) LED verification

Steps:
  1. Make sure WLAN connection is established and Bluetooth is enabled. 2. WLAN/Bluetooth LED should light 3. Switch WLAN and Bluetooth off from a hardware switch (if present) 4. Switch them back on 5. Switch WLAN and Bluetooth off from the panel applet 6. Switch them back on

Verification:

Did the WLAN/Bluetooth LED light as expected?

Plugin:

manual

Media Card tests

The following test units are covered in this category:

mediacard/cf-insert

Verify the system’s media card reader can detect a Compact Flash card insertion.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a Compact Flash (CF) media card

Steps:
  1. Commence the test and then insert a CF card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/cf-remove

Ensure the system detects CF card removal from the card reader correctly.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a CF card from the system’s card reader.

Steps:
  1. Commence the test and then remove the CF card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/cf-storage

Automate testing for reading and writing to the CF card after mediacard/cf-insert test.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/cf-insert test is run. It tests reading and writing to the CF card.

User:

root

Plugin:

shell

mediacard/mmc-insert

Test the system’s media card reader for MMC insertion detection.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a Multimedia Card (MMC) media

Steps:
  1. Commence the test and then insert an MMC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/mmc-remove

Test the detection of MMC card removal from the system’s card reader.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of the MMC card from the system’s card reader.

Steps:
  1. Commence the test and then remove the MMC card from the reader. (Note: this test will time out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/mmc-storage

Test automated execution for reading and writing to the MMC card.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/mmc-insert test is run. It tests reading and writing to the MMC card.

User:

root

Plugin:

shell

mediacard/ms-insert

Verify the detection of Memory Stick (MS) card insertion by the system’s media card reader.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a Memory Stick (MS) media card

Steps:
  1. Commence the test and then insert a MS card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/ms-remove

Test if the system detects the removal of an MS card correctly.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of an MS card from the system’s card reader.

Steps:
  1. Commence the test and then remove the MS card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/ms-storage

Automated test for reading and writing to the MS card after mediacard/ms-insert test.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/ms-insert test is run. It tests reading and writing to the MS card.

User:

root

Plugin:

shell

mediacard/msp-insert

Verify the system’s media card reader can detect a Memory Stick Pro (MSP) card insertion.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a Memory Stick Pro (MSP) media card

Steps:
  1. Commence the test and then insert a MSP card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/msp-remove

Ensure MSP card removal is correctly detected by the system.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a MSP card from the system’s card reader.

Steps:
  1. Commence the test and remove the MSP card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/msp-storage

Automated test for reading and writing to the MSP card after mediacard/msp-insert test.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/msp-insert test is run. It tests reading and writing to the MSP card.

User:

root

Plugin:

shell

mediacard/sd-insert

Test that insertion of an SD card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of an UNLOCKED Secure Digital (SD) media card

Steps:
  1. Commence the test and then insert an UNLOCKED SD card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/sd-remove

Test that removal of an SD card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of an SD card from the system’s card reader.

Steps:
  1. Commence the test and then remove the SD card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/sd-storage

Test reading & writing to an SD Card

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/sd-insert test is run. It tests reading and writing to the SD card.

User:

root

Plugin:

shell

mediacard/sdhc-insert

Test that insertion of an SDHC card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a UNLOCKED Secure Digital High-Capacity (SDHC) media card

Steps:
  1. Commence the test and then insert an UNLOCKED SDHC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

mediacard/sdhc-remove

Test that removal of an SDHC card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of an SDHC card from the system’s card reader.

Steps:
  1. Commence the test and then remove the SDHC card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

mediacard/sdhc-storage

Test reading & writing to a SDHC Card

Category ID:

mediacard

Status:

Blocking

Purpose:

<missing purpose>

Description:

This test is automated and executes after the mediacard/sdhc-insert test is run. It tests reading and writing to the SDHC card.

After-suspend:

True

User:

root

Plugin:

shell

mediacard/sdxc-insert

Test that insertion of an SDXC card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a Secure Digital Extended Capacity (SDXC) media card

Steps:
  1. Commence the test and then insert an UNLOCKED SDXC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/sdxc-remove

Test that removal of an SDXC card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a SDXC card from the system’s card reader.

Steps:
  1. Commence the test and then remove the SDXC card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/sdxc-storage

Test reading & writing to an SDXC Card

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/sdxc-insert test is run. It tests reading and writing to the SDXC card.

User:

root

Plugin:

shell

mediacard/xd-insert

Ensure the system’s media card reader detects the insertion of an xD media card.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of an Extreme Digital (xD) media card.

Steps:
  1. Commence the test and then insert an xD card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/xd-remove

Automated test for detecting the removal of a xD card from the system’s card reader.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a xD card from the system’s card reader.

Steps:
  1. Commence the test and then remove the xD card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/xd-storage

Automated test to verify reading and writing functionality of the xD card after mediacard/xd-insert test.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/xd-insert test is run. It tests reading and writing to the xD card.

User:

root

Plugin:

shell

Memory tests

The following test units are covered in this category:

memory/info

Check the amount of memory reported by meminfo against DMI

Category ID:

memory

Status:

Blocking

Purpose:

This test checks the amount of memory which is reported in meminfo against the size of the memory modules detected by DMI.

User:

root

Plugin:

shell

Miscellaneous tests

The following test units are covered in this category:

miscellanea/device_check

Device Check

Category ID:

miscellanea

Status:

Blocking

Purpose:

Device check

Steps:
  1. Commence the test 2. Compare items on System Manifest to the devices known to udev

Verification:

Do the devices reported by udev match the devices on the Manifest?

Plugin:

user-interact-verify

miscellanea/secure_boot_mode_gadget

Test that {gadget} Ubuntu Core system booted with Secure Boot active

Unit type:

template

Category ID:

miscellanea

Status:

Blocking

From template:

miscellanea/secure_boot_mode_gadget

Template resource:

model_assertion

miscellanea/submission-resources

Check that data for a complete result are present

Category ID:

miscellanea

Status:

Blocking

Purpose:

<missing purpose>

Description:

A meta-job that verifies the data necessary for a complete result submission are present. Failure indicates that the results are incomplete and may be rejected.

Plugin:

shell

Monitor tests

The following test units are covered in this category:

monitor/displayport_hotplug

Can hotplug monitor (DisplayPort)

Category ID:

monitor

Status:

Blocking

Purpose:

This test will check the DisplayPort port and the ability to do hotplugging.

Steps:

Skip this test if your system does not have a DisplayPort port. 1. If a display is already connected, unplug it. 2. (Re-)Connect a display to the DisplayPort port on your system

Verification:

Was the interface displayed correctly on the screen?

After-suspend:

True

Plugin:

manual

monitor/dvi

Monitor works (DVI)

Category ID:

monitor

Status:

Blocking

Purpose:

Check output to display through DVI port

Steps:
  1. Connect display to DVI port 2. Check the display

Verification:

Output to display works

After-suspend:

True

Plugin:

manual

monitor/dvi-to-vga

Monitor works (DVI-to-VGA)

Category ID:

monitor

Status:

Blocking

Purpose:

Check output to display through VGA adaptor on DVI port

Steps:
  1. Connect display to VGA adaptor on DVI port 2. Check the display

Verification:

Output to display works

After-suspend:

True

Plugin:

manual

monitor/hdmi

Monitor works (HDMI)

Category ID:

monitor

Status:

Blocking

Purpose:

Check output to display through HDMI port

Steps:
  1. Connect display to HDMI port 2. Check the display

Verification:

Output to display works

After-suspend:

True

Plugin:

manual

monitor/hdmi-to-vga

Monitor works (HDMI-to-VGA)

Category ID:

monitor

Status:

Blocking

Purpose:

Check output to display through VGA adaptor on HDMI port

Steps:
  1. Connect display to VGA adaptor on HDMI port 2. Check the display

Verification:

Output to display works

After-suspend:

True

Plugin:

manual

monitor/vga

Monitor works (VGA)

Category ID:

monitor

Status:

Blocking

Purpose:

Check output to display through VGA port

Steps:
  1. Connect display to VGA port 2. Check the display

Verification:

Output to display works

After-suspend:

True

Plugin:

manual

monitor/zapper-edid

Check if the system automatically changes the resolution based on EDID

Category ID:

monitor

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

ZAPPER_HOST

Plugin:

shell

Non-device specific networking tests

The following test units are covered in this category:

ipv6_detect

Test if the kernel is IPv6 ready

Category ID:

networking

Status:

Blocking

Purpose:

Test if the running kernel supports IPv6.

Steps:

<missing steps>

Verification:

<missing verification>

After-suspend:

True

networking/info_device__index___interface

Network Information of device {__index__} ({interface})

Unit type:

template

Category ID:

networking

Status:

Blocking

After-suspend:

True

From template:

networking/info_device__index___interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’

networking/predictable_names

Verify that all network interfaces have predictable names.

Category ID:

networking

Status:

Blocking

Purpose:

Verify that all network interfaces have predictable names.

Plugin:

shell

Power Management tests

The following test units are covered in this category:

power-management/cold-reboot

Cold reboot

Category ID:

power-management

Status:

Blocking

Purpose:

This test powers off the system and then powers it on using RTC

Environment variable:

COLD_REBOOT_DELAY, RTC_DEVICE_FILE

User:

root

Plugin:

shell

power-management/post-cold-reboot

Post cold reboot service check

Category ID:

power-management

Status:

Blocking

Purpose:

Check there are no failed services after the cold reboot

Plugin:

shell

power-management/post-warm-reboot

Post warm reboot service check

Category ID:

power-management

Status:

Blocking

Purpose:

Check there are no failed services after the warm reboot

Plugin:

shell

power-management/warm-reboot

Warm reboot

Category ID:

power-management

Status:

Blocking

Purpose:

This tests reboots the system using the reboot command

User:

root

Plugin:

shell

watchdog/detect

Detect the presence of a Hardware Watchdog

Category ID:

power-management

Status:

Blocking

Purpose:

<missing purpose>

Steps:

<missing steps>

Verification:

<missing verification>

Description:

<missing description>

watchdog/post-trigger-system-reset-auto

Post watchdog reset service check

Category ID:

power-management

Status:

Blocking

Purpose:

Check there are no failed services after the watchdog triggered

Plugin:

shell

watchdog/systemd-config

Check if the hardware watchdog is properly configured

Category ID:

power-management

Status:

Blocking

Purpose:

<missing purpose>

Steps:

<missing steps>

Verification:

<missing verification>

Description:

<missing description>

watchdog/trigger-system-reset-auto

Test that the watchdog module can trigger a system reset

Category ID:

power-management

Status:

Blocking

Purpose:

Ensure that the watchdog module can successfully initiate a system reset.

User:

root

Plugin:

shell

Quadrature Encoder Peripherals

The following test units are covered in this category:

qep/qep-device-driver-for-qep-device

Verify PCI Device {qep-device} is using the correct driver

Unit type:

template

Category ID:

qep

Status:

Blocking

Description:

Checks that the device exists in lspci and that the device driver associated is correct. Device ID should be 8086:{qep-device} and the driver is always intel_qep.

From template:

qep/qep-device-driver-for-qep-device

Template resource:

qep/qep-devices

qep/qep-device-node-for-qep-device

Verify device directory exists for {qep-device}

Unit type:

template

Category ID:

qep

Status:

Blocking

From template:

qep/qep-device-node-for-qep-device

Template resource:

qep/qep-devices

Real Time Clock (RTC)

The following test units are covered in this category:

rtc/battery

RTC battery tracks the time and ensures the system can wake up from power off state.

Category ID:

rtc_category

Status:

Blocking

Purpose:

RTC battery backup power can send system wakeup events.

Steps:
  1. Start the test to power off the system (wakeup scheduled in 30s).

Verification:

RTC can wake up the system successfully. environ: RTC_DEVICE_FILE

User:

root

Plugin:

user-interact

rtc/rtc_alarm_rtc

Check that RTC alarm of {rtc} works

Unit type:

template

Category ID:

rtc_category

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

rtc/rtc_alarm_rtc

Template resource:

rtc_list

rtc/rtc_clock_rtc

Check that {rtc} clock is synchronized with system clock

Unit type:

template

Category ID:

rtc_category

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

rtc/rtc_clock_rtc

Template resource:

rtc_list

rtc/rtc_number

Check the number of RTC as expected.

Category ID:

rtc_category

Status:

Blocking

Purpose:

Check the number of RTC on platform as expected.

After-suspend:

True

Environment variable:

TOTAL_RTC_NUM

User:

root

Plugin:

shell

Serial Port

The following test units are covered in this category:

serial/loopback-dev

Serial loopback test of {dev}

Unit type:

template

Category ID:

serial

Status:

Blocking

After-suspend:

True

From template:

serial/loopback-dev

Template resource:

serial_ports_static

serial/rs232-console

Serial debugging console is enabled and operational

Category ID:

serial

Status:

Blocking

Purpose:

Check user can log into system through serial port from another machine

Steps:
  1. Connect USB to DB9 null modem adapter cable to the serial port of the test machine. 2. Connect the cable to a USB port of another Ubuntu machine (client). 3. Install screen on the client (if not done in Prerequisite). 4. Execute the following command on the client: sudo screen /dev/ttyUSB0 5. Start the getty service on the test machine: sudo systemctl start getty@[rs232 device, e.g., /dev/ttyS0].service 6. Log into the test machine from the terminal on the client.

Verification:
  1. The output to the client is fine after the getty service is started. 2. Log into the test machine from the terminal on the client successfully.

After-suspend:

True

Plugin:

manual

Snapd

The following test units are covered in this category:

snappy/os-refresh

Refresh the system using the snap tool

Category ID:

snapd

Status:

Blocking

Purpose:

Check “core” can be refreshed by snap refresh

Steps:
  1. Check version number snap list core 2. Update snap refresh core –edge 3. Reboot the system and log in sudo reboot 4. Check version number snap list core

Verification:

Check core version is updated using the edge channel

Plugin:

manual

snappy/os-revert

Rollback system update using the snap tool

Category ID:

snapd

Status:

Blocking

Purpose:

Check core can be reverted by snap revert

Steps:
  1. Check version number (Note the version number) snap list core 2. Revert snap revert core 3. Reboot the system and log in sudo reboot 4. Check version number snap list core 5. Remove reverted version (and associated data) to make system clean snap remove core –revision=<edge_revision>

Verification:

Check core version at step 4 is back to its stable version

Plugin:

manual

snappy/snap-install

Test the snap install command is working

Category ID:

snapd

Status:

Blocking

Purpose:

The store should contain the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap. The test makes sure this can be downloaded and installed on the system.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/snap-list

Test that the snap list command is working.

Category ID:

snapd

Status:

Blocking

Purpose:

If snap list command is working then should at least find the ubuntu-core package.

Plugin:

shell

snappy/snap-refresh-automated

Test whether the snap refresh command is working.

Category ID:

snapd

Status:

Blocking

Purpose:

The test will install the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap from the stable channel and then refreshes it to the edge channel and compares the revision before and after the refresh.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/snap-remove

Test the snap remove command is working.

Category ID:

snapd

Status:

Blocking

Purpose:

After having installed the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap, check it can be removed.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/snap-reupdate-automated

Test the snap refresh command works after blacklisting.

Category ID:

snapd

Status:

Blocking

Purpose:

Checks that the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap can be refreshed after removal of the blacklisted revision.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/snap-revert-automated

Test the snap revert command is working.

Category ID:

snapd

Status:

Blocking

Purpose:

Checks if the edge channel {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap is reverted back to the one from stable.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/test-snaps-confinement

Test all the snaps’ confinement

Category ID:

snapd

Status:

Blocking

Purpose:

Test all the snaps’ confinement, devmode, revision. Make sure the confinement is “strict”, devmode is “False”, and revision should not be sideloaded.

Plugin:

shell

snappy/test-store-config-store

Test that image is using the correct snappy store configuration.

Unit type:

template

Category ID:

snapd

Status:

Blocking

Environment variable:

STORE_ID

From template:

snappy/test-store-config-store

Template resource:

com.canonical.certification::model_assertion

snappy/test-store-install-beta

Snappy install command - beta channel store

Category ID:

snapd

Status:

Blocking

Purpose:

Test the snappy install command is able to install and remove snap in beta channel store.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/test-store-install-edge

Snappy install command - edge channel store

Category ID:

snapd

Status:

Blocking

Purpose:

Test the snappy install command is able to install a snap in the edge channel store.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/test-system-confinement

Test if the system confinement is strict

Category ID:

snapd

Status:

Blocking

Purpose:

Test if the system confinement is “strict” If not, list the missing features

Plugin:

shell

SocketCAN interface tests

The following test units are covered in this category:

socketcan/send_packet_local_eff_virtual

Virtual CAN device support test (Local test with raw socket and EFF)

Category ID:

socketcan

Status:

Blocking

Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device

After-suspend:

True

User:

root

Plugin:

shell

socketcan/send_packet_local_eff_interface

CAN device support test for {interface} (Raw, Local, EFF)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_local_eff_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

socketcan/send_packet_local_fd_virtual

Virtual CAN device support test (Raw, Local, FD)

Category ID:

socketcan

Status:

Blocking

Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket; this is only a local test as the broadcast packet is received on the same device.

After-suspend:

True

User:

root

Plugin:

shell

socketcan/send_packet_local_fd_interface

CAN device support test for {interface} (Raw, Local, FD)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_local_fd_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

socketcan/send_packet_local_sff_virtual

Virtual CAN device support test (Raw, Local)

Category ID:

socketcan

Status:

Blocking

Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device

After-suspend:

True

User:

root

Plugin:

shell

socketcan/send_packet_local_sff_interface

CAN device support test for {interface} (Raw, Local)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_local_sff_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

socketcan/send_packet_remote_eff_interface

CAN device support test {interface} (Raw, Remote, EFF)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_remote_eff_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

socketcan/send_packet_remote_fd_interface

CAN device support test {interface} (Raw, Remote, FD)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_remote_fd_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

socketcan/send_packet_remote_sff_interface

CAN device support test {interface} (Raw, Remote)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_remote_sff_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

TPM 2.0 (Trusted Platform Module)

The following test units are covered in this category:

clevis-encrypt-tpm2/detect-ecc-capabilities

Ensure the TPM has required capabilities for clevis ECC test

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

clevis-encrypt-tpm2/detect-rsa-capabilities

Ensure the TPM has required capabilities for clevis RSA test

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

clevis-encrypt-tpm2/ecc

clevis encrypt/decrypt key ecc

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

clevis-encrypt-tpm2/precheck

clevis encrypt/decrypt precheck

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

clevis-encrypt-tpm2/rsa

clevis encrypt/decrypt key rsa

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

tpm2/fwts-event-log-dump

Dump the contents of the TPM Event Log

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

The information in the TPM Event Log can be useful in debugging problems with TPM command support and adherance to standards. This can be of particular help when checking whether a device can support Ubuntu Core Full Disk Encryption.

User:

root

Plugin:

attachment

Ubuntu Core OS feature tests

The following test units are covered in this category:

ubuntucore/os-fail-boot-description

Automatically rollback after failed boot after upgrade

Unit type:

template

Category ID:

ubuntucore

Status:

Blocking

From template:

ubuntucore/os-fail-boot-description

Template resource:

lsb

Template filter:

lsb.distributor_id == ‘Ubuntu Core’

ubuntucore/os-recovery-mode

Reboot into recovery mode and log into the system using prior credentials.

Category ID:

ubuntucore

Status:

Blocking

Purpose:

Check if the system will reboot to recovery mode successfully

Steps:
  1. Reboot the system to recovery mode $ sudo snap reboot –recover 2. The system should respond to the command and reboot itself 3. Wait until the system completes the reboot process 4. Check if the system is running in recovery mode $ cat /proc/cmdline 5. Reboot the system back to normal run mode $ sudo snap reboot 6. Check again if the system is running in run mode $ cat /proc/cmdline

Verification:

Check if kernel cmdline when the system is in recovery mode includes: ‘snapd_recovery_mode=recover’ Check if kernel cmdline when the system is in normal run mode includes: ‘snapd_recovery_mode=run’

Plugin:

manual

ubuntucore/os-reinstall-mode

Reboot into reinstall mode and trigger a factory reset on the device.

Category ID:

ubuntucore

Status:

Blocking

Purpose:

Check if the system will reboot to reinstall mode and reinitialize the device with a fresh factory reset.

Steps:

WARNING: ALL EXISTING DATA ON THIS DEVICE WILL BE WIPED!! 1. Check the current serial-assertion device-key $ ls /var/lib/snapd/save/device/private-keys-v1 2. Clear TPM first if this device has enabled secure boot & FDE For x86-based platforms: $ sudo su $ echo 5 > /sys/class/tpm/tpm0/ppi/request For ARM-based platforms: There is no generic command for ARM-based platforms, please refer to the device user manual. 3. Reboot the system to reinstall mode $ sudo snap reboot –install 4. The system should respond to the command and reboot itself. 5. Wait until the system completes the installation and initialization process. 6. Check the serial-assertion device-key after installation completes. $ ls /var/lib/snapd/save/device/private-keys-v1

Verification:

Check if a new serial-assertion device-key got generated after reinstallation completes.

Plugin:

manual

ubuntucore/sshd

SSH is enabled and operational

Category ID:

ubuntucore

Status:

Blocking

Purpose:

Check if a user can access the system through SSH from another machine

Steps:
  1. Execute the following command on another machine in the same network: $ ssh [user id]@[ip address of the testing system] 2. Enter the password to log in

Verification:

Can log into the system through SSH from another machine

Plugin:

manual

USB tests

The following test units are covered in this category:

usb-c-otg/g_ether

Check DUT can be detected as USB ethernet device

Category ID:

usb

Status:

Blocking

Purpose:

<missing purpose>

Steps:
  1. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT. 2. Press Enter to configure the IP address for the USB OTG ethernet interface on the DUT. (IP will be 192.168.9.1/24) 3. On the host, configure the IP address for the USB OTG ethernet interface. It should be in the same subnet as the DUT. (e.g., 192.168.9.2/24) 4. Try to ping between the host and DUT.

Verification:

The host and DUT can ping each other.

Description:

<missing description>

After-suspend:

True

User:

root

Plugin:

user-interact-verify

usb-c-otg/g_ether-cleanup

Cleanup USB OTG ethernet interface setup after ethernet device test

Category ID:

usb

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

usb-c-otg/g_mass_storage

Check DUT can be detected as a mass storage device

Category ID:

usb

Status:

Blocking

Purpose:

Check that after connecting the device under test (DUT) to another device (host), the DUT can be detected as a mass storage device by the host.

Steps:
  1. Press Enter to setup a 16 MB FAT32 image on the device. 2. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT.

Verification:

The host detects and mounts the mass storage device. It has read and write permissions on it.

After-suspend:

True

User:

root

Plugin:

user-interact-verify

usb-c-otg/g_mass_storage-cleanup

Cleanup mass storage setup after mass storage device test

Category ID:

usb

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

usb-c-otg/g_serial

Check if USB OTG can work as a serial port.

Category ID:

usb

Status:

Blocking

Purpose:

Check if USB OTG can work as a serial port.

Steps:
  1. Press enter to probe USB OTG g_serial 2. Connect a USB cable from an Ubuntu host to the USB OTG (it’s the USB Type C connector) port on DUT. 3. On the host, check if ttyACM0 has been probed $ ls /dev/ttyACM0 4. On the host, listen to the ttyACM0 port as the receiving side. $ sudo su $ cat /dev/ttyACM0 5. On DUT, send a string via /dev/ttyGS0 as the sending side. $ sudo su $ echo 123 > /dev/ttyGS0 6. Check if the string “123” was received on the host side. 7. Check the string sending from the host to DUT by repeating steps 4-6, but swap the send and receive sides.

Verification:

Does string send and receive function correctly?

After-suspend:

True

User:

root

Plugin:

user-interact-verify

usb-c-otg/g_serial-cleanup

Cleanup USB OTG serial interface setup after serial device test

Category ID:

usb

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

usb-c/c-to-a-adapter/hid

USB HID work on USB Type-C port using a “USB Type-C to Type-A” adapter

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port using a “USB Type-C to Type-A” adapter

Steps:
  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port using a “USB Type-C to Type-A” adapter 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text

Verification:

Did the device work as expected?

After-suspend:

True

Plugin:

manual

usb-c/c-to-a-adapter/insert

Verify that the system detects a USB 3 storage device using a “Type-C to Type-A” adapter.

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that the system correctly detects the insertion of a USB 3 storage device in a USB Type-C connector using a “Type-C to Type-A” adapter. NOTE: Make sure the USB storage device has a partition before starting the test.

Steps:
  1. Commence the test 2. Connect a USB 3 storage device to a USB Type-C port using a “Type-C to Type-A” adapter. 3. Do not unplug the device after the test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb-c/c-to-a-adapter/remove

Check the detection of a USB 3 storage device removal, connected via a “Type-C to Type-A” adapter.

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a USB 3 storage device connected to a USB Type-C port using a “USB Type-C to Type-A” adapter.

Steps:
  1. Begin the test 2. Disconnect a USB 3 storage device from a USB Type-C port using a “Type-C to Type-A” adapter

Verification:

The verification of this test is automated. Do not modify the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb-c/c-to-a-adapter/storage-automated

Automate the execution process post usb-c/c-to-a-adapter/insert test.

Category ID:

usb

Status:

Blocking

Purpose:

This test is automated and executes after the usb-c/c-to-a-adapter/insert test is run.

After-suspend:

True

User:

root

Plugin:

shell

usb-c/hid

USB HID work on USB Type-C port

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port

Steps:
  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text

Verification:

Did the device work as expected?

After-suspend:

True

Plugin:

manual

usb-c/insert

USB 3.0 storage device insertion detected on USB Type-C port

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that the system correctly detects the insertion of a USB 3.0 storage device in a USB Type-C connector. NOTE: Make sure the USB storage device has a partition before starting the test.

Steps:
  1. Commence the test. 2. Connect a USB 3.0 storage device to a USB Type-C port. 3. Do not unplug the device after the test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb-c/remove

USB 3.0 storage device removal detected on USB Type-C port

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a USB 3.0 storage device connected to a USB Type-C port.

Steps:
  1. Commence the test 2. Connect a USB 3.0 storage device to a USB Type-C port 3. Disconnect the USB 3.0 storage device from the USB Type-C port

Verification:

The system should automatically detect the removal of the USB 3.0 storage device. The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb-c/storage-automated

USB 3.0 storage device read & write works on USB Type-C port

Category ID:

usb

Status:

Blocking

Purpose:

This test is automated and executes after the usb-c/insert test is run.

After-suspend:

True

User:

root

Plugin:

shell

usb/hid

USB keyboard works

Category ID:

usb

Status:

Blocking

Purpose:

Check USB input device works

Steps:
  1. Connect USB keyboard 2. Input something with USB keyboard

Verification:

What was just input is displayed correctly

After-suspend:

True

Plugin:

manual

usb/insert

USB 2.0 storage device insertion detected

Category ID:

usb

Status:

Blocking

Purpose:

Check system can detect USB 2.0 storage when inserted. NOTE: Make sure the USB storage device has a partition before starting the test.

Steps:
  1. Press continue 2. Wait until the message “INSERT NOW” is printed on the screen 3. Connect USB 2.0 storage device

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb/remove

USB 2.0 storage device removal detected

Category ID:

usb

Status:

Blocking

Purpose:

Check system can detect removal of a USB 2.0 storage device

Steps:
  1. Press continue 2. Wait until the message “REMOVE NOW” is printed on the screen 3. Disconnect USB 2.0 storage device

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb/storage-automated

USB 2.0 storage device read & write works

Category ID:

usb

Status:

Blocking

Purpose:

Check system can read/write to USB 2.0 storage correctly

After-suspend:

True

User:

root

Plugin:

shell

usb/storage-detect

Detect storage partitions on a device on the USB bus

Category ID:

usb

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

usb/zapper-usb-insert-port_alias

Zapper-driven USB insertion test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb/zapper-usb-insert-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’

usb/zapper-usb-remove-port_alias

Zapper-driven USB removal test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb/zapper-usb-remove-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’

usb3/insert

USB 3.0 storage device insertion detected

Category ID:

usb

Status:

Blocking

Purpose:

Check system can detect insertion of a USB 3.0 storage device. NOTE: Make sure the USB storage device has a partition before starting the test.

Steps:
  1. Press continue. 2. Wait until the message “INSERT NOW” is displayed on the screen. 3. Connect USB 3.0 storage device.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb3/remove

USB 3.0 storage device removal detected

Category ID:

usb

Status:

Blocking

Purpose:

Check system can detect removal of a USB 3.0 storage device

Steps:
  1. Press continue 2. Wait until the message “REMOVE NOW” is printed on the screen 3. Disconnect USB 3.0 storage device

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb3/storage-automated

USB 3.0 storage device read & write works

Category ID:

usb

Status:

Blocking

Purpose:

Check system can read/write to USB 3.0 storage devices correctly

After-suspend:

True

User:

root

Plugin:

shell

usb3/zapper-usb-insert-port_alias

Zapper-driven USB 3.0 insertion test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb3/zapper-usb-insert-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’

usb3/zapper-usb-remove-port_alias

Zapper-driven USB 3.0 removal test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb3/zapper-usb-remove-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’

Wi-Fi access point

The following test units are covered in this category:

wireless/nmcli_wifi_ap_a_interface

Create 802.11a Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

wireless/nmcli_wifi_ap_a_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/nmcli_wifi_ap_bg_interface

Create 802.11b/g Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

From template:

wireless/nmcli_wifi_ap_bg_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_open_b_no_sta_interface_auto

Create open 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

Environment variable:

WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_open_b_no_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_open_b_no_sta_interface_manual

Create open 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

From template:

wireless/wifi_ap_open_b_no_sta_interface_manual

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_open_g_no_sta_interface_auto

Create an open 802.11g Wi-Fi AP on {interface} with no STA connected.

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

Environment variable:

WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_open_g_no_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_open_g_no_sta_interface_manual

Create open 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

From template:

wireless/wifi_ap_open_g_no_sta_interface_manual

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_setup_wizard_interface_auto

Create Access Point on {interface} using wifi-ap.setup-wizard

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

Environment variable:

WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_setup_wizard_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_b_no_sta_interface_auto

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

Environment variable:

WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_wpa_b_no_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_b_no_sta_interface_manual

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

From template:

wireless/wifi_ap_wpa_b_no_sta_interface_manual

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_b_with_sta_interface_auto

Create a WPA2 802.11b Wi-Fi Access Point on {interface} with active STA

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

Description:

<missing description>

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, OPEN_BG_SSID, WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_wpa_b_with_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_g_no_sta_interface_auto

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

Environment variable:

WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_wpa_g_no_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_g_no_sta_interface_manual

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

From template:

wireless/wifi_ap_wpa_g_no_sta_interface_manual

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_g_with_sta_interface_auto

Create WPA2 802.11g Wi-Fi Access Point on {interface} with active STA

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

Description:

<missing description>

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, OPEN_BG_SSID, WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_wpa_g_with_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless networking tests

The following test units are covered in this category:

wireless/check_iwlwifi_microcode_crash_interface

Check there have been no iwlwifi crashes

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/check_iwlwifi_microcode_crash_interface

Template resource:

device

Template filter:

device.driver == ‘iwlwifi’

wireless/detect

Detect if at least one Wireless LAN device is detected

Category ID:

wireless

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

wireless/wireless_connection_open_ac_nm_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_ac_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_ac_np_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_AC_SSID

From template:

wireless/wireless_connection_open_ac_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_ax_nm_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_ax_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_ax_np_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_AX_SSID

From template:

wireless/wireless_connection_open_ax_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_be_nm_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_be_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_be_np_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_BE_SSID

From template:

wireless/wireless_connection_open_be_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_bg_nm_interface

Connect to an unencrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_bg_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_bg_np_interface

Connect to unencrypted 802.11b/g Wi-Fi network on {{ interface }} using netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_BG_SSID

From template:

wireless/wireless_connection_open_bg_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_n_nm_interface

Connect to an unencrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_n_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_n_np_interface

Connect to unencrypted 802.11n Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_N_SSID

From template:

wireless/wireless_connection_open_n_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa3_ax_nm_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa3_ax_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa3_ax_np_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA3_AX_PSK, WPA3_AX_SSID

From template:

wireless/wireless_connection_wpa3_ax_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa3_be_nm_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa3_be_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa3_be_np_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA3_BE_PSK, WPA3_BE_SSID

From template:

wireless/wireless_connection_wpa3_be_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_ac_nm_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_ac_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_ac_np_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_AC_PSK, WPA_AC_SSID

From template:

wireless/wireless_connection_wpa_ac_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_ax_nm_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_ax_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_ax_np_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_AX_PSK, WPA_AX_SSID

From template:

wireless/wireless_connection_wpa_ax_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_be_nm_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_be_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_be_np_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_BE_PSK, WPA_BE_SSID

From template:

wireless/wireless_connection_wpa_be_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_bg_nm_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_bg_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_bg_np_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_BG_PSK, WPA_BG_SSID

From template:

wireless/wireless_connection_wpa_bg_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_n_nm_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_n_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_n_np_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }} using netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_N_PSK, WPA_N_SSID

From template:

wireless/wireless_connection_wpa_n_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_scanning_interface

Test system can discover Wi-Fi networks on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_scanning_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless Wide Area Network

The following test units are covered in this category:

wwan/3gpp-scan-manufacturer-model-hw_id-auto

Scan for available 3GPP networks with the modem

Unit type:

template

Category ID:

wwan

Status:

Blocking

Description:

Scan for available 3GPP networks with the target modem

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH

From template:

wwan/3gpp-scan-manufacturer-model-hw_id-auto

Template resource:

wwan_resource

wwan/check-sim-present-manual

Check if a SIM card is present in a slot connected to the modem

Category ID:

wwan

Status:

Blocking

Purpose:

Check if a SIM card is present in a slot connected to the modem

Steps:
  1. Open another terminal on SUT (or press Ctrl+Z to suspend Checkbox) 2. Run sudo mmcli -m 0 | grep SIM 3. Run fg to jump back to Checkbox (if you’re running in the same terminal)

Verification:

Check the output. Was the SIM information printed?

After-suspend:

True

Plugin:

manual

wwan/check-sim-present-manufacturer-model-hw_id-auto

Check if a SIM card is present in a slot connected to the modem

Unit type:

template

Category ID:

wwan

Status:

Blocking

Description:

Check if a SIM card is present in a slot connected to the modem

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH

From template:

wwan/check-sim-present-manufacturer-model-hw_id-auto

Template resource:

wwan_resource

wwan/detect

Identify if WWAN module is missing

Category ID:

wwan

Status:

Blocking

Purpose:

Tests that there is a WWAN module present and indicates that testing of it should follow.

After-suspend:

True

User:

root

Plugin:

shell

wwan/detect-manual

Check if WWAN module is available

Category ID:

wwan

Status:

Blocking

Purpose:

Check if WWAN module is available and ready to be used.

Steps:
  1. Open another terminal on SUT (or press ctrl+z to suspend Checkbox) 2. Run sudo mmcli -L command. 3. Run fg to jump back to checkbox (if you’re running in the same terminal)

Verification:

Check the output. Was the modem listed?

After-suspend:

True

Plugin:

manual

wwan/gsm-connection-interrupted-manual

Verify a GSM broadband connection can be reconnected after the signal is lost

Category ID:

wwan

Status:

Blocking

Purpose:

Ensure that the data connection can be revived after losing signal

Steps:
  1. Open another terminal on SUT (or press ctrl+z to suspend Checkbox). 2. Run sudo nmcli c add type gsm ifname {{ __checkbox_env__.get(“WWAN_CONTROL_IF”, “<device name>”) }} con-name GSM apn {{ __checkbox_env__.get(“WWAN_APN”, “<your.carrier.com>”) }} {%- if “WWAN_CONTROL_IF” not in __checkbox_env__ %} Replacing <your.carrier.com> with the appropriate APN name, and <device name> with the appropriate device (device can be found by running sudo nmcli d). {%- endif %} {%- if “WWAN_NET_IF” not in __checkbox_env__ %} 3. Ping Google (Run: ping -I <ifname> 8.8.8.8 -c 5). {%- else %} 3. Ping Google (Run: ping -I {{ __checkbox_env__.get(“WWAN_NET_IF”) }} 8.8.8.8 -c 5). {%- endif %} 4. Place the system in a Faraday bag. 5. Ping Google (and verify that the connection no longer works). 6. Remove the Faraday bag. {%- if “WWAN_NET_IF” not in __checkbox_env__ %} 7. Ping Google (Run: ping -I <ifname> 8.8.8.8 -c 5). {%- else %} 7. Ping Google (Run: ping -I {{ __checkbox_env__.get(“WWAN_NET_IF”) }} 8.8.8.8 -c 5). {%- endif %} 8. Disconnect from the network (Run: sudo nmcli r wwan off). 9. Run fg to jump back to Checkbox (if you’re running in the same terminal).

Verification:

Was the connection revived after removing the Faraday bag?

After-suspend:

True

Plugin:

manual

wwan/gsm-connection-manufacturer-model-hw_id-auto

Verify a GSM broadband modem can create a data connection

Unit type:

template

Category ID:

wwan

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, WWAN_APN, WWAN_CONTROL_IF, WWAN_NET_IF, WWAN_SETUPTIME

From template:

wwan/gsm-connection-manufacturer-model-hw_id-auto

Template resource:

wwan_resource

Non-blocking

Informational tests

The following test units are covered in this category:

manifest

Hardware Manifest

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

This job loads the hardware manifest and exposes it as a resource.

Plugin:

resource

client-cert-iot-server-24-04

Note

The certification tests presented in this document are validated by Checkbox version 4.2.0.dev76.

Blocking

Advanced Configuration and Power Interface

The following test units are covered in this category:

acpi/oem_osi

Test ACPI OEM _OSI strings

Category ID:

acpi

Status:

Blocking

Purpose:

This checks if the deprecated OEM _OSI strings are still used by checking the ACPI DSDT and SSDT tables

User:

root

Plugin:

shell

Audio tests

Output needs to be undistorted between 0%-100%. Output lines tested:

  • Internal speakers

  • 3.5mm headphones

  • HDMI audio output

  • DisplayPort audio output

Input needs to be recorded undistorted between 0%-100%. Input lines tested:

  • Internal microphone

  • 3.5mm microphone

Plug detection: when a new audio line input or output is plugged in the system, it needs to be recognized.

The following test units are covered in this category:

audio/alsa-loopback-automated

Captured sound matches played one (automated)

Category ID:

audio

Status:

Blocking

Purpose:

Check if sound that is ‘hearable’ by capture device. This test is intended mostly for devices without internal speakers/microphones, but with a line-in and a line-out ports to which a loopback cable can be connected.

After-suspend:

True

Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH, LD_LIBRARY_PATH

User:

root

Plugin:

shell

audio/alsa-playback

Playback works

Category ID:

audio

Status:

Blocking

Purpose:

Check if sound is played through ALSA on the default output

Steps:
  1. Make sure speakers or headphones are connected to the device 2. Commence the test

Verification:

Did you hear the sound?

After-suspend:

True

Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH

User:

root

Plugin:

user-interact-verify

audio/detect-capture-devices

Check that at least one audio capture device exists

Category ID:

audio

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

audio/detect-playback-devices

Check that at least one audio playback device exists

Category ID:

audio

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

Bluetooth - BlueZ Self Tests

The following test units are covered in this category:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test

BlueZ-{bluez-internal-bnep-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

After-suspend:

True

From template:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test

Template resource:

bluez-internal-bnep-tests

bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test

BlueZ-{bluez-internal-hci-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

Description:

Runs a specific test from the hci test suite

After-suspend:

True

From template:

bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test

Template resource:

bluez-internal-hci-tests

bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test

BlueZ-{bluez-internal-rfcomm-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

After-suspend:

True

From template:

bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test

Template resource:

bluez-internal-rfcomm-tests

bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test

BlueZ-{bluez-internal-uc-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

After-suspend:

True

From template:

bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test

Template resource:

bluez-internal-uc-tests

Bluetooth tests

Bluetooth LE (Smart and Smart Ready) is tested for device scanning and pairing. Apart from pairing, several profiles are specifically tested and required:

  • Eddystone Beacon

  • HID Over GATT Profile (HOGP), Low-Energy keyboard or mouse with basic functionality

The following test units are covered in this category:

bluetooth/audio-a2dp

Verify Bluetooth device’s High Fidelity Playback (A2DP) capability.

Category ID:

bluetooth

Status:

Blocking

Purpose:

This test will check the High Fidelity Playback (A2DP) capability of your Bluetooth device, to see if you can hear audio from it.

Steps:
  1. Enable and pair the Bluetooth headset 2. Click “Test” to play a brief tone on your Bluetooth device, if it failed to set the Mode to A2DP, please select the device and change it manually in the “Sound Settings”

Verification:

Did you hear the tone?

After-suspend:

True

Plugin:

user-interact-verify

bluetooth/audio_record_playback

Verify Bluetooth HSP/HFP profile capability for recording and playback.

Category ID:

bluetooth

Status:

Blocking

Purpose:

This test will check the Headset Head Unit (HSP/HFP) capability of your Bluetooth device, to check if you can record sounds.

Steps:
  1. Enable and pair the bluetooth headset. 2. Click “Test”, then speak into your Bluetooth microphone. 3. After a few seconds, your speech will be played back to you.

Verification:

Did you hear your speech played back?

After-suspend:

True

Plugin:

user-interact-verify

bluetooth/bluetooth_obex_send

Bluetooth OBEX send

Category ID:

bluetooth

Status:

Blocking

Purpose:

This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable

After-suspend:

True

Environment variable:

BTDEVADDR, PLAINBOX_PROVIDER_DATA

Plugin:

shell

bluetooth/bluez-controller-detect

Check bluez lists a controller if rfkill detects one

Category ID:

bluetooth

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

bluetooth/detect

Make sure at least one bluetooth device is detected

Category ID:

bluetooth

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

bluetooth/detect-output

Store Bluetooth device information for reports.

Category ID:

bluetooth

Status:

Blocking

Purpose:

Automated test to store Bluetooth device information in the Checkbox report

After-suspend:

True

Plugin:

shell

bluetooth/zapper-a2dp

Check if the system can connect to a Bluetooth speaker using A2DP profile

Category ID:

bluetooth

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

ZAPPER_HOST

Plugin:

shell

bluetooth4/HOGP-keyboard

Verify HOGP keyboard functionality with Bluetooth Smart.

Category ID:

bluetooth

Status:

Blocking

Purpose:

This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart keyboard.

Steps:
  1. Enable a Bluetooth Smart keyboard, and put it into pairing mode. 2. Commence the test to do the auto-pairing, you will be asked to select the targeting keyboard from the list. 3. After it’s paired and connected, enter some text with your keyboard.

Verification:

Did the Bluetooth Smart keyboard work as expected?

After-suspend:

True

Plugin:

user-interact-verify

bluetooth4/beacon_eddystone_url_interface

Test system can get beacon EddyStone URL advertisements on the {interface} adapter

Unit type:

template

Category ID:

bluetooth

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

bluetooth4/beacon_eddystone_url_interface

Template resource:

device

Template filter:

device.category == ‘BLUETOOTH’

Camera tests

The following test units are covered in this category:

camera/multiple-resolution-images-rpi-attachment_name

Attach an image from the multiple resolution images test on rpi

Unit type:

template

Category ID:

camera

Status:

Blocking

Description:

This test will attach one of the images used for the multiple resolution images test.

From template:

camera/multiple-resolution-images-rpi-attachment_name

Template resource:

device

Template filter:

device.category == ‘MMAL’ and device.name != ‘’

camera/multiple-resolution-images-rpi_name

Webcam multiple resolution capture test for Pi Camera

Unit type:

template

Category ID:

camera

Status:

Blocking

From template:

camera/multiple-resolution-images-rpi_name

Template resource:

device

Template filter:

device.category == ‘MMAL’ and device.name != ‘’

camera/multiple-resolution-images_name

Webcam multiple resolution capture test for {product_slug}

Unit type:

template

Category ID:

camera

Status:

Blocking

After-suspend:

True

From template:

camera/multiple-resolution-images_name

Template resource:

device

Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

camera/roundtrip-qrcode_name

Test video output and camera {{ name }} by displaying and reading a QR code

Unit type:

template

Category ID:

camera

Status:

Blocking

From template:

camera/roundtrip-qrcode_name

Template resource:

device

Template filter:

device.category in (‘CAPTURE’, ‘MMAL’) and device.name != ‘’

CPU tests

x86_64 and ARM processors are tested to ensure proper functionality. We will test specific features as:

  • CPU’s performance states (frequency up and down in runtime)

  • CPU’s sleep states (cpu on and off in runtime)

  • Running CPU at its maximum frequency

We will also include a general stress test performed for 120 minutes to verify that the system can handle a sustained high load for a period of time. This test uses the tool “stress-ng” available in the Universe repositories.

For Intel CPU’s, the IPDT (Intel Processor Diagnostic Tool) test suite will be run. The diagnostic checks for brand identification, verifies the processor operating frequency, tests specific processor features, and performs a stress test on the processor.

The following test units are covered in this category:

cpu/arm64_vfp_support_platform

Validate that the Floating Point Unit is running on {platform} device

Unit type:

template

Category ID:

cpu

Status:

Blocking

Description:

Validate that the Floating Point Unit is running on {platform} device.

From template:

cpu/arm64_vfp_support_platform

Template resource:

cpuinfo

Template filter:

cpuinfo.platform == ‘aarch64’

cpu/armhf_vfp_support_platform

Validate that the Vector Floating Point Unit is running on {platform} device

Unit type:

template

Category ID:

cpu

Status:

Blocking

Description:

Validate that the Vector Floating Point Unit is running on {platform} device.

From template:

cpu/armhf_vfp_support_platform

Template resource:

cpuinfo

Template filter:

cpuinfo.platform == ‘armv7l’

cpu/clocktest

Tests the CPU for clock jitter

Category ID:

cpu

Status:

Blocking

Purpose:

Runs a test for clock jitter on SMP machines.

After-suspend:

True

Plugin:

shell

cpu/cstates

Run C-States tests

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Uses the Firmware Test Suite (fwts) to test the power saving states of the CPU.

Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP

User:

root

Plugin:

shell

cpu/cstates_results.log

Attach C-States test log

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Attaches the FWTS desktop diagnosis results log to the submission.

Plugin:

attachment

cpu/maxfreq_test

Test that the CPU can run at its max frequency

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.

Environment variable:

LD_LIBRARY_PATH, SNAP

User:

root

Plugin:

shell

cpu/maxfreq_test-log-attach

Attach CPU max frequency log

Category ID:

cpu

Status:

Blocking

Purpose:

Attaches the log generated by cpu/maxfreq_test to the results submission.

Plugin:

attachment

cpu/offlining_test

Test offlining of each CPU core

Category ID:

cpu

Status:

Blocking

Purpose:

Attempts to offline each core in a multicore system.

After-suspend:

True

User:

root

Plugin:

shell

cpu/scaling_test

Test the CPU scaling capabilities

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Use Firmware Test Suite (fwts cpufreq) to test the scaling capabilities of the CPU.

Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP

User:

root

Plugin:

shell

cpu/scaling_test-log-attach

Attach CPU scaling capabilities log

Category ID:

cpu

Status:

Blocking

Purpose:

Attaches the log generated by cpu/scaling_test to the results submission.

Plugin:

attachment

cpu/topology

Check CPU topology for accuracy between proc and sysfs

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.

After-suspend:

True

Plugin:

shell

Disk tests

The following test units are covered in this category:

disk/check-software-raid

Validate the configuration of software RAID devices are expected

Category ID:

disk

Status:

Blocking

Purpose:

<missing purpose>

Description:

Examine the system to detect Software RAID devices are created and the RAID mode are expected the SOFTWARE_RAID_LEVEL variable is needed for this tests. e.g. SOFTWARE_RAID_LEVEL=”raid0 raid1 raid5”

Environment variable:

SOFTWARE_RAID_LEVEL

User:

root

Plugin:

shell

disk/detect

Gathers information about each disk detected

Category ID:

disk

Status:

Blocking

Purpose:

Uses lsblk to gather information about each disk detected on the system under test.

After-suspend:

True

Plugin:

shell

disk/encryption/check-fde-tpm

Disk decryption after TPM change

Category ID:

disk

Status:

Blocking

Purpose:

The device partition is encrypted using TPM master key. To unseal the master key from TPM, PCR7 (Platform Configuration Register 7) needs to be identical to the value it had when the master key was sealed into TPM. Every time the device boots, it checks PCR7 to unseal TPM and retrieves master key from TPM to decrypt its data partition. If TPM PCR7 is modified (e.g. by flashing the BIOS), the device won’t be able to get the master key and decrypt its data partition. This test verifies the system’s resilience against unauthorized modifications by ensuring it cannot boot if the PCR7 value is altered.

Steps:

NOTE: YOU’LL HAVE TO RE-INSTALL THE IMAGE AFTER THIS TEST. 1. Install the image and make sure it boots and you can log in. 2. Ensure the BIOS is set up correctly (e.g., TPM enabled, UEFI boot mode). 3. Depending on the DUT, choose one of these methods to clean TPM: a. Turn the device off and upgrade/downgrade the BIOS or modify Secure Boot state. b. Clean TPM via BIOS menu. c. Install checkbox, execute “checkbox-[project name].checkbox-cli run com.canonical.certification::tpm2.0_3.0.4/tpm2_takeownership”. 4. Start or reboot the device.

Verification:

Mark this test as “Passed” if the device cannot boot anymore.

Plugin:

manual

disk/encryption/detect

Test that Full Disk Encryption is in use

Category ID:

disk

Status:

Blocking

Purpose:

Examine the system to detect if one of the standard full disk encryption implementations is in use

User:

root

Plugin:

shell

disk/read_performance_name

Disk performance test for {product_slug}

Unit type:

template

Category ID:

disk

Status:

Blocking

After-suspend:

True

Environment variable:

DISK_MDADM_READ_PERF, DISK_NVME_READ_PERF, DISK_READ_PERF

From template:

disk/read_performance_name

Template resource:

device

Template filter:

device.category == ‘DISK’

disk/stats_name

Disk statistics for {product_slug}

Unit type:

template

Category ID:

disk

Status:

Blocking

After-suspend:

True

From template:

disk/stats_name

Template resource:

device

Template filter:

device.category == ‘DISK’ and device.name != ‘’

disk/storage_device_name

Disk I/O stress test for {product_slug}

Unit type:

template

Category ID:

disk

Status:

Blocking

After-suspend:

True

From template:

disk/storage_device_name

Template resource:

device

Template filter:

device.category == ‘DISK’

thunderbolt3/insert

Storage insert detection on Thunderbolt 3 port

Category ID:

disk

Status:

Blocking

Purpose:

This test will check if the insertion of a Thunderbolt 3 HDD could be detected

Steps:
  1. Click ‘Test’ to begin the test. This test will timeout and fail if the insertion has not been detected within 40 seconds. 2. Plug a Thunderbolt HDD into an available Thunderbolt 3 port, if it’s not mounted automatically, please click the HDD icon to mount it.

Verification:

The verification of this test is automated. Do not change the automatically selected result

After-suspend:

True

User:

root

Plugin:

user-interact

thunderbolt3/remove

Storage removal detection on Thunderbolt 3 port

Category ID:

disk

Status:

Blocking

Purpose:

This test will check if the system can detect the removal of a Thunderbolt 3 HDD

Steps:
  1. Click ‘Test’ to begin the test. This test will timeout and fail if the removal has not been detected within 20 seconds. 2. Remove the previously attached Thunderbolt HDD from the Thunderbolt port.

Verification:

The verification of this test is automated. Do not change the automatically selected result

After-suspend:

True

User:

root

Plugin:

user-interact

thunderbolt3/storage-test

Storage test on Thunderbolt 3

Category ID:

disk

Status:

Blocking

Purpose:

This is an automated test which performs read/write operations on an attached Thunderbolt HDD

After-suspend:

True

User:

root

Plugin:

shell

Docker containers

The following test units are covered in this category:

docker/build-single_arch

Test docker build with a single container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/build-single_arch

Template resource:

docker_resource

docker/commit_arch

Test docker commit a change to a single container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/commit_arch

Template resource:

docker_resource

docker/compose-and-basic_arch

Test docker compose and basic command

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/compose-and-basic_arch

Template resource:

docker_resource

docker/compose-restart_arch

Test compose a container with restart policy applied

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/compose-restart_arch

Template resource:

docker_resource

docker/compose-single_arch

Test docker compose with a single container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/compose-single_arch

Template resource:

docker_resource

docker/copy_arch

Test copy a file bwtween a container and local filesystem

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/copy_arch

Template resource:

docker_resource

docker/deploy-registry_arch

Deploy a registry server and run it on localhost

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/deploy-registry_arch

Template resource:

docker_resource

docker/diff_arch

Test changes to files in Ubuntu container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/diff_arch

Template resource:

docker_resource

docker/export-and-import_arch

Test docker import and export a docker container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/export-and-import_arch

Template resource:

docker_resource

docker/info

Display system-wide information about docker

Category ID:

docker

Status:

Blocking

Purpose:

<missing purpose>

Steps:

<missing steps>

Verification:

<missing verification>

Description:

<missing description>

User:

root

docker/inspect_arch

Test query low-level information on a docker object

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/inspect_arch

Template resource:

docker_resource

docker/interative_arch

Test an interactive shell in Ubuntu container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/interative_arch

Template resource:

docker_resource

docker/kill_arch

Test killing containers

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/kill_arch

Template resource:

docker_resource

docker/restart-always_arch

Test container restart policy with always applied

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/restart-always_arch

Template resource:

docker_resource

docker/restart-on-failure_arch

Test container restart policy with on_failure applied

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/restart-on-failure_arch

Template resource:

docker_resource

docker/run_arch

Download and run ubuntu container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/run_arch

Template resource:

docker_resource

docker/save-and-load_arch

Test docker save and load a docker image

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/save-and-load_arch

Template resource:

docker_resource

docker/start-stop_arch

Start and stop a single container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/start-stop_arch

Template resource:

docker_resource

docker/update_arch

Test update configuration of one container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/update_arch

Template resource:

docker_resource

docker/version

Display docker version information

Category ID:

docker

Status:

Blocking

Purpose:

<missing purpose>

Steps:

<missing steps>

Verification:

<missing verification>

Description:

<missing description>

User:

root

Eeprom

The following test units are covered in this category:

eeprom/read-write

Test EEPROM read and write functions for the system.

Category ID:

eeprom

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

User:

root

Plugin:

shell

Error Detection And Correction (EDAC) Memory Controllers

The following test units are covered in this category:

edac/default-report

Attach the default report from EDAC util

Category ID:

edac

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

attachment

edac/detect

Detect if the EDAC drivers are loaded and Memory Controllers are found

Category ID:

edac

Status:

Blocking

Purpose:

<missing purpose>

Steps:

<missing steps>

Verification:

<missing verification>

Description:

<missing description>

Ethernet Device tests

Connections are tested for functionality, but not for performance.

The following test units are covered in this category:

ethernet/detect

Detect if at least one ethernet device is detected

Category ID:

ethernet

Status:

Blocking

Purpose:

Test to detect and return information about available network controllers on the system under test.

After-suspend:

True

Plugin:

shell

ethernet/hotplug-interface

Ensure hotplugging works on port {{ interface }}

Unit type:

template

Category ID:

ethernet

Status:

Blocking

After-suspend:

True

From template:

ethernet/hotplug-interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’

ethernet/ping_interface

Can ping another machine over Ethernet port {interface}

Unit type:

template

Category ID:

ethernet

Status:

Blocking

After-suspend:

True

From template:

ethernet/ping_interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’

ethernet/wol_S3_interface

Wake on LAN (WOL) test from S3 - {interface}

Unit type:

template

Category ID:

ethernet

Status:

Blocking

From template:

ethernet/wol_S3_interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’ and device.interface != ‘UNKNOWN’

ethernet/wol_S4_interface

Wake on LAN (WOL) test from S4 - {interface}

Unit type:

template

Category ID:

ethernet

Status:

Blocking

From template:

ethernet/wol_S4_interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’

ethernet/wol_S5_interface

Wake on LAN (WOL) test from S5 - {interface}

Unit type:

template

Category ID:

ethernet

Status:

Blocking

From template:

ethernet/wol_S5_interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’ and device.mac != ‘UNKNOWN’

Firmware tests

The Ubuntu image must be installed using the factory default bootloader firmware (for example BIOS, UEFI or uboot as applicable) and with the default options (including SecureBoot, if that’s the default setting). Firmware needs to be compliant with Canonical Firmware Test Suite (FWTS).

It is recommended that after running Canonical fwts with the list of tests defined in the Appendix A, ideally, no CRITICAL or HIGH failures should be reported, but those are not automatically certification blockers.

The following test units are covered in this category:

firmware/fwts_desktop_diagnosis

Run FWTS QA-concerned desktop-specific diagnosis tests.

Category ID:

firmware

Status:

Blocking

Purpose:

<missing purpose>

Description:

Run Firmware Test Suite (fwts) QA-concerned desktop-specific diagnosis tests.

Environment variable:

PLAINBOX_SESSION_SHARE

User:

root

Plugin:

shell

firmware/fwts_desktop_diagnosis_results.log.gz

Attach FWTS desktop diagnosis log to submission

Category ID:

firmware

Status:

Blocking

Purpose:

<missing purpose>

Description:

Attaches the FWTS desktop diagnosis results log to the submission

Plugin:

attachment

Gathers information about the DUT

The following test units are covered in this category:

connections

Collect information about connections

Category ID:

information_gathering

Status:

Blocking

Purpose:

<missing purpose>

Description:

Generates a list of plug and slot connections on the device

Plugin:

resource

rtc

Creates resource info for RTC

Category ID:

information_gathering

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

resource

serial_assertion

Collect serial assertions on the device

Category ID:

information_gathering

Status:

Blocking

Purpose:

<missing purpose>

Description:

Queries the snapd REST API for serial assertions present on the device.

Plugin:

resource

sleep

Create resource info for supported sleep states

Category ID:

information_gathering

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

resource

snap

Collect information about installed snap packages

Category ID:

information_gathering

Status:

Blocking

Purpose:

<missing purpose>

Description:

Generates a list of snap packages

Plugin:

resource

General Purpose I/O

We test the functionality of individual GPIO lines when the associated controller driver in the kernel implements a GPIO Sysfs Interface via the gpiolib implementers framework. In such cases, the GPIO system may be tested in two ways:

  • Direct:

    • GPIO controllers are exposed through sysfs

    • GPIO lines are accessible by the user

  • Indirect:

    • Communication with device connected via GPIO

The following test units are covered in this category:

gpio/gpiomem_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via /dev/gpiomem

Unit type:

template

Category ID:

gpio

Status:

Blocking

After-suspend:

True

From template:

gpio/gpiomem_loopback_pairs_model

Template resource:

com.canonical.certification::model_assertion

Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)

gpio/sysfs_loopback_pairs_model

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template

Category ID:

gpio

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

gpio/sysfs_loopback_pairs_model

Template resource:

com.canonical.certification::model_assertion

Template filter:

model_assertion.model in (“pi2”, “pi3”, “ubuntu-core-18-pi2”, “ubuntu-core-18-pi3”)

gpio/sysfs_loopback_pairs_vendor_product

Test GPIO lines exposed on headers can be controlled via sysfs

Unit type:

template

Category ID:

gpio

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

gpio/sysfs_loopback_pairs_vendor_product

Template resource:

dmi

Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’, ‘UPN-EHL01’)

Informational tests

The following test units are covered in this category:

info/systemd-analyze

System boot-up performance statistics

Category ID:

info

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

info/systemd-analyze-critical-chain

Print the tree of the time-critical chain of SystemD

Category ID:

info

Status:

Blocking

Purpose:

This job prints a tree of the time-critical chain of SystemD units.

Plugin:

shell

lspci_attachment

Attach a list of PCI devices

Category ID:

info

Status:

Blocking

Purpose:

Attaches very verbose lspci output.

Plugin:

attachment

lsusb_attachment

Attach output of lsusb

Category ID:

info

Status:

Blocking

Purpose:

Attaches a list of detected USB devices.

After-suspend:

True

User:

root

Plugin:

attachment

net_if_management_attachment

Collect logging from the net_if_management job

Category ID:

info

Status:

Blocking

Purpose:

<missing purpose>

Description:

Allows logging of debug info and errors by the associated resource job

Plugin:

attachment

parts_meta_info_attachment

Attaches an information about all parts that constituted this snap

Category ID:

info

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

CHECKBOX_RUNTIME, SNAP

User:

root

Plugin:

attachment

Input Devices tests

The following test units are covered in this category:

input/zapper-keyboard

Check if the system recognizes keyboard events from Zapper

Category ID:

input

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

ZAPPER_HOST

User:

root

Plugin:

shell

Intel Integrated Sensor Hub Transport Protocol (ISHTP)

The following test units are covered in this category:

eclite/module-detect

Verifies that the ishtp_eclite module is loaded

Category ID:

intel-ishtp

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

eclite/temperature-reading-from-thermal-acpitz

Read the temperature of acpitz to ensure Eclite is functional

Category ID:

intel-ishtp

Status:

Blocking

Purpose:

For testing the feature of Eclite, one way is to monitor the temperature of acpitz thermal.

User:

root

Plugin:

shell

ishtp/device-detect

Verify that at least 1 device entry exists in /sys/bus/ishtp/devices

Category ID:

intel-ishtp

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

ishtp/module-detect

Verifies that the intel_ish_ipc module for ISHTP is loaded

Category ID:

intel-ishtp

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

Intel Management Engine Interface (MEI)

The following test units are covered in this category:

mei/check-device

Detect if the Intel MEI device is available

Category ID:

mei

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

mei/check-module

Detect if the Intel MEI kernel module is loaded

Category ID:

mei

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

mei/get-firmware-version

Retrieve MEI firmware version by MEI interface

Category ID:

mei

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

I²C (Inter-Integrated Circuit)

All devices attached to the I2C bus must be detectable. This includes:

  • Temperature sensors

  • Humidity sensors

  • Accelerometers

The following test units are covered in this category:

i2c/i2c-bus-detect

Check presence of an I²C bus

Category ID:

i2c

Status:

Blocking

Purpose:

If an expected number of I²C buses is provided, the job will verify the detected number is correct. If the expected number of buses is not provided, the job will pass if at least one I²C bus is detected.

Environment variable:

I2C_BUS_NUMBER

User:

root

Plugin:

shell

i2c/i2c-device-detect

Check if any I²C devices can be detected

Category ID:

i2c

Status:

Blocking

Purpose:

The test will pass if there’s at least one I²C device detected on any I²C bus.

User:

root

Plugin:

shell

LED tests

When LEDs exist, they will be tested by following some basic expectations here. The actual behavior may vary depending on the hardware design. To ensure that the behavior is working as expected, please be sure to test against specifications obtained from OEM, as each OEM may have different defined behavior for LEDs. The following LEDs are tested:

  • Power

  • Serial Port LEDs (indicating activity)

The following test units are covered in this category:

led-indicator/gpio-controller-leds-name

Check control of {name} LED indicator.

Unit type:

template

Category ID:

led

Status:

Blocking

After-suspend:

True

From template:

led-indicator/gpio-controller-leds-name

Template resource:

led-indicator/gpio-controller-leds

led-indicator/gpio-leds-name

Check control of {name} LED indicator.

Unit type:

template

Category ID:

led

Status:

Blocking

After-suspend:

True

From template:

led-indicator/gpio-leds-name

Template resource:

led-indicator/gpio-leds

led-indicator/sysfs-leds-manual

Check control of {name} LED.

Unit type:

template

Category ID:

led

Status:

Blocking

After-suspend:

True

From template:

led-indicator/sysfs-leds-manual

Template resource:

led-indicator/sysfs-leds

led/fn

Test the Fn key LED functionality by activating/deactivating the Fn keys locking.

Category ID:

led

Status:

Blocking

Purpose:

This test will test the Fn key LED.

Steps:
  1. Press the Fn+Fn Lock key to activate/deactivate Fn keys locking. 2. The Fn key LED should be switched on/off every time the key is pressed.

Verification:

Did the Fn key LED light as expected?

After-suspend:

True

Plugin:

manual

led/power

Power LED behavior when powered

Category ID:

led

Status:

Blocking

Purpose:

Check power led is on when system is powered on

Steps:
  1. Check power led when system is powered on

Verification:

Power led is on when system is powered on

After-suspend:

True

Plugin:

manual

led/serial

Serial ports LED behavior

Category ID:

led

Status:

Blocking

Purpose:

Check serial ports LED behavior is correct

Steps:
  1. Start the test to send data to all serial ports (/dev/ttyS*)

Verification:

All serial ports LED are on for a few seconds (3-4s)

User:

root

Plugin:

user-interact-verify

led/sysfs_led_brightness_off_vendor_product

Ensure the leds_aaeon driver properly sets all LEDs to off or minimum brightness by running a test.

Unit type:

template

Category ID:

led

Status:

Blocking

After-suspend:

True

From template:

led/sysfs_led_brightness_off_vendor_product

Template resource:

dmi

Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’)

led/sysfs_led_brightness_on_vendor_product

Verify the functionality of the leds_aaeon driver by ensuring all external LEDs achieve maximum brightness.

Unit type:

template

Category ID:

led

Status:

Blocking

After-suspend:

True

From template:

led/sysfs_led_brightness_on_vendor_product

Template resource:

dmi

Template filter:

dmi.category == ‘SYSTEM’ and dmi.vendor == ‘AAEON’ and dmi.product in (‘UPX-TGL01’)

led/wireless

Verify the WLAN/Bluetooth LED functionality by toggling wireless connections.

Category ID:

led

Status:

Blocking

Purpose:

Wireless (WLAN + Bluetooth) LED verification

Steps:
  1. Make sure WLAN connection is established and Bluetooth is enabled. 2. WLAN/Bluetooth LED should light 3. Switch WLAN and Bluetooth off from a hardware switch (if present) 4. Switch them back on 5. Switch WLAN and Bluetooth off from the panel applet 6. Switch them back on

Verification:

Did the WLAN/Bluetooth LED light as expected?

Plugin:

manual

Media Card tests

Media Card readers are tested for read and write for the following type of cards:

  • CF

  • MMC

  • MS

  • MSP

  • SD

  • SDHC

  • SDXC

  • XD

The following test units are covered in this category:

mediacard/cf-insert

Verify the system’s media card reader can detect a Compact Flash card insertion.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a Compact Flash (CF) media card

Steps:
  1. Commence the test and then insert a CF card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/cf-remove

Ensure the system detects CF card removal from the card reader correctly.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a CF card from the system’s card reader.

Steps:
  1. Commence the test and then remove the CF card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/cf-storage

Automate testing for reading and writing to the CF card after mediacard/cf-insert test.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/cf-insert test is run. It tests reading and writing to the CF card.

User:

root

Plugin:

shell

mediacard/mmc-insert

Test the system’s media card reader for MMC insertion detection.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a Multimedia Card (MMC) media

Steps:
  1. Commence the test and then insert an MMC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/mmc-remove

Test the detection of MMC card removal from the system’s card reader.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of the MMC card from the system’s card reader.

Steps:
  1. Commence the test and then remove the MMC card from the reader. (Note: this test will time out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/mmc-storage

Test automated execution for reading and writing to the MMC card.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/mmc-insert test is run. It tests reading and writing to the MMC card.

User:

root

Plugin:

shell

mediacard/ms-insert

Verify the detection of Memory Stick (MS) card insertion by the system’s media card reader.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a Memory Stick (MS) media card

Steps:
  1. Commence the test and then insert a MS card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/ms-remove

Test if the system detects the removal of an MS card correctly.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of an MS card from the system’s card reader.

Steps:
  1. Commence the test and then remove the MS card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/ms-storage

Automated test for reading and writing to the MS card after mediacard/ms-insert test.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/ms-insert test is run. It tests reading and writing to the MS card.

User:

root

Plugin:

shell

mediacard/msp-insert

Verify the system’s media card reader can detect a Memory Stick Pro (MSP) card insertion.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a Memory Stick Pro (MSP) media card

Steps:
  1. Commence the test and then insert a MSP card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/msp-remove

Ensure MSP card removal is correctly detected by the system.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a MSP card from the system’s card reader.

Steps:
  1. Commence the test and remove the MSP card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/msp-storage

Automated test for reading and writing to the MSP card after mediacard/msp-insert test.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/msp-insert test is run. It tests reading and writing to the MSP card.

User:

root

Plugin:

shell

mediacard/sd-insert

Test that insertion of an SD card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of an UNLOCKED Secure Digital (SD) media card

Steps:
  1. Commence the test and then insert an UNLOCKED SD card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/sd-remove

Test that removal of an SD card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of an SD card from the system’s card reader.

Steps:
  1. Commence the test and then remove the SD card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/sd-storage

Test reading & writing to an SD Card

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/sd-insert test is run. It tests reading and writing to the SD card.

User:

root

Plugin:

shell

mediacard/sdhc-insert

Test that insertion of an SDHC card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a UNLOCKED Secure Digital High-Capacity (SDHC) media card

Steps:
  1. Commence the test and then insert an UNLOCKED SDHC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

mediacard/sdhc-remove

Test that removal of an SDHC card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of an SDHC card from the system’s card reader.

Steps:
  1. Commence the test and then remove the SDHC card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

mediacard/sdhc-storage

Test reading & writing to a SDHC Card

Category ID:

mediacard

Status:

Blocking

Purpose:

<missing purpose>

Description:

This test is automated and executes after the mediacard/sdhc-insert test is run. It tests reading and writing to the SDHC card.

After-suspend:

True

User:

root

Plugin:

shell

mediacard/sdxc-insert

Test that insertion of an SDXC card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of a Secure Digital Extended Capacity (SDXC) media card

Steps:
  1. Commence the test and then insert an UNLOCKED SDXC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/sdxc-remove

Test that removal of an SDXC card is detected

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a SDXC card from the system’s card reader.

Steps:
  1. Commence the test and then remove the SDXC card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/sdxc-storage

Test reading & writing to an SDXC Card

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/sdxc-insert test is run. It tests reading and writing to the SDXC card.

User:

root

Plugin:

shell

mediacard/xd-insert

Ensure the system’s media card reader detects the insertion of an xD media card.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system’s media card reader can detect the insertion of an Extreme Digital (xD) media card.

Steps:
  1. Commence the test and then insert an xD card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/xd-remove

Automated test for detecting the removal of a xD card from the system’s card reader.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a xD card from the system’s card reader.

Steps:
  1. Commence the test and then remove the xD card from the reader. (Note: this test will time-out after 20 seconds.)

Verification:

The verification of this test is automated. Do not change the automatically selected result.

User:

root

Plugin:

user-interact

mediacard/xd-storage

Automated test to verify reading and writing functionality of the xD card after mediacard/xd-insert test.

Category ID:

mediacard

Status:

Blocking

Purpose:

This test is automated and executes after the mediacard/xd-insert test is run. It tests reading and writing to the xD card.

User:

root

Plugin:

shell

Memory tests

Proper detection of the amount of memory installed is required (the amount of memory installed is the memory seen by the OS).

The following test units are covered in this category:

memory/info

Check the amount of memory reported by meminfo against DMI

Category ID:

memory

Status:

Blocking

Purpose:

This test checks the amount of memory which is reported in meminfo against the size of the memory modules detected by DMI.

User:

root

Plugin:

shell

Miscellaneous tests

The following test units are covered in this category:

install/apt-get-gets-updates

Ensure apt can access repositories and get updates without installing them, to aid in recovery from broken updates.

Category ID:

miscellanea

Status:

Blocking

Purpose:

Tests to see that apt can access repositories and get updates (does not install updates). This is done to confirm that you could recover from an incomplete or broken update.

User:

root

Plugin:

shell

miscellanea/check_prerelease

Test that the system is not a pre-release version

Category ID:

miscellanea

Status:

Blocking

Purpose:

Test to verify that the system uses production, rather than pre-release, versions of the kernel and the OS.

Plugin:

shell

miscellanea/debsums

Check the MD5 sums of installed Debian packages

Category ID:

miscellanea

Status:

Blocking

Purpose:

Verify installed Debian package files against MD5 checksum lists from /var/lib/dpkg/info/*.md5sums.

User:

root

Plugin:

shell

miscellanea/device_check

Device Check

Category ID:

miscellanea

Status:

Blocking

Purpose:

Device check

Steps:
  1. Commence the test 2. Compare items on System Manifest to the devices known to udev

Verification:

Do the devices reported by udev match the devices on the Manifest?

Plugin:

user-interact-verify

miscellanea/grub_file_check

Check if the file core.efi exists to make sure shim and grub can be upgraded

Category ID:

miscellanea

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

miscellanea/oops

Run FWTS OOPS check

Category ID:

miscellanea

Status:

Blocking

Purpose:

<missing purpose>

Description:

Run Firmware Test Suite (FWTS) oops tests.

Environment variable:

PLAINBOX_SESSION_SHARE

User:

root

Plugin:

shell

miscellanea/oops_results.log

Attach the FWTS oops results for submission.

Category ID:

miscellanea

Status:

Blocking

Purpose:

Attaches the FWTS oops results log to the submission

Plugin:

attachment

miscellanea/secure_boot_mode_gadget

Test that {gadget} Ubuntu Core system booted with Secure Boot active

Unit type:

template

Category ID:

miscellanea

Status:

Blocking

From template:

miscellanea/secure_boot_mode_gadget

Template resource:

model_assertion

miscellanea/submission-resources

Check that data for a complete result are present

Category ID:

miscellanea

Status:

Blocking

Purpose:

<missing purpose>

Description:

A meta-job that verifies the data necessary for a complete result submission are present. Failure indicates that the results are incomplete and may be rejected.

Plugin:

shell

miscellanea/ubuntu-desktop-minimal-recommends

Check that all the recommended packages for ubuntu-desktop-minimal are installed

Category ID:

miscellanea

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

miscellanea/ubuntu-desktop-recommends

Check that all the recommended packages for ubuntu-desktop are installed

Category ID:

miscellanea

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

Monitor tests

Each of the available external video ports (supported ports are HDMI, DisplayPort, DVI) are tested one by one. Output to the display must work i.e. a console is presented.

The following test units are covered in this category:

monitor/displayport_hotplug

Can hotplug monitor (DisplayPort)

Category ID:

monitor

Status:

Blocking

Purpose:

This test will check the DisplayPort port and the ability to do hotplugging.

Steps:

Skip this test if your system does not have a DisplayPort port. 1. If a display is already connected, unplug it. 2. (Re-)Connect a display to the DisplayPort port on your system

Verification:

Was the interface displayed correctly on the screen?

After-suspend:

True

Plugin:

manual

monitor/dvi

Monitor works (DVI)

Category ID:

monitor

Status:

Blocking

Purpose:

Check output to display through DVI port

Steps:
  1. Connect display to DVI port 2. Check the display

Verification:

Output to display works

After-suspend:

True

Plugin:

manual

monitor/dvi-to-vga

Monitor works (DVI-to-VGA)

Category ID:

monitor

Status:

Blocking

Purpose:

Check output to display through VGA adaptor on DVI port

Steps:
  1. Connect display to VGA adaptor on DVI port 2. Check the display

Verification:

Output to display works

After-suspend:

True

Plugin:

manual

monitor/hdmi

Monitor works (HDMI)

Category ID:

monitor

Status:

Blocking

Purpose:

Check output to display through HDMI port

Steps:
  1. Connect display to HDMI port 2. Check the display

Verification:

Output to display works

After-suspend:

True

Plugin:

manual

monitor/hdmi-to-vga

Monitor works (HDMI-to-VGA)

Category ID:

monitor

Status:

Blocking

Purpose:

Check output to display through VGA adaptor on HDMI port

Steps:
  1. Connect display to VGA adaptor on HDMI port 2. Check the display

Verification:

Output to display works

After-suspend:

True

Plugin:

manual

monitor/vga

Monitor works (VGA)

Category ID:

monitor

Status:

Blocking

Purpose:

Check output to display through VGA port

Steps:
  1. Connect display to VGA port 2. Check the display

Verification:

Output to display works

After-suspend:

True

Plugin:

manual

monitor/zapper-edid

Check if the system automatically changes the resolution based on EDID

Category ID:

monitor

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

ZAPPER_HOST

Plugin:

shell

Non-device specific networking tests

The following test units are covered in this category:

ipv6_detect

Test if the kernel is IPv6 ready

Category ID:

networking

Status:

Blocking

Purpose:

Test if the running kernel supports IPv6.

Steps:

<missing steps>

Verification:

<missing verification>

After-suspend:

True

networking/info_device__index___interface

Network Information of device {__index__} ({interface})

Unit type:

template

Category ID:

networking

Status:

Blocking

After-suspend:

True

From template:

networking/info_device__index___interface

Template resource:

device

Template filter:

device.category == ‘NETWORK’

networking/predictable_names

Verify that all network interfaces have predictable names.

Category ID:

networking

Status:

Blocking

Purpose:

Verify that all network interfaces have predictable names.

Plugin:

shell

Power Management tests

Warm reboot is tested such that the system must be able to perform the reboot command and services must be restarted such that systemctl does not identify a failed state.

Cold reboot is performed where an RTC is available (see next section). The wakealarm is used to reboot the system after a period of rest and services must be restarted such that systemctl does not identify a failed state.

The following test units are covered in this category:

power-management/cold-reboot

Cold reboot

Category ID:

power-management

Status:

Blocking

Purpose:

This test powers off the system and then powers it on using RTC

Environment variable:

COLD_REBOOT_DELAY, RTC_DEVICE_FILE

User:

root

Plugin:

shell

power-management/post-cold-reboot

Post cold reboot service check

Category ID:

power-management

Status:

Blocking

Purpose:

Check there are no failed services after the cold reboot

Plugin:

shell

power-management/post-warm-reboot

Post warm reboot service check

Category ID:

power-management

Status:

Blocking

Purpose:

Check there are no failed services after the warm reboot

Plugin:

shell

power-management/warm-reboot

Warm reboot

Category ID:

power-management

Status:

Blocking

Purpose:

This tests reboots the system using the reboot command

User:

root

Plugin:

shell

watchdog/detect

Detect the presence of a Hardware Watchdog

Category ID:

power-management

Status:

Blocking

Purpose:

<missing purpose>

Steps:

<missing steps>

Verification:

<missing verification>

Description:

<missing description>

watchdog/post-trigger-system-reset-auto

Post watchdog reset service check

Category ID:

power-management

Status:

Blocking

Purpose:

Check there are no failed services after the watchdog triggered

Plugin:

shell

watchdog/systemd-config

Check if the hardware watchdog is properly configured

Category ID:

power-management

Status:

Blocking

Purpose:

<missing purpose>

Steps:

<missing steps>

Verification:

<missing verification>

Description:

<missing description>

watchdog/trigger-system-reset-auto

Test that the watchdog module can trigger a system reset

Category ID:

power-management

Status:

Blocking

Purpose:

Ensure that the watchdog module can successfully initiate a system reset.

User:

root

Plugin:

shell

Quadrature Encoder Peripherals

The following test units are covered in this category:

qep/qep-device-driver-for-qep-device

Verify PCI Device {qep-device} is using the correct driver

Unit type:

template

Category ID:

qep

Status:

Blocking

Description:

Checks that the device exists in lspci and that the device driver associated is correct. Device ID should be 8086:{qep-device} and the driver is always intel_qep.

From template:

qep/qep-device-driver-for-qep-device

Template resource:

qep/qep-devices

qep/qep-device-node-for-qep-device

Verify device directory exists for {qep-device}

Unit type:

template

Category ID:

qep

Status:

Blocking

From template:

qep/qep-device-node-for-qep-device

Template resource:

qep/qep-devices

Real Time Clock (RTC)

The following test units are covered in this category:

rtc/battery

RTC battery tracks the time and ensures the system can wake up from power off state.

Category ID:

rtc_category

Status:

Blocking

Purpose:

RTC battery backup power can send system wakeup events.

Steps:
  1. Start the test to power off the system (wakeup scheduled in 30s).

Verification:

RTC can wake up the system successfully. environ: RTC_DEVICE_FILE

User:

root

Plugin:

user-interact

rtc/rtc_alarm_rtc

Check that RTC alarm of {rtc} works

Unit type:

template

Category ID:

rtc_category

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

rtc/rtc_alarm_rtc

Template resource:

rtc_list

rtc/rtc_clock_rtc

Check that {rtc} clock is synchronized with system clock

Unit type:

template

Category ID:

rtc_category

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

rtc/rtc_clock_rtc

Template resource:

rtc_list

rtc/rtc_number

Check the number of RTC as expected.

Category ID:

rtc_category

Status:

Blocking

Purpose:

Check the number of RTC on platform as expected.

After-suspend:

True

Environment variable:

TOTAL_RTC_NUM

User:

root

Plugin:

shell

Serial Port

Tests are carried out on ports that provide access via the Linux tty layer. The exact tests performed depend on the physical characteristics of the driver/receiver hardware. The possible tests include:

  • Ensure expected number of devices are available

  • Looped tests:

  • RS232 Ports: perform loopback test to ensure RX/TX

  • RS422/485 Ports: connect together to ensure RX/TX

  • Machine to Machine tests: confirm that a connection can be made to another

PC device and RX/TX is operational

The following test units are covered in this category:

serial/loopback-dev

Serial loopback test of {dev}

Unit type:

template

Category ID:

serial

Status:

Blocking

After-suspend:

True

From template:

serial/loopback-dev

Template resource:

serial_ports_static

serial/rs232-console

Serial debugging console is enabled and operational

Category ID:

serial

Status:

Blocking

Purpose:

Check user can log into system through serial port from another machine

Steps:
  1. Connect USB to DB9 null modem adapter cable to the serial port of the test machine. 2. Connect the cable to a USB port of another Ubuntu machine (client). 3. Install screen on the client (if not done in Prerequisite). 4. Execute the following command on the client: sudo screen /dev/ttyUSB0 5. Start the getty service on the test machine: sudo systemctl start getty@[rs232 device, e.g., /dev/ttyS0].service 6. Log into the test machine from the terminal on the client.

Verification:
  1. The output to the client is fine after the getty service is started. 2. Log into the test machine from the terminal on the client successfully.

After-suspend:

True

Plugin:

manual

Snapd

The following test units are covered in this category:

snappy/snap-install

Test the snap install command is working

Category ID:

snapd

Status:

Blocking

Purpose:

The store should contain the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap. The test makes sure this can be downloaded and installed on the system.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/snap-list

Test that the snap list command is working.

Category ID:

snapd

Status:

Blocking

Purpose:

If snap list command is working then should at least find the ubuntu-core package.

Plugin:

shell

snappy/snap-refresh-automated

Test whether the snap refresh command is working.

Category ID:

snapd

Status:

Blocking

Purpose:

The test will install the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap from the stable channel and then refreshes it to the edge channel and compares the revision before and after the refresh.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/snap-remove

Test the snap remove command is working.

Category ID:

snapd

Status:

Blocking

Purpose:

After having installed the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap, check it can be removed.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/snap-reupdate-automated

Test the snap refresh command works after blacklisting.

Category ID:

snapd

Status:

Blocking

Purpose:

Checks that the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap can be refreshed after removal of the blacklisted revision.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/snap-revert-automated

Test the snap revert command is working.

Category ID:

snapd

Status:

Blocking

Purpose:

Checks if the edge channel {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap is reverted back to the one from stable.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/test-snaps-confinement

Test all the snaps’ confinement

Category ID:

snapd

Status:

Blocking

Purpose:

Test all the snaps’ confinement, devmode, revision. Make sure the confinement is “strict”, devmode is “False”, and revision should not be sideloaded.

Plugin:

shell

snappy/test-store-config-store

Test that image is using the correct snappy store configuration.

Unit type:

template

Category ID:

snapd

Status:

Blocking

Environment variable:

STORE_ID

From template:

snappy/test-store-config-store

Template resource:

com.canonical.certification::model_assertion

snappy/test-store-install-beta

Snappy install command - beta channel store

Category ID:

snapd

Status:

Blocking

Purpose:

Test the snappy install command is able to install and remove snap in beta channel store.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/test-store-install-edge

Snappy install command - edge channel store

Category ID:

snapd

Status:

Blocking

Purpose:

Test the snappy install command is able to install a snap in the edge channel store.

Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP

User:

root

Plugin:

shell

snappy/test-system-confinement

Test if the system confinement is strict

Category ID:

snapd

Status:

Blocking

Purpose:

Test if the system confinement is “strict” If not, list the missing features

Plugin:

shell

SocketCAN interface tests

The following test units are covered in this category:

socketcan/send_packet_local_eff_virtual

Virtual CAN device support test (Local test with raw socket and EFF)

Category ID:

socketcan

Status:

Blocking

Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device

After-suspend:

True

User:

root

Plugin:

shell

socketcan/send_packet_local_eff_interface

CAN device support test for {interface} (Raw, Local, EFF)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_local_eff_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

socketcan/send_packet_local_fd_virtual

Virtual CAN device support test (Raw, Local, FD)

Category ID:

socketcan

Status:

Blocking

Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket; this is only a local test as the broadcast packet is received on the same device.

After-suspend:

True

User:

root

Plugin:

shell

socketcan/send_packet_local_fd_interface

CAN device support test for {interface} (Raw, Local, FD)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_local_fd_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

socketcan/send_packet_local_sff_virtual

Virtual CAN device support test (Raw, Local)

Category ID:

socketcan

Status:

Blocking

Purpose:

Test that the kernel supports CAN networking by sending packets to a virtual device using a raw socket, this is only a local test as the broadcast packet is received on the same device

After-suspend:

True

User:

root

Plugin:

shell

socketcan/send_packet_local_sff_interface

CAN device support test for {interface} (Raw, Local)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_local_sff_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

socketcan/send_packet_remote_eff_interface

CAN device support test {interface} (Raw, Remote, EFF)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_remote_eff_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

socketcan/send_packet_remote_fd_interface

CAN device support test {interface} (Raw, Remote, FD)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_remote_fd_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

socketcan/send_packet_remote_sff_interface

CAN device support test {interface} (Raw, Remote)

Unit type:

template

Category ID:

socketcan

Status:

Blocking

After-suspend:

True

From template:

socketcan/send_packet_remote_sff_interface

Template resource:

device

Template filter:

device.category == ‘SOCKETCAN’

TPM 2.0 (Trusted Platform Module)

On Intel and AMD x86 platforms that include TPM 2.0 compliant modules, it is required that all commands necessary to support Ubuntu’s Full Disk Encryption functionality are supported.

The following test units are covered in this category:

clevis-encrypt-tpm2/detect-ecc-capabilities

Ensure the TPM has required capabilities for clevis ECC test

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

clevis-encrypt-tpm2/detect-rsa-capabilities

Ensure the TPM has required capabilities for clevis RSA test

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

clevis-encrypt-tpm2/ecc

clevis encrypt/decrypt key ecc

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

clevis-encrypt-tpm2/precheck

clevis encrypt/decrypt precheck

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

clevis-encrypt-tpm2/rsa

clevis encrypt/decrypt key rsa

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

tpm2/fwts-event-log-dump

Dump the contents of the TPM Event Log

Category ID:

tpm2

Status:

Blocking

Purpose:

<missing purpose>

Description:

The information in the TPM Event Log can be useful in debugging problems with TPM command support and adherance to standards. This can be of particular help when checking whether a device can support Ubuntu Core Full Disk Encryption.

User:

root

Plugin:

attachment

USB tests

USB 2.0

USB storage devices must work on all available USB ports. USB Human Interface Devices (HID), specifically keyboard or mouse, should be working properly on any USB port.

USB 3.0

USB storage devices must work on all available USB ports. USB Human Interface Devices (HID), specifically keyboard or mouse, should be working properly on any USB port.

USB Type C (USB 3.1)

USB Type C (USB 3.1) supports various types of devices (e.g. Video, Power) through the use of adapters or peripherals. The following adapters/peripherals should work:

  • Storage devices

  • Keyboard or mouse (basic functionality)

  • When DisplayPort over USB Type-C is advertised:

  • Display hot plugging and the following display are required to work:

    mirrored, extended, internal only, external only.

  • Audio output needs to be undistorted over this port.

The following test units are covered in this category:

usb-c-otg/g_ether

Check DUT can be detected as USB ethernet device

Category ID:

usb

Status:

Blocking

Purpose:

<missing purpose>

Steps:
  1. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT. 2. Press Enter to configure the IP address for the USB OTG ethernet interface on the DUT. (IP will be 192.168.9.1/24) 3. On the host, configure the IP address for the USB OTG ethernet interface. It should be in the same subnet as the DUT. (e.g., 192.168.9.2/24) 4. Try to ping between the host and DUT.

Verification:

The host and DUT can ping each other.

Description:

<missing description>

After-suspend:

True

User:

root

Plugin:

user-interact-verify

usb-c-otg/g_ether-cleanup

Cleanup USB OTG ethernet interface setup after ethernet device test

Category ID:

usb

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

usb-c-otg/g_mass_storage

Check DUT can be detected as a mass storage device

Category ID:

usb

Status:

Blocking

Purpose:

Check that after connecting the device under test (DUT) to another device (host), the DUT can be detected as a mass storage device by the host.

Steps:
  1. Press Enter to setup a 16 MB FAT32 image on the device. 2. From an Ubuntu host, connect a cable to the USB OTG port (it’s the USB Type-C connector) of the DUT.

Verification:

The host detects and mounts the mass storage device. It has read and write permissions on it.

After-suspend:

True

User:

root

Plugin:

user-interact-verify

usb-c-otg/g_mass_storage-cleanup

Cleanup mass storage setup after mass storage device test

Category ID:

usb

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

usb-c-otg/g_serial

Check if USB OTG can work as a serial port.

Category ID:

usb

Status:

Blocking

Purpose:

Check if USB OTG can work as a serial port.

Steps:
  1. Press enter to probe USB OTG g_serial 2. Connect a USB cable from an Ubuntu host to the USB OTG (it’s the USB Type C connector) port on DUT. 3. On the host, check if ttyACM0 has been probed $ ls /dev/ttyACM0 4. On the host, listen to the ttyACM0 port as the receiving side. $ sudo su $ cat /dev/ttyACM0 5. On DUT, send a string via /dev/ttyGS0 as the sending side. $ sudo su $ echo 123 > /dev/ttyGS0 6. Check if the string “123” was received on the host side. 7. Check the string sending from the host to DUT by repeating steps 4-6, but swap the send and receive sides.

Verification:

Does string send and receive function correctly?

After-suspend:

True

User:

root

Plugin:

user-interact-verify

usb-c-otg/g_serial-cleanup

Cleanup USB OTG serial interface setup after serial device test

Category ID:

usb

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

usb-c/c-to-a-adapter/hid

USB HID work on USB Type-C port using a “USB Type-C to Type-A” adapter

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port using a “USB Type-C to Type-A” adapter

Steps:
  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port using a “USB Type-C to Type-A” adapter 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text

Verification:

Did the device work as expected?

After-suspend:

True

Plugin:

manual

usb-c/c-to-a-adapter/insert

Verify that the system detects a USB 3 storage device using a “Type-C to Type-A” adapter.

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that the system correctly detects the insertion of a USB 3 storage device in a USB Type-C connector using a “Type-C to Type-A” adapter. NOTE: Make sure the USB storage device has a partition before starting the test.

Steps:
  1. Commence the test 2. Connect a USB 3 storage device to a USB Type-C port using a “Type-C to Type-A” adapter. 3. Do not unplug the device after the test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb-c/c-to-a-adapter/remove

Check the detection of a USB 3 storage device removal, connected via a “Type-C to Type-A” adapter.

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a USB 3 storage device connected to a USB Type-C port using a “USB Type-C to Type-A” adapter.

Steps:
  1. Begin the test 2. Disconnect a USB 3 storage device from a USB Type-C port using a “Type-C to Type-A” adapter

Verification:

The verification of this test is automated. Do not modify the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb-c/c-to-a-adapter/storage-automated

Automate the execution process post usb-c/c-to-a-adapter/insert test.

Category ID:

usb

Status:

Blocking

Purpose:

This test is automated and executes after the usb-c/c-to-a-adapter/insert test is run.

After-suspend:

True

User:

root

Plugin:

shell

usb-c/hid

USB HID work on USB Type-C port

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port

Steps:
  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text

Verification:

Did the device work as expected?

After-suspend:

True

Plugin:

manual

usb-c/insert

USB 3.0 storage device insertion detected on USB Type-C port

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that the system correctly detects the insertion of a USB 3.0 storage device in a USB Type-C connector. NOTE: Make sure the USB storage device has a partition before starting the test.

Steps:
  1. Commence the test. 2. Connect a USB 3.0 storage device to a USB Type-C port. 3. Do not unplug the device after the test.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb-c/remove

USB 3.0 storage device removal detected on USB Type-C port

Category ID:

usb

Status:

Blocking

Purpose:

This test will check that the system correctly detects the removal of a USB 3.0 storage device connected to a USB Type-C port.

Steps:
  1. Commence the test 2. Connect a USB 3.0 storage device to a USB Type-C port 3. Disconnect the USB 3.0 storage device from the USB Type-C port

Verification:

The system should automatically detect the removal of the USB 3.0 storage device. The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb-c/storage-automated

USB 3.0 storage device read & write works on USB Type-C port

Category ID:

usb

Status:

Blocking

Purpose:

This test is automated and executes after the usb-c/insert test is run.

After-suspend:

True

User:

root

Plugin:

shell

usb/hid

USB keyboard works

Category ID:

usb

Status:

Blocking

Purpose:

Check USB input device works

Steps:
  1. Connect USB keyboard 2. Input something with USB keyboard

Verification:

What was just input is displayed correctly

After-suspend:

True

Plugin:

manual

usb/insert

USB 2.0 storage device insertion detected

Category ID:

usb

Status:

Blocking

Purpose:

Check system can detect USB 2.0 storage when inserted. NOTE: Make sure the USB storage device has a partition before starting the test.

Steps:
  1. Press continue 2. Wait until the message “INSERT NOW” is printed on the screen 3. Connect USB 2.0 storage device

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb/remove

USB 2.0 storage device removal detected

Category ID:

usb

Status:

Blocking

Purpose:

Check system can detect removal of a USB 2.0 storage device

Steps:
  1. Press continue 2. Wait until the message “REMOVE NOW” is printed on the screen 3. Disconnect USB 2.0 storage device

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb/storage-automated

USB 2.0 storage device read & write works

Category ID:

usb

Status:

Blocking

Purpose:

Check system can read/write to USB 2.0 storage correctly

After-suspend:

True

User:

root

Plugin:

shell

usb/storage-detect

Detect storage partitions on a device on the USB bus

Category ID:

usb

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

usb/zapper-usb-insert-port_alias

Zapper-driven USB insertion test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb/zapper-usb-insert-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’

usb/zapper-usb-remove-port_alias

Zapper-driven USB removal test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb/zapper-usb-remove-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘2’

usb3/insert

USB 3.0 storage device insertion detected

Category ID:

usb

Status:

Blocking

Purpose:

Check system can detect insertion of a USB 3.0 storage device. NOTE: Make sure the USB storage device has a partition before starting the test.

Steps:
  1. Press continue. 2. Wait until the message “INSERT NOW” is displayed on the screen. 3. Connect USB 3.0 storage device.

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb3/remove

USB 3.0 storage device removal detected

Category ID:

usb

Status:

Blocking

Purpose:

Check system can detect removal of a USB 3.0 storage device

Steps:
  1. Press continue 2. Wait until the message “REMOVE NOW” is printed on the screen 3. Disconnect USB 3.0 storage device

Verification:

The verification of this test is automated. Do not change the automatically selected result.

After-suspend:

True

User:

root

Plugin:

user-interact

usb3/storage-automated

USB 3.0 storage device read & write works

Category ID:

usb

Status:

Blocking

Purpose:

Check system can read/write to USB 3.0 storage devices correctly

After-suspend:

True

User:

root

Plugin:

shell

usb3/zapper-usb-insert-port_alias

Zapper-driven USB 3.0 insertion test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb3/zapper-usb-insert-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’

usb3/zapper-usb-remove-port_alias

Zapper-driven USB 3.0 removal test on port {{ port_alias }}

Unit type:

template

Category ID:

usb

Status:

Blocking

Description:

<missing description>

Environment variable:

ZAPPER_HOST

From template:

usb3/zapper-usb-remove-port_alias

Template resource:

zapper_capabilities

Template filter:

zapper_capabilities.capability == ‘USB hotplug’ and zapper_capabilities.usb_version == ‘3’

Wi-Fi access point

The following test units are covered in this category:

wireless/nmcli_wifi_ap_a_interface

Create 802.11a Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

wireless/nmcli_wifi_ap_a_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/nmcli_wifi_ap_bg_interface

Create 802.11b/g Wi-Fi AP on {{ interface }} using NetworkManager

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

From template:

wireless/nmcli_wifi_ap_bg_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_open_b_no_sta_interface_auto

Create open 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

Environment variable:

WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_open_b_no_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_open_b_no_sta_interface_manual

Create open 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

From template:

wireless/wifi_ap_open_b_no_sta_interface_manual

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_open_g_no_sta_interface_auto

Create an open 802.11g Wi-Fi AP on {interface} with no STA connected.

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

Environment variable:

WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_open_g_no_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_open_g_no_sta_interface_manual

Create open 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

From template:

wireless/wifi_ap_open_g_no_sta_interface_manual

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_setup_wizard_interface_auto

Create Access Point on {interface} using wifi-ap.setup-wizard

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

Environment variable:

WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_setup_wizard_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_b_no_sta_interface_auto

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

Environment variable:

WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_wpa_b_no_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_b_no_sta_interface_manual

Create WPA2 802.11b Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

From template:

wireless/wifi_ap_wpa_b_no_sta_interface_manual

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_b_with_sta_interface_auto

Create a WPA2 802.11b Wi-Fi Access Point on {interface} with active STA

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

Description:

<missing description>

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, OPEN_BG_SSID, WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_wpa_b_with_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_g_no_sta_interface_auto

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

Environment variable:

WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_wpa_g_no_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_g_no_sta_interface_manual

Create WPA2 802.11g Wi-Fi AP on {interface} with no STA (Manual)

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

After-suspend:

True

From template:

wireless/wifi_ap_wpa_g_no_sta_interface_manual

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wifi_ap_wpa_g_with_sta_interface_auto

Create WPA2 802.11g Wi-Fi Access Point on {interface} with active STA

Unit type:

template

Category ID:

wifi_ap

Status:

Blocking

Description:

<missing description>

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, OPEN_BG_SSID, WIFI_AP_SETUPTIME

From template:

wireless/wifi_ap_wpa_g_with_sta_interface_auto

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless networking tests

Wi-Fi interfaces are tested for connection to access points configured for 802.11 b/g/n/ac/ax protocols.

The following test units are covered in this category:

wireless/check_iwlwifi_microcode_crash_interface

Check there have been no iwlwifi crashes

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/check_iwlwifi_microcode_crash_interface

Template resource:

device

Template filter:

device.driver == ‘iwlwifi’

wireless/detect

Detect if at least one Wireless LAN device is detected

Category ID:

wireless

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

wireless/wireless_connection_open_ac_nm_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_ac_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_ac_np_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_AC_SSID

From template:

wireless/wireless_connection_open_ac_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_ax_nm_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_ax_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_ax_np_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_AX_SSID

From template:

wireless/wireless_connection_open_ax_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_be_nm_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_be_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_be_np_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_BE_SSID

From template:

wireless/wireless_connection_open_be_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_bg_nm_interface

Connect to an unencrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_bg_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_bg_np_interface

Connect to unencrypted 802.11b/g Wi-Fi network on {{ interface }} using netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_BG_SSID

From template:

wireless/wireless_connection_open_bg_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_n_nm_interface

Connect to an unencrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_open_n_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_open_n_np_interface

Connect to unencrypted 802.11n Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, OPEN_N_SSID

From template:

wireless/wireless_connection_open_n_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa3_ax_nm_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa3_ax_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa3_ax_np_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA3_AX_PSK, WPA3_AX_SSID

From template:

wireless/wireless_connection_wpa3_ax_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa3_be_nm_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa3_be_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa3_be_np_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA3_BE_PSK, WPA3_BE_SSID

From template:

wireless/wireless_connection_wpa3_be_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_ac_nm_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_ac_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_ac_np_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_AC_PSK, WPA_AC_SSID

From template:

wireless/wireless_connection_wpa_ac_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_ax_nm_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_ax_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_ax_np_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_AX_PSK, WPA_AX_SSID

From template:

wireless/wireless_connection_wpa_ax_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_be_nm_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_be_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_be_np_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_BE_PSK, WPA_BE_SSID

From template:

wireless/wireless_connection_wpa_be_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_bg_nm_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_bg_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_bg_np_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }} - netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_BG_PSK, WPA_BG_SSID

From template:

wireless/wireless_connection_wpa_bg_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_n_nm_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_connection_wpa_n_nm_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_connection_wpa_n_np_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }} using netplan

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, NET_DRIVER_INFO, WPA_N_PSK, WPA_N_SSID

From template:

wireless/wireless_connection_wpa_n_np_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

wireless/wireless_scanning_interface

Test system can discover Wi-Fi networks on {{ interface }}

Unit type:

template

Category ID:

wireless

Status:

Blocking

After-suspend:

True

From template:

wireless/wireless_scanning_interface

Template resource:

device

Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Wireless Wide Area Network

WWAN interfaces are tested for connection to 3G/4G/LTE services.

The following test units are covered in this category:

wwan/3gpp-scan-manufacturer-model-hw_id-auto

Scan for available 3GPP networks with the modem

Unit type:

template

Category ID:

wwan

Status:

Blocking

Description:

Scan for available 3GPP networks with the target modem

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH

From template:

wwan/3gpp-scan-manufacturer-model-hw_id-auto

Template resource:

wwan_resource

wwan/check-sim-present-manual

Check if a SIM card is present in a slot connected to the modem

Category ID:

wwan

Status:

Blocking

Purpose:

Check if a SIM card is present in a slot connected to the modem

Steps:
  1. Open another terminal on SUT (or press Ctrl+Z to suspend Checkbox) 2. Run sudo mmcli -m 0 | grep SIM 3. Run fg to jump back to Checkbox (if you’re running in the same terminal)

Verification:

Check the output. Was the SIM information printed?

After-suspend:

True

Plugin:

manual

wwan/check-sim-present-manufacturer-model-hw_id-auto

Check if a SIM card is present in a slot connected to the modem

Unit type:

template

Category ID:

wwan

Status:

Blocking

Description:

Check if a SIM card is present in a slot connected to the modem

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH

From template:

wwan/check-sim-present-manufacturer-model-hw_id-auto

Template resource:

wwan_resource

wwan/detect

Identify if WWAN module is missing

Category ID:

wwan

Status:

Blocking

Purpose:

Tests that there is a WWAN module present and indicates that testing of it should follow.

After-suspend:

True

User:

root

Plugin:

shell

wwan/detect-manual

Check if WWAN module is available

Category ID:

wwan

Status:

Blocking

Purpose:

Check if WWAN module is available and ready to be used.

Steps:
  1. Open another terminal on SUT (or press ctrl+z to suspend Checkbox) 2. Run sudo mmcli -L command. 3. Run fg to jump back to checkbox (if you’re running in the same terminal)

Verification:

Check the output. Was the modem listed?

After-suspend:

True

Plugin:

manual

wwan/gsm-connection-interrupted-manual

Verify a GSM broadband connection can be reconnected after the signal is lost

Category ID:

wwan

Status:

Blocking

Purpose:

Ensure that the data connection can be revived after losing signal

Steps:
  1. Open another terminal on SUT (or press ctrl+z to suspend Checkbox). 2. Run sudo nmcli c add type gsm ifname {{ __checkbox_env__.get(“WWAN_CONTROL_IF”, “<device name>”) }} con-name GSM apn {{ __checkbox_env__.get(“WWAN_APN”, “<your.carrier.com>”) }} {%- if “WWAN_CONTROL_IF” not in __checkbox_env__ %} Replacing <your.carrier.com> with the appropriate APN name, and <device name> with the appropriate device (device can be found by running sudo nmcli d). {%- endif %} {%- if “WWAN_NET_IF” not in __checkbox_env__ %} 3. Ping Google (Run: ping -I <ifname> 8.8.8.8 -c 5). {%- else %} 3. Ping Google (Run: ping -I {{ __checkbox_env__.get(“WWAN_NET_IF”) }} 8.8.8.8 -c 5). {%- endif %} 4. Place the system in a Faraday bag. 5. Ping Google (and verify that the connection no longer works). 6. Remove the Faraday bag. {%- if “WWAN_NET_IF” not in __checkbox_env__ %} 7. Ping Google (Run: ping -I <ifname> 8.8.8.8 -c 5). {%- else %} 7. Ping Google (Run: ping -I {{ __checkbox_env__.get(“WWAN_NET_IF”) }} 8.8.8.8 -c 5). {%- endif %} 8. Disconnect from the network (Run: sudo nmcli r wwan off). 9. Run fg to jump back to Checkbox (if you’re running in the same terminal).

Verification:

Was the connection revived after removing the Faraday bag?

After-suspend:

True

Plugin:

manual

wwan/gsm-connection-manufacturer-model-hw_id-auto

Verify a GSM broadband modem can create a data connection

Unit type:

template

Category ID:

wwan

Status:

Blocking

After-suspend:

True

Environment variable:

LD_LIBRARY_PATH, WWAN_APN, WWAN_CONTROL_IF, WWAN_NET_IF, WWAN_SETUPTIME

From template:

wwan/gsm-connection-manufacturer-model-hw_id-auto

Template resource:

wwan_resource

Non-blocking

Informational tests

The following test units are covered in this category:

manifest

Hardware Manifest

Category ID:

info

Status:

Non-blocking

Purpose:

<missing purpose>

Description:

This job loads the hardware manifest and exposes it as a resource.

Plugin:

resource

client-cert-iot-desktop-24-04

Note

The certification tests presented in this document are validated by Checkbox version 4.2.0.dev76.

Blocking

Advanced Configuration and Power Interface

The following test units are covered in this category:

acpi/oem_osi

Test ACPI OEM _OSI strings

Category ID:

acpi

Status:

Blocking

Purpose:

This checks if the deprecated OEM _OSI strings are still used by checking the ACPI DSDT and SSDT tables

User:

root

Plugin:

shell

Audio tests

Output needs to be undistorted between 0%-100%. Output lines tested:

  • Internal speakers

  • 3.5mm headphones

  • HDMI audio output

  • DisplayPort audio output

Input needs to be recorded undistorted between 0%-100%. Input lines tested:

  • Internal microphone

  • 3.5mm microphone

Plug detection: when a new audio line input or output is plugged in the system, it needs to be recognized.

The following test units are covered in this category:

audio/alsa-loopback-automated

Captured sound matches played one (automated)

Category ID:

audio

Status:

Blocking

Purpose:

Check if sound that is ‘hearable’ by capture device. This test is intended mostly for devices without internal speakers/microphones, but with a line-in and a line-out ports to which a loopback cable can be connected.

After-suspend:

True

Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH, LD_LIBRARY_PATH

User:

root

Plugin:

shell

audio/alsa-playback

Playback works

Category ID:

audio

Status:

Blocking

Purpose:

Check if sound is played through ALSA on the default output

Steps:
  1. Make sure speakers or headphones are connected to the device 2. Commence the test

Verification:

Did you hear the sound?

After-suspend:

True

Environment variable:

ALSADEVICE, ALSA_CONFIG_PATH

User:

root

Plugin:

user-interact-verify

audio/detect-capture-devices

Check that at least one audio capture device exists

Category ID:

audio

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

audio/detect-playback-devices

Check that at least one audio playback device exists

Category ID:

audio

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

After-suspend:

True

Plugin:

shell

Bluetooth - BlueZ Self Tests

The following test units are covered in this category:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test

BlueZ-{bluez-internal-bnep-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

After-suspend:

True

From template:

bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test

Template resource:

bluez-internal-bnep-tests

bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test

BlueZ-{bluez-internal-hci-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

Description:

Runs a specific test from the hci test suite

After-suspend:

True

From template:

bluetooth/bluez-internal-hci-tests_bluez-internal-hci-test

Template resource:

bluez-internal-hci-tests

bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test

BlueZ-{bluez-internal-rfcomm-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

After-suspend:

True

From template:

bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test

Template resource:

bluez-internal-rfcomm-tests

bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test

BlueZ-{bluez-internal-uc-test}

Unit type:

template

Category ID:

bluetooth_bluez5_selftests

Status:

Blocking

After-suspend:

True

From template:

bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test

Template resource:

bluez-internal-uc-tests

Bluetooth tests

Bluetooth LE (Smart and Smart Ready) is tested for device scanning and pairing. Apart from pairing, several profiles are specifically tested and required:

  • Eddystone Beacon

  • HID Over GATT Profile (HOGP), Low-Energy keyboard or mouse with basic functionality

The following test units are covered in this category:

bluetooth/audio-a2dp

Verify Bluetooth device’s High Fidelity Playback (A2DP) capability.

Category ID:

bluetooth

Status:

Blocking

Purpose:

This test will check the High Fidelity Playback (A2DP) capability of your Bluetooth device, to see if you can hear audio from it.

Steps:
  1. Enable and pair the Bluetooth headset 2. Click “Test” to play a brief tone on your Bluetooth device, if it failed to set the Mode to A2DP, please select the device and change it manually in the “Sound Settings”

Verification:

Did you hear the tone?

After-suspend:

True

Plugin:

user-interact-verify

bluetooth/audio_record_playback

Verify Bluetooth HSP/HFP profile capability for recording and playback.

Category ID:

bluetooth

Status:

Blocking

Purpose:

This test will check the Headset Head Unit (HSP/HFP) capability of your Bluetooth device, to check if you can record sounds.

Steps:
  1. Enable and pair the bluetooth headset. 2. Click “Test”, then speak into your Bluetooth microphone. 3. After a few seconds, your speech will be played back to you.

Verification:

Did you hear your speech played back?

After-suspend:

True

Plugin:

user-interact-verify

bluetooth/bluetooth_obex_send

Bluetooth OBEX send

Category ID:

bluetooth

Status:

Blocking

Purpose:

This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable

After-suspend:

True

Environment variable:

BTDEVADDR, PLAINBOX_PROVIDER_DATA

Plugin:

shell

bluetooth/bluez-controller-detect

Check bluez lists a controller if rfkill detects one

Category ID:

bluetooth

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

User:

root

Plugin:

shell

bluetooth/detect

Make sure at least one bluetooth device is detected

Category ID:

bluetooth

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Plugin:

shell

bluetooth/detect-output

Store Bluetooth device information for reports.

Category ID:

bluetooth

Status:

Blocking

Purpose:

Automated test to store Bluetooth device information in the Checkbox report

After-suspend:

True

Plugin:

shell

bluetooth/zapper-a2dp

Check if the system can connect to a Bluetooth speaker using A2DP profile

Category ID:

bluetooth

Status:

Blocking

Purpose:

<missing purpose>

Description:

<missing description>

Environment variable:

ZAPPER_HOST

Plugin:

shell

bluetooth4/HOGP-keyboard

Verify HOGP keyboard functionality with Bluetooth Smart.

Category ID:

bluetooth

Status:

Blocking

Purpose:

This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart keyboard.

Steps:
  1. Enable a Bluetooth Smart keyboard, and put it into pairing mode. 2. Commence the test to do the auto-pairing, you will be asked to select the targeting keyboard from the list. 3. After it’s paired and connected, enter some text with your keyboard.

Verification:

Did the Bluetooth Smart keyboard work as expected?

After-suspend:

True

Plugin:

user-interact-verify

bluetooth4/HOGP-mouse

Test the functionality of a Bluetooth Smart mouse using HID Over GATT Profile.

Category ID:

bluetooth

Status:

Blocking

Purpose:

This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart mouse.

Steps:
  1. Enable a Bluetooth Smart mouse, and put it into pairing mode. 2. Commence the test to do the auto-pairing; you will be asked to select the targeting mouse from the list. 3. After it’s paired and connected, perform actions such as moving the pointer, right and left button clicks, and double clicks.

Verification:

Did the Bluetooth Smart mouse work as expected?

After-suspend:

True

Plugin:

user-interact-verify

bluetooth4/beacon_eddystone_url_interface

Test system can get beacon EddyStone URL advertisements on the {interface} adapter

Unit type:

template

Category ID:

bluetooth

Status:

Blocking

Description:

<missing description>

After-suspend:

True

From template:

bluetooth4/beacon_eddystone_url_interface

Template resource:

device

Template filter:

device.category == ‘BLUETOOTH’

Camera tests

The following test units are covered in this category:

camera/multiple-resolution-images-rpi-attachment_name

Attach an image from the multiple resolution images test on rpi

Unit type:

template

Category ID:

camera

Status:

Blocking

Description:

This test will attach one of the images used for the multiple resolution images test.

From template:

camera/multiple-resolution-images-rpi-attachment_name

Template resource:

device

Template filter:

device.category == ‘MMAL’ and device.name != ‘’

camera/multiple-resolution-images-rpi_name

Webcam multiple resolution capture test for Pi Camera

Unit type:

template

Category ID:

camera

Status:

Blocking

From template:

camera/multiple-resolution-images-rpi_name

Template resource:

device

Template filter:

device.category == ‘MMAL’ and device.name != ‘’

camera/multiple-resolution-images_name

Webcam multiple resolution capture test for {product_slug}

Unit type:

template

Category ID:

camera

Status:

Blocking

After-suspend:

True

From template:

camera/multiple-resolution-images_name

Template resource:

device

Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

camera/roundtrip-qrcode_name

Test video output and camera {{ name }} by displaying and reading a QR code

Unit type:

template

Category ID:

camera

Status:

Blocking

From template:

camera/roundtrip-qrcode_name

Template resource:

device

Template filter:

device.category in (‘CAPTURE’, ‘MMAL’) and device.name != ‘’

CPU tests

x86_64 and ARM processors are tested to ensure proper functionality. We will test specific features as:

  • CPU’s performance states (frequency up and down in runtime)

  • CPU’s sleep states (cpu on and off in runtime)

  • Running CPU at its maximum frequency

We will also include a general stress test performed for 120 minutes to verify that the system can handle a sustained high load for a period of time. This test uses the tool “stress-ng” available in the Universe repositories.

For Intel CPU’s, the IPDT (Intel Processor Diagnostic Tool) test suite will be run. The diagnostic checks for brand identification, verifies the processor operating frequency, tests specific processor features, and performs a stress test on the processor.

The following test units are covered in this category:

cpu/arm64_vfp_support_platform

Validate that the Floating Point Unit is running on {platform} device

Unit type:

template

Category ID:

cpu

Status:

Blocking

Description:

Validate that the Floating Point Unit is running on {platform} device.

From template:

cpu/arm64_vfp_support_platform

Template resource:

cpuinfo

Template filter:

cpuinfo.platform == ‘aarch64’

cpu/armhf_vfp_support_platform

Validate that the Vector Floating Point Unit is running on {platform} device

Unit type:

template

Category ID:

cpu

Status:

Blocking

Description:

Validate that the Vector Floating Point Unit is running on {platform} device.

From template:

cpu/armhf_vfp_support_platform

Template resource:

cpuinfo

Template filter:

cpuinfo.platform == ‘armv7l’

cpu/clocktest

Tests the CPU for clock jitter

Category ID:

cpu

Status:

Blocking

Purpose:

Runs a test for clock jitter on SMP machines.

After-suspend:

True

Plugin:

shell

cpu/cstates

Run C-States tests

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Uses the Firmware Test Suite (fwts) to test the power saving states of the CPU.

Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP

User:

root

Plugin:

shell

cpu/cstates_results.log

Attach C-States test log

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Attaches the FWTS desktop diagnosis results log to the submission.

Plugin:

attachment

cpu/maxfreq_test

Test that the CPU can run at its max frequency

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.

Environment variable:

LD_LIBRARY_PATH, SNAP

User:

root

Plugin:

shell

cpu/maxfreq_test-log-attach

Attach CPU max frequency log

Category ID:

cpu

Status:

Blocking

Purpose:

Attaches the log generated by cpu/maxfreq_test to the results submission.

Plugin:

attachment

cpu/offlining_test

Test offlining of each CPU core

Category ID:

cpu

Status:

Blocking

Purpose:

Attempts to offline each core in a multicore system.

After-suspend:

True

User:

root

Plugin:

shell

cpu/scaling_test

Test the CPU scaling capabilities

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Use Firmware Test Suite (fwts cpufreq) to test the scaling capabilities of the CPU.

Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP

User:

root

Plugin:

shell

cpu/scaling_test-log-attach

Attach CPU scaling capabilities log

Category ID:

cpu

Status:

Blocking

Purpose:

Attaches the log generated by cpu/scaling_test to the results submission.

Plugin:

attachment

cpu/topology

Check CPU topology for accuracy between proc and sysfs

Category ID:

cpu

Status:

Blocking

Purpose:

<missing purpose>

Description:

Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.

After-suspend:

True

Plugin:

shell

Disk tests

The following test units are covered in this category:

disk/check-software-raid

Validate the configuration of software RAID devices are expected

Category ID:

disk

Status:

Blocking

Purpose:

<missing purpose>

Description:

Examine the system to detect Software RAID devices are created and the RAID mode are expected the SOFTWARE_RAID_LEVEL variable is needed for this tests. e.g. SOFTWARE_RAID_LEVEL=”raid0 raid1 raid5”

Environment variable:

SOFTWARE_RAID_LEVEL

User:

root

Plugin:

shell

disk/detect

Gathers information about each disk detected

Category ID:

disk

Status:

Blocking

Purpose:

Uses lsblk to gather information about each disk detected on the system under test.

After-suspend:

True

Plugin:

shell

disk/encryption/check-fde-tpm

Disk decryption after TPM change

Category ID:

disk

Status:

Blocking

Purpose:

The device partition is encrypted using TPM master key. To unseal the master key from TPM, PCR7 (Platform Configuration Register 7) needs to be identical to the value it had when the master key was sealed into TPM. Every time the device boots, it checks PCR7 to unseal TPM and retrieves master key from TPM to decrypt its data partition. If TPM PCR7 is modified (e.g. by flashing the BIOS), the device won’t be able to get the master key and decrypt its data partition. This test verifies the system’s resilience against unauthorized modifications by ensuring it cannot boot if the PCR7 value is altered.

Steps:

NOTE: YOU’LL HAVE TO RE-INSTALL THE IMAGE AFTER THIS TEST. 1. Install the image and make sure it boots and you can log in. 2. Ensure the BIOS is set up correctly (e.g., TPM enabled, UEFI boot mode). 3. Depending on the DUT, choose one of these methods to clean TPM: a. Turn the device off and upgrade/downgrade the BIOS or modify Secure Boot state. b. Clean TPM via BIOS menu. c. Install checkbox, execute “checkbox-[project name].checkbox-cli run com.canonical.certification::tpm2.0_3.0.4/tpm2_takeownership”. 4. Start or reboot the device.

Verification:

Mark this test as “Passed” if the device cannot boot anymore.

Plugin:

manual

disk/encryption/detect

Test that Full Disk Encryption is in use

Category ID:

disk

Status:

Blocking

Purpose:

Examine the system to detect if one of the standard full disk encryption implementations is in use

User:

root

Plugin:

shell

disk/read_performance_name

Disk performance test for {product_slug}

Unit type:

template

Category ID:

disk

Status:

Blocking

After-suspend:

True

Environment variable:

DISK_MDADM_READ_PERF, DISK_NVME_READ_PERF, DISK_READ_PERF

From template:

disk/read_performance_name

Template resource:

device

Template filter:

device.category == ‘DISK’

disk/stats_name

Disk statistics for {product_slug}

Unit type:

template

Category ID:

disk

Status:

Blocking

After-suspend:

True

From template:

disk/stats_name

Template resource:

device

Template filter:

device.category == ‘DISK’ and device.name != ‘’

disk/storage_device_name

Disk I/O stress test for {product_slug}

Unit type:

template

Category ID:

disk

Status:

Blocking

After-suspend:

True

From template:

disk/storage_device_name

Template resource:

device

Template filter:

device.category == ‘DISK’

thunderbolt3/daisy-chain

Daisy-chain testing for Thunderbolt 3 storage and display device

Category ID:

disk

Status:

Blocking

Purpose:

This test will check if your system can support daisy-chaining of a storage and a monitor over Thunderbolt 3 port

Steps:
  1. Connect your Thunderbolt monitor to your system 2. Connect and mount your Thunderbolt HDD to another Thunderbolt 3 port of the monitor (you can do this with HDD first as well) 3. Click ‘Test’ to perform the storage test on the Thunderbolt HDD

Verification:
  1. The verification for storage is automated, please select the result combined with the result for the display. 2. Was the desktop displayed correctly on the Thunderbolt-connected screen?

After-suspend:

True

User:

root

Plugin:

user-interact-verify

thunderbolt3/insert

Storage insert detection on Thunderbolt 3 port

Category ID:

disk

Status:

Blocking

Purpose:

This test will check if the insertion of a Thunderbolt 3 HDD could be detected

Steps:
  1. Click ‘Test’ to begin the test. This test will timeout and fail if the insertion has not been detected within 40 seconds. 2. Plug a Thunderbolt HDD into an available Thunderbolt 3 port, if it’s not mounted automatically, please click the HDD icon to mount it.

Verification:

The verification of this test is automated. Do not change the automatically selected result

After-suspend:

True

User:

root

Plugin:

user-interact

thunderbolt3/remove

Storage removal detection on Thunderbolt 3 port

Category ID:

disk

Status:

Blocking

Purpose:

This test will check if the system can detect the removal of a Thunderbolt 3 HDD

Steps:
  1. Click ‘Test’ to begin the test. This test will timeout and fail if the removal has not been detected within 20 seconds. 2. Remove the previously attached Thunderbolt HDD from the Thunderbolt port.

Verification:

The verification of this test is automated. Do not change the automatically selected result

After-suspend:

True

User:

root

Plugin:

user-interact

thunderbolt3/storage-test

Storage test on Thunderbolt 3

Category ID:

disk

Status:

Blocking

Purpose:

This is an automated test which performs read/write operations on an attached Thunderbolt HDD

After-suspend:

True

User:

root

Plugin:

shell

Docker containers

The following test units are covered in this category:

docker/build-single_arch

Test docker build with a single container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/build-single_arch

Template resource:

docker_resource

docker/commit_arch

Test docker commit a change to a single container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/commit_arch

Template resource:

docker_resource

docker/compose-and-basic_arch

Test docker compose and basic command

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/compose-and-basic_arch

Template resource:

docker_resource

docker/compose-restart_arch

Test compose a container with restart policy applied

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/compose-restart_arch

Template resource:

docker_resource

docker/compose-single_arch

Test docker compose with a single container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/compose-single_arch

Template resource:

docker_resource

docker/copy_arch

Test copy a file bwtween a container and local filesystem

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/copy_arch

Template resource:

docker_resource

docker/deploy-registry_arch

Deploy a registry server and run it on localhost

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/deploy-registry_arch

Template resource:

docker_resource

docker/diff_arch

Test changes to files in Ubuntu container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/diff_arch

Template resource:

docker_resource

docker/export-and-import_arch

Test docker import and export a docker container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/export-and-import_arch

Template resource:

docker_resource

docker/info

Display system-wide information about docker

Category ID:

docker

Status:

Blocking

Purpose:

<missing purpose>

Steps:

<missing steps>

Verification:

<missing verification>

Description:

<missing description>

User:

root

docker/inspect_arch

Test query low-level information on a docker object

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/inspect_arch

Template resource:

docker_resource

docker/interative_arch

Test an interactive shell in Ubuntu container

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/interative_arch

Template resource:

docker_resource

docker/kill_arch

Test killing containers

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/kill_arch

Template resource:

docker_resource

docker/restart-always_arch

Test container restart policy with always applied

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/restart-always_arch

Template resource:

docker_resource

docker/restart-on-failure_arch

Test container restart policy with on_failure applied

Unit type:

template

Category ID:

docker

Status:

Blocking

Description:

<missing description>

From template:

docker/restart-on-failure_arch

Template resource:

docker_resource

docker/run_arch

Download and run ubuntu container