Ubuntu Desktop Certified Hardware Coverage for 24.04 LTS

Introduction

This document lists the coverage for certification of Ubuntu Desktop 24.04 LTS.

For each test job, one of the following certification statuses is specified:

Blocking

Features that are required for certification. If any of the blocking tests fails, the certification will fail.

Non-blocking

Features that are tested but not mandatory for certification. Failure in non-blocking tests will not prevent certification. However, a note will be added to the certificate to inform potential customers or users.

Note

Only categories of hardware are tested and not specific types of hardware. For example, tests are run to verify USB controllers work, but the type of peripheral(s) used during those tests are not specified.

Coverage is flexible based on customer requirements (for example, if a device’s use cases don’t require LEDs, then LEDs can be non-blocking)

Full test descriptions can be found in Canonical certification site for partners:

https://certification.canonical.com

client-cert-desktop-24-04

Note

The certification tests presented in this document are validated by Checkbox version 4.2.0.dev76.

Blocking

Audio tests

Output sound needs to be undistorted between 0%-100%. Output lines tested:

  • Internal speakers

  • 3.5mm headphones

  • 3.5mm headset

  • HDMI audio output

  • DisplayPort audio output

  • Thunderbolt 3 audio output

Input needs to be recorded undistorted between 0%-100%. Input lines tested:

  • Internal microphone

  • 3.5mm microphone

  • 3.5mm headset

Plug detection: when a new audio line input or output is plugged in the system, it needs to be recognized.

The following test units are covered in this category:

audio/alsa_record_playback_external

Verify external microphone sound recording and playback.

Category ID:

audio
Status:

Blocking
Purpose:

This test will check that recording sound using an external microphone works correctly
Steps:

  1. Connect a microphone to your microphone port 2. Click “Test”, then speak into the external microphone 3. After a few seconds, your speech will be played back to you
Verification:

Did you hear your speech played back?
After-suspend:

True
Plugin:

user-interact-verify
audio/alsa_record_playback_internal

Test internal microphone recording and playback functionality.

Category ID:

audio
Status:

Blocking
Purpose:

This test will check that recording sound using the onboard microphone works correctly
Steps:

  1. Disconnect any external microphones that you have plugged in 2. Click “Test”, then speak into your internal microphone 3. After a few seconds, your speech will be played back to you.
Verification:

Did you hear your speech played back?
After-suspend:

True
Plugin:

user-interact-verify
audio/microphone-plug-detection

Ensure that the system can detect when a microphone is plugged in.

Category ID:

audio
Status:

Blocking
Purpose:

Check that system detects a microphone being plugged in
Steps:

  1. Prepare a microphone with a standard 3.5mm jack 2. Locate the microphone jack on the device under test. Keep in mind that it may be shared with the headphone jack. 3. Run the test (you have 30 seconds from now on) 4. Plug the microphone into the appropriate jack 5. Unplug the device for subsequent tests.
Verification:

Verification is automatic, no action is required. The test times out after 30 seconds (and fails in that case).
After-suspend:

True
Plugin:

user-interact
audio/playback_auto

Ensure the internal speakers are working by playing a tone.

Category ID:

audio
Status:

Blocking
Purpose:

This test will check that internal speakers work correctly
Steps:

  1. Make sure that no external speakers or headphones are connected 2. Commence the test to play a brief tone on your audio device
Verification:

Did you hear a tone?
After-suspend:

True
Plugin:

user-interact-verify
audio/playback_headphones

Verify headphone connectivity and audio playback quality.

Category ID:

audio
Status:

Blocking
Purpose:

This test will check that the headphones connector works correctly.
Steps:

  1. Connect a pair of headphones to your audio device. 2. Commence the test to play a sound through your audio device.
Verification:

Did you hear a sound through the headphones, and did the sound play without any distortion, clicks, or other strange noises from your headphones?
After-suspend:

True
Plugin:

user-interact-verify
audio/speaker-headphone-plug-detection

Ensure the system automatically detects when speakers or headphones are plugged in.

Category ID:

audio
Status:

Blocking
Purpose:

Check that system detects speakers or headphones being plugged in
Steps:

  1. Prepare a pair of headphones or speakers with a standard 3.5mm jack 2. Locate the speaker / headphone jack on the device under test 3. Run the test (you have 30 seconds from now on) 4. Plug headphones or speakers into the appropriate jack 5. Unplug the device for subsequent tests.
Verification:

Verification is automatic, no action is required. The test times out after 30 seconds (and fails in that case).
After-suspend:

True
Plugin:

user-interact

Bluetooth tests

Bluetooth LE (Smart and Smart Ready) is tested for device scanning and pairing. Apart from pairing, several profiles are specifically tested and required:

  • Audio (A2DP)

  • HID Over GATT Profile (HOGP), Low-Energy keyboard or mouse with basic functionality

  • File transfer (OBEX)

  • Bluetooth LE advertisement (Scan)

The following test units are covered in this category:

bluetooth/audio-a2dp

Verify Bluetooth device’s High Fidelity Playback (A2DP) capability.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check the High Fidelity Playback (A2DP) capability of your Bluetooth device, to see if you can hear audio from it.
Steps:

  1. Enable and pair the Bluetooth headset 2. Click “Test” to play a brief tone on your Bluetooth device, if it failed to set the Mode to A2DP, please select the device and change it manually in the “Sound Settings”
Verification:

Did you hear the tone?
After-suspend:

True
Plugin:

user-interact-verify
bluetooth/audio_record_playback

Verify Bluetooth HSP/HFP profile capability for recording and playback.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check the Headset Head Unit (HSP/HFP) capability of your Bluetooth device, to check if you can record sounds.
Steps:

  1. Enable and pair the bluetooth headset. 2. Click “Test”, then speak into your Bluetooth microphone. 3. After a few seconds, your speech will be played back to you.
Verification:

Did you hear your speech played back?
After-suspend:

True
Plugin:

user-interact-verify
bluetooth/bluetooth_obex_send

Bluetooth OBEX send

Category ID:

bluetooth
Status:

Blocking
Purpose:

This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable
After-suspend:

True
Environment variable:

BTDEVADDR, PLAINBOX_PROVIDER_DATA
Plugin:

shell
bluetooth/detect-output

Store Bluetooth device information for reports.

Category ID:

bluetooth
Status:

Blocking
Purpose:

Automated test to store Bluetooth device information in the Checkbox report
After-suspend:

True
Plugin:

shell
bluetooth4/HOGP-keyboard

Verify HOGP keyboard functionality with Bluetooth Smart.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart keyboard.
Steps:

  1. Enable a Bluetooth Smart keyboard, and put it into pairing mode. 2. Commence the test to do the auto-pairing, you will be asked to select the targeting keyboard from the list. 3. After it’s paired and connected, enter some text with your keyboard.
Verification:

Did the Bluetooth Smart keyboard work as expected?
After-suspend:

True
Plugin:

user-interact-verify
bluetooth4/HOGP-mouse

Test the functionality of a Bluetooth Smart mouse using HID Over GATT Profile.

Category ID:

bluetooth
Status:

Blocking
Purpose:

This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart mouse.
Steps:

  1. Enable a Bluetooth Smart mouse, and put it into pairing mode. 2. Commence the test to do the auto-pairing; you will be asked to select the targeting mouse from the list. 3. After it’s paired and connected, perform actions such as moving the pointer, right and left button clicks, and double clicks.
Verification:

Did the Bluetooth Smart mouse work as expected?
After-suspend:

True
Plugin:

user-interact-verify
bluetooth4/beacon_eddystone_url_interface

Test system can get beacon EddyStone URL advertisements on the {interface} adapter

Unit type:

template
Category ID:

bluetooth
Status:

Blocking
Description:

<missing description>
After-suspend:

True
From template:

bluetooth4/beacon_eddystone_url_interface
Template resource:

device
Template filter:

device.category == ‘BLUETOOTH’

Camera tests

Internal main webcams are tested to be able to take static pictures and video.

The following test units are covered in this category:

camera/detect

This Automated test attempts to detect a camera.

Category ID:

camera
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

shell
camera/display_name

Webcam video display test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Blocking
After-suspend:

True
From template:

camera/display_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’
camera/led_name

Webcam LED test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Blocking
After-suspend:

True
From template:

camera/led_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’
camera/multiple-resolution-images_name

Webcam multiple resolution capture test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Blocking
After-suspend:

True
From template:

camera/multiple-resolution-images_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’
camera/still_name

Webcam still image capture test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Blocking
After-suspend:

True
From template:

camera/still_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

CPU tests

x86_64 and ARM processors are tested to ensure proper functionality. We will test specific features as:

  • CPU’s performance states (frequency up and down in runtime)

  • CPU’s sleep states (cpu on and off in runtime)

  • Running CPU at its maximum frequency

We will also include a general stress test performed for 120 minutes to verify that the system can handle a sustained high load for a period of time. This test uses the tool “stress-ng” available in the Universe repositories.

The following test units are covered in this category:

cpu/cstates

Run C-States tests

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Uses the Firmware Test Suite (fwts) to test the power saving states of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
cpu/maxfreq_test

Test that the CPU can run at its max frequency

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency.
Environment variable:

LD_LIBRARY_PATH, SNAP
User:

root
Plugin:

shell
cpu/offlining_test

Test offlining of each CPU core

Category ID:

cpu
Status:

Blocking
Purpose:

Attempts to offline each core in a multicore system.
After-suspend:

True
User:

root
Plugin:

shell
cpu/scaling_test

Test the CPU scaling capabilities

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Use Firmware Test Suite (fwts cpufreq) to test the scaling capabilities of the CPU.
Environment variable:

LD_LIBRARY_PATH, PLAINBOX_SESSION_SHARE, SNAP
User:

root
Plugin:

shell
cpu/topology

Check CPU topology for accuracy between proc and sysfs

Category ID:

cpu
Status:

Blocking
Purpose:

<missing purpose>
Description:

Parses information about CPU topology provided by proc and sysfs and checks that they are consistent.
After-suspend:

True
Plugin:

shell

Disk tests

The following test units are covered in this category:

disk/detect

Gathers information about each disk detected

Category ID:

disk
Status:

Blocking
Purpose:

Uses lsblk to gather information about each disk detected on the system under test.
After-suspend:

True
Plugin:

shell
disk/read_performance_name

Disk performance test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
After-suspend:

True
Environment variable:

DISK_MDADM_READ_PERF, DISK_NVME_READ_PERF, DISK_READ_PERF
From template:

disk/read_performance_name
Template resource:

device
Template filter:

device.category == ‘DISK’
disk/storage_device_name

Disk I/O stress test for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Blocking
After-suspend:

True
From template:

disk/storage_device_name
Template resource:

device
Template filter:

device.category == ‘DISK’
thunderbolt3/insert

Storage insert detection on Thunderbolt 3 port

Category ID:

disk
Status:

Blocking
Purpose:

This test will check if the insertion of a Thunderbolt 3 HDD could be detected
Steps:

  1. Click ‘Test’ to begin the test. This test will timeout and fail if the insertion has not been detected within 40 seconds. 2. Plug a Thunderbolt HDD into an available Thunderbolt 3 port, if it’s not mounted automatically, please click the HDD icon to mount it.
Verification:

The verification of this test is automated. Do not change the automatically selected result
After-suspend:

True
User:

root
Plugin:

user-interact
thunderbolt3/remove

Storage removal detection on Thunderbolt 3 port

Category ID:

disk
Status:

Blocking
Purpose:

This test will check if the system can detect the removal of a Thunderbolt 3 HDD
Steps:

  1. Click ‘Test’ to begin the test. This test will timeout and fail if the removal has not been detected within 20 seconds. 2. Remove the previously attached Thunderbolt HDD from the Thunderbolt port.
Verification:

The verification of this test is automated. Do not change the automatically selected result
After-suspend:

True
User:

root
Plugin:

user-interact
thunderbolt3/storage-test

Storage test on Thunderbolt 3

Category ID:

disk
Status:

Blocking
Purpose:

This is an automated test which performs read/write operations on an attached Thunderbolt HDD
After-suspend:

True
User:

root
Plugin:

shell

Ethernet Device tests

  • Connections are tested for functionality, but not for performance.

  • Hot plugging must be supported and working.

The following test units are covered in this category:

ethernet/detect

Detect if at least one ethernet device is detected

Category ID:

ethernet
Status:

Blocking
Purpose:

Test to detect and return information about available network controllers on the system under test.
After-suspend:

True
Plugin:

shell
ethernet/hotplug-interface

Ensure hotplugging works on port {{ interface }}

Unit type:

template
Category ID:

ethernet
Status:

Blocking
After-suspend:

True
From template:

ethernet/hotplug-interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’

Graphics tests

The ability to run the desktop environment out of the box is required. When certifying, stock Ubuntu open source drivers need to work to complete the installation of Ubuntu, although proprietary drivers are accepted, if they are installable through Ubuntu Drivers.

Laptops

  • The integrated display is tested with its highest resolution (up to 4k). At its highest resolution, the image should look clean, without any type of corruption.

  • Each of the available external video ports (currently supported: HDMI, DisplayPort, Mini DisplayPort, Thunderbolt 3) are tested one by one. Hot plugging and different modes (mirror, extended, just internal, just external) are required to work. If several external ports are available, they are not required (nor tested) to work simultaneously.

  • Display brightness. It should be possible to dim the brightness of the internal display.

  • The system must support Gnome Shell 46 and basic 3D rendering.

  • On systems with Hybrid graphics, BIOS default mode must be “Hybrid”. This is the only mode required to work for certification.

  • Hybrid graphics:

    • On systems with an integrated and an NVidia discrete GPU, users should be able to run graphics workloads in the discrete GPU.

    • On systems with an integrated and an AMD discrete GPU, users should be able to run graphics workloads in the discrete GPU.

    • On systems with an integrated and an Intel discrete GPU, users should be able to run graphics workloads in the discrete GPU.

Desktops with an integrated display (aka All-In-Ones)

  • The integrated display is tested with its highest resolution. At its highest resolution (up to 4k), the image should look clean, without any type of corruption.

  • The integrated display must support Gnome Shell 46 and basic 3D rendering.

  • The system must support display rotation (0°, 90°, 180°, 270°)

  • Each of the available external video ports (currently supported HDMI, DisplayPort, Mini DisplayPort, Thunderbolt 3) are tested one by one. Different modes (mirror, extended, just internal, just external) are required to work. If several external ports are available, they are not required (nor tested) to work simultaneously.

Desktops without an integrated display

  • The system is tested using the default BIOS Graphics settings.

  • Each of the available external video ports (currently supported HDMI, DisplayPort, Mini DisplayPort, Thunderbolt 3) are tested one by one, at their highest resolution (up to 4k). Different modes (mirror, extended, just internal, just external) are required to work.

  • If several external ports are available, no matter how many ports are designed to work synchronously on the graphic controller, only two of them (regardless of the combination) are supported simultaneously (dual-head), unless the hardware does not support two monitors setup.

  • The system must support Gnome Shell 46 and basic 3D rendering.

  • The system must support display rotation (0°, 90°, 180°, 270°)

  • Desktops with multiple discrete GPUs are not supported; we only support one discrete GPU, as above.

Convertible Laptops (aka 2-in-1s)

  • The system must support display rotation (0°, 90°, 180°, 270°)

  • The desktop environment must rotate automatically when sensors detect a rotation

  • A swipe up from the bottom edge of the touchscreen should bring up the on-screen keyboard

The following test units are covered in this category:

graphics/VESA_drivers_not_in_use

Test that VESA drivers are not in use

Category ID:

graphics
Status:

Blocking
Purpose:

<missing purpose>
Description:

Check that VESA drivers are not in use
After-suspend:

True
Plugin:

shell
graphics/valid_glxgears

Test that glxgears works for current video card

Category ID:

graphics
Status:

Blocking
Purpose:

Tests the basic 3D capabilities of your current video card. This test covers all devices without an integrated display, such as desktops.
Steps:

  1. Click “Test” to execute an OpenGL demo. Press ESC at any time to close. 2. Verify that the animation is not jerky or slow.
Verification:

  1. Did the 3d animation appear? 2. Was the animation free from slowness/jerkiness?
After-suspend:

True
Plugin:

user-interact-verify
graphics/valid_glxgears_fullscreen

Test that glxgears works in full screen mode for current video card

Category ID:

graphics
Status:

Blocking
Purpose:

Tests the basic full screen 3D capabilities of your current video card. This test covers all devices without an integrated display, such as desktops.
Steps:

  1. Click “Test” to execute an OpenGL demo. Press ESC at any time to close. 2. Verify that the animation is not jerky or slow.
Verification:

  1. Did the 3d animation appear? 2. Was the animation free from slowness/jerkiness?
After-suspend:

True
Plugin:

user-interact-verify

Hotkey tests

  • Super key (Windows logo key). The Super key needs to show all open windows.

  • Volume. Output volume needs to react to up and down volume keys

  • Volume mute. Audio output needs to be muted and unmuted when pressing the volume mute key.

  • Microphone mute key: Audio input should be stopped and could be continued when pressing again.

  • Brightness. Internal display brightness needs to react to brightness keys.

  • Monitor. Several display modes need to react to the monitor hotkey.

  • Media control. Keys that control media play need to be able to control a video played through “Totem” or an audio file played through “Rhythmbox”

  • Wireless. Soft and hard wireless keys need to turn on and off wireless and Bluetooth in the system

  • Non-RGB Keyboard backlight. Backlight of the internal keyboard needs to be turned on and off when pressing the keyboard backlight key.

  • Power button. When the system is booted, pressing the power button will bring up the dialog to power off, reboot or log out from the system.

The following test units are covered in this category:

keys/brightness

Test the brightness keys functionality on the keyboard.

Category ID:

keys
Status:

Blocking
Purpose:

This test will test the brightness keys
Steps:

  1. Press the brightness buttons on the keyboard
Verification:

Did the brightness change according to your key presses?
After-suspend:

True
Plugin:

manual
keys/keyboard-backlight

Test the functionality of the keyboard backlight toggle key.

Category ID:

keys
Status:

Blocking
Purpose:

Verify that the keyboard backlight toggle key works properly
Steps:

  1. Tap the keyboard backlight key 2. Confirm that the keyboard backlight was toggled to the opposite state 3. Tap the keyboard backlight key again 4. Confirm that the keyboard backlight was toggled to the opposite state
Verification:

Did the keyboard backlight state change on each press?
After-suspend:

True
Plugin:

manual
keys/lock-screen

Test the functionality of the screen lock key.

Category ID:

keys
Status:

Blocking
Purpose:

This test will test the screen lock key
Steps:

  1. Press the lock screen button on the keyboard in 30 seconds. 2. If the screen is locked, move the mouse or press any key to activate the prompt. 3. Input the password to unlock the screen.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
Plugin:

user-interact
keys/media-control

Test media keys functionality on the keyboard.

Category ID:

keys
Status:

Blocking
Purpose:

This test will test the media keys of your keyboard
Steps:

  1. Click test to open a window on which to test the media keys. 2. If all the keys work, the test will be marked as passed.
Verification:

Do the keys work as expected?
After-suspend:

True
User:

root
Plugin:

user-interact
keys/microphone-mute

Test the microphone mute key functionality.

Category ID:

keys
Status:

Blocking
Purpose:

This test will test the mute key for your microphone
Steps:

  1. Click “Test” then speak: “Imagination is more important than knowledge” (or anything else) into your microphone. 2. While you are speaking, please press the mute key for the microphone to mute it and press it again to unmute. 3. After a few seconds, your speech will be played back to you. If the key works, your speech should be interrupted for a few seconds.
Verification:

Does the microphone mute key work as expected?
After-suspend:

True
Plugin:

user-interact-verify
keys/mute

This test checks if the mute key on the keyboard operates as expected.

Category ID:

keys
Status:

Blocking
Purpose:

This test will test the mute key of your keyboard
Steps:

  1. Click test to open a window on which to test the mute key. 2. If the key works, the test will pass and the window will close.
Verification:

Does the mute key work as expected?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
keys/power-button

Test if the power management prompt appears upon pressing the power button.

Category ID:

keys
Status:

Blocking
Purpose:

This test will test the power button.
Steps:

  1. Press the power button. 2. Press cancel to quit.
Verification:

Did the power management prompt pop up when pressing the power button?
After-suspend:

True
Plugin:

manual
keys/power-button-event

Check power button event filtering

Category ID:

keys
Status:

Blocking
Purpose:

This test will check if power button event has reported correctly, the listener will wait for 10 seconds.
Steps:

  1. Run the test (you have 10 seconds from now on) 2. Long-press power button about 3~4 seconds one time (the PBTN/PWRB event will show below) 3. Check the number of output PBTN/PWRB event should be Once.
Verification:

<missing verification>
After-suspend:

True
Plugin:

user-interact
keys/sleep

Test the functionality of the sleep key on the keyboard.

Category ID:

keys
Status:

Blocking
Purpose:

This test will test the sleep key
Steps:

  1. Press the sleep key on the keyboard 2. Wake your system up by pressing the power button
Verification:

Did the system go to sleep after pressing the sleep key?
After-suspend:

True
Plugin:

manual
keys/super

Test the functionality of the super key on your keyboard.

Category ID:

keys
Status:

Blocking
Purpose:

This test will test the super key of your keyboard
Steps:

  1. Click test to open a window on which to test the super key. 2. If the key works, the test will pass, and the window will close.
Verification:

Does the super key work as expected?
After-suspend:

True
User:

root
Plugin:

user-interact
keys/video-out

Validate the functionality of the External Video hot key with an external monitor.

Category ID:

keys
Status:

Blocking
Purpose:

Validate that the External Video hot key is working as expected
Steps:

  1. Plug in an external monitor 2. Press the display hot key to change the monitors configuration
Verification:

Check that the video signal can be mirrored, extended, displayed on external or onboard only.
After-suspend:

True
Plugin:

manual
keys/volume

Test the functionality of the volume keys on the keyboard.

Category ID:

keys
Status:

Blocking
Purpose:

This test will test the volume keys of your keyboard
Steps:

  1. Click test to open a window on which to test the volume keys. 2. If all the keys work, the test will be marked as passed.
Verification:

Do the keys work as expected?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
keys/wireless

Test the functionality of the wireless key on keyboards with special keys.

Category ID:

keys
Status:

Blocking
Purpose:

This test will test the wireless key
Steps:

  1. Press the wireless key on the keyboard 2. Check that the wifi LED turns off or changes color 3. Check that wireless is disabled 4. Press the same key again 5. Check that the wifi LED turns on or changes color 6. Check that wireless is enabled
Verification:

Did the wireless turn off on the first press and on again on the second? (NOTE: the LED functionality will be reviewed in a following test. Please only consider the functionality of the wifi itself here.)
After-suspend:

True
Plugin:

manual

Input Devices tests

Integrated input devices

  • Internal keyboard (basic functionality)

  • TrackPoint

The following test units are covered in this category:

input/accelerometer

Test the accelerometer’s detection and operation as a joystick device.

Category ID:

input
Status:

Blocking
Purpose:

This test will test your accelerometer to see if it is detected and operational as a joystick device.
Steps:

  1. Click on Test 2. Tilt your hardware in the directions onscreen until the axis threshold is met.
Verification:

Is your accelerometer properly detected? Can you use the device?
After-suspend:

True
User:

root
Plugin:

user-interact
input/clicking_product_slug_category___index__

Check button functionality for {product}

Unit type:

template
Category ID:

input
Status:

Blocking
From template:

input/clicking_product_slug_category___index__
Template resource:

device
Template filter:

device.category == ‘MOUSE’ or device.category == ‘TOUCHPAD’
input/keyboard

Test the functionality of your keyboard.

Category ID:

input
Status:

Blocking
Purpose:

This test will test your keyboard
Steps:

  1. Click on Test 2. On the open text area, use your keyboard to type something
Verification:

Is your keyboard working properly?
After-suspend:

True
Plugin:

user-interact-verify
input/pointing_product_slug_category___index__

Check pointing functionality for {product}

Unit type:

template
Category ID:

input
Status:

Blocking
After-suspend:

True
From template:

input/pointing_product_slug_category___index__
Template resource:

device
Template filter:

device.category == ‘MOUSE’ or device.category == ‘TOUCHPAD’ or device.category == ‘TOUCHSCREEN’

LED tests

LED indicators are very common in laptops and some types of desktops. When those exist, they will be tested by following some basic expectations here. The actual behavior may vary depending on the hardware design. To ensure that the behavior is working as expected, please be sure to test against specifications obtained from OEM, as each OEM may have different defined behavior for LEDs.

  • Power/suspend button LED. This needs to work as prescribed by OEM’s expected LED behavior.

  • Suspend chassis LED. This needs to work as prescribed by OEM’s expected LED behavior.

  • Volume mute LED. This needs to work as prescribed by OEM’s expected LED behavior.

  • Microphone mute LED. Fixed light when input volume is muted.

  • Caps lock LED. Fixed light when input is set to all caps.

  • Num lock LED. Fixed light when numeric keypad is on.

  • Power supply LED. This needs to work as prescribed by OEM’s expected LED behavior.

  • Camera LED. Fixed light when the camera is on.

The following test units are covered in this category:

led/caps-lock

Verify the functionality of the Caps Lock LED by pressing the Caps Lock key.

Category ID:

led
Status:

Blocking
Purpose:

Block Caps Lock key LED verification
Steps:

  1. Press “Caps Lock Key” to activate/deactivate caps lock 2. Caps Lock LED should be switched on/off every time the key is pressed
Verification:

Did the Caps Lock LED light as expected?
After-suspend:

True
Plugin:

manual
led/microphone-mute

Verify the functionality of the Microphone Mute LED.

Category ID:

led
Status:

Blocking
Purpose:

Microphone Mute LED verification.
Steps:

  1. Press the Microphone Mute key twice and observe the Microphone LED to determine if it either turned off and on or changed colors.
Verification:

Did the Microphone Mute LED turn on and off or change color as expected?
After-suspend:

True
Plugin:

manual
led/mute

Verify the functionality of the Audio Mute LED.

Category ID:

led
Status:

Blocking
Purpose:

Audio Mute LED verification.
Steps:

  1. Press the Mute key twice and observe the Audio LED to determine if it either turned off and on or changed colors.
Verification:

Did the Audio LED turn on and off or change color as expected?
After-suspend:

True
Plugin:

manual
led/numeric-keypad

Verify the function of the numeric keypad LED by toggling and typing with it.

Category ID:

led
Status:

Blocking
Purpose:

Numeric keypad LED verification
Steps:

  1. Press “Num Lock” key to toggle numeric keypad LED 2. Click on the “Test” button to open a window to verify your typing 3. Type using the numeric keypad both when the LED is on and off
Verification:

  1. Numeric keypad LED status should toggle every time the “Num Lock” key is pressed 2. Numbers should only be entered in the keyboard verification window when the LED is on
After-suspend:

True
Plugin:

user-interact-verify
led/power

Power LED behavior when powered

Category ID:

led
Status:

Blocking
Purpose:

Check power led is on when system is powered on
Steps:

  1. Check power led when system is powered on
Verification:

Power led is on when system is powered on
After-suspend:

True
Plugin:

manual
led/suspend

Verify if the Suspend LED blinks or changes color while the system is suspended.

Category ID:

led
Status:

Blocking
Purpose:

Suspend LED verification.
Steps:

  1. The Suspend LED should blink or change color while the system is suspended
Verification:

Did the Suspend LED blink or change color while the system was suspended?
Plugin:

manual
led/wireless

Verify the WLAN/Bluetooth LED functionality by toggling wireless connections.

Category ID:

led
Status:

Blocking
Purpose:

Wireless (WLAN + Bluetooth) LED verification
Steps:

  1. Make sure WLAN connection is established and Bluetooth is enabled. 2. WLAN/Bluetooth LED should light 3. Switch WLAN and Bluetooth off from a hardware switch (if present) 4. Switch them back on 5. Switch WLAN and Bluetooth off from the panel applet 6. Switch them back on
Verification:

Did the WLAN/Bluetooth LED light as expected?
Plugin:

manual

Media Card tests

Media Card readers are tested for read and write for the following type or cards:

  • SD

  • SDHC

The following test units are covered in this category:

mediacard/sdhc-insert

Test that insertion of an SDHC card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system’s media card reader can detect the insertion of a UNLOCKED Secure Digital High-Capacity (SDHC) media card
Steps:

  1. Commence the test and then insert an UNLOCKED SDHC card into the reader. (Note: this test will time-out after 20 seconds.) 2. Do not remove the device after this test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
mediacard/sdhc-remove

Test that removal of an SDHC card is detected

Category ID:

mediacard
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of an SDHC card from the system’s card reader.
Steps:

  1. Commence the test and then remove the SDHC card from the reader. (Note: this test will time-out after 20 seconds.)
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
mediacard/sdhc-storage

Test reading & writing to a SDHC Card

Category ID:

mediacard
Status:

Blocking
Purpose:

<missing purpose>
Description:

This test is automated and executes after the mediacard/sdhc-insert test is run. It tests reading and writing to the SDHC card.
After-suspend:

True
User:

root
Plugin:

shell

Memory tests

Proper detection of the amount of memory installed is required (the amount of memory installed is the memory seen by the OS).

The following test units are covered in this category:

memory/info

Check the amount of memory reported by meminfo against DMI

Category ID:

memory
Status:

Blocking
Purpose:

This test checks the amount of memory which is reported in meminfo against the size of the memory modules detected by DMI.
User:

root
Plugin:

shell

Miscellaneous tests

The following test units are covered in this category:

miscellanea/check_prerelease

Test that the system is not a pre-release version

Category ID:

miscellanea
Status:

Blocking
Purpose:

Test to verify that the system uses production, rather than pre-release, versions of the kernel and the OS.
Plugin:

shell
miscellanea/debsums

Check the MD5 sums of installed Debian packages

Category ID:

miscellanea
Status:

Blocking
Purpose:

Verify installed Debian package files against MD5 checksum lists from /var/lib/dpkg/info/*.md5sums.
User:

root
Plugin:

shell
miscellanea/grub_file_check

Check if the file core.efi exists to make sure shim and grub can be upgraded

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
miscellanea/oops

Run FWTS OOPS check

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (FWTS) oops tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
miscellanea/ubuntu-desktop-minimal-recommends

Check that all the recommended packages for ubuntu-desktop-minimal are installed

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
miscellanea/ubuntu-desktop-recommends

Check that all the recommended packages for ubuntu-desktop are installed

Category ID:

miscellanea
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell

Monitor tests

The following test units are covered in this category:

monitor/multi-head

Verify multi-monitor output functionality on desktop systems.

Category ID:

monitor
Status:

Blocking
Purpose:

This test verifies that multi-monitor output works on your desktop system. This is not the same test as the external monitor tests you would run on your laptop. You will need two monitors to perform this test.
Steps:

Skip this test if your video card does not support multiple monitors. 1. If your second monitor is not already connected, connect it now. 2. Open the “Displays” tool (open the dash and search for “Displays”). 3. Configure your output to provide one desktop across both monitors. 4. Open any application and drag it from one monitor to the next.
Verification:

Was the stretched desktop displayed correctly across both screens?
After-suspend:

True
Plugin:

manual

Non-device specific networking tests

The following test units are covered in this category:

networking/gateway_ping

Verifies if the system can connect to the Internet.

Category ID:

networking
Status:

Blocking
Purpose:

Tests whether the system has a working Internet connection.
After-suspend:

True
User:

root
Plugin:

shell
networking/info_device__index___interface

Network Information of device {__index__} ({interface})

Unit type:

template
Category ID:

networking
Status:

Blocking
After-suspend:

True
From template:

networking/info_device__index___interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’
networking/ntp

Test NTP server synchronization capability.

Category ID:

networking
Status:

Blocking
Purpose:

Test to see if we can sync local clock to an NTP server
After-suspend:

True
User:

root
Plugin:

shell

Optical Drive tests

Optical drives are tested for read based on specifications of the optical unit. Depending on the unit theoretical feature support, we will test:

  • CD read capabilities (data only)

  • DVD read capabilities (data only)

  • Blu Ray-disc read capabilities (data only)

The following test units are covered in this category:

optical/detect

Displays discovered optical drives

Category ID:

optical
Status:

Blocking
Purpose:

Detects optical drives (CD/DVD) attached to the system.
Plugin:

shell
optical/read-automated_name

Tests read functionality of optical drive {name}

Unit type:

template
Category ID:

optical
Status:

Blocking
After-suspend:

True
From template:

optical/read-automated_name
Template resource:

device
Template filter:

device.category == ‘CDROM’

Power Management tests

Suspend/Resume

A 30 cycle suspend/resume stress test is performed using the fwts suspend test. Only the default suspend method (S3 or s2idle) is required to work. The test is passed if all 30 cycles complete without failure. Any errors reported in the fwts log for the 30 cycle suspend/resume stress test are informational only and do not affect the outcome of the test, however, we do recommend examining and fixing any failures noted, as they indicate firmware non-compliance with standards.

Apart from the stress test, a single cycle suspend/resume is performed, if it’s a hybrid graphic system, suspend and graphic related functionalities are required to work flawlessly on the On-demand mode, and the following features and devices are tested and need to work after suspend:

  • CPU

  • Memory

  • Networking (Wifi, Ethernet)

  • Audio

  • Bluetooth

  • Display resolutions should be consistent before and after suspend

  • USB and Thunderbolt controllers

  • Input devices

  • Mediacards

Cold/Warm boot

Both cold boot and warm boot are tested and required to work. Cold reboot is performed where an RTC is available (see next section).

The following features and devices are tested and need to work after cold/warm boot:

  • USB and Thunderbolt controllers

The following test units are covered in this category:

power-management/cold-reboot

Cold reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This test powers off the system and then powers it on using RTC
Environment variable:

COLD_REBOOT_DELAY, RTC_DEVICE_FILE
User:

root
Plugin:

shell
power-management/fwts_wakealarm

Executes ACPI Wakealarm test to validate functionality.

Category ID:

power-management
Status:

Blocking
Purpose:

Test ACPI Wakealarm (fwts wakealarm)
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
power-management/lid

Check if the laptop lid sensors cause the system to suspend when the lid is closed.

Category ID:

power-management
Status:

Blocking
Purpose:

This test will check your lid sensors.
Steps:

  1. Close your laptop lid.
Verification:

Does closing your laptop lid cause your system to suspend?
Plugin:

manual
power-management/lid_close_suspend_open

Test the functionality of the laptop’s lid sensor for suspend/resume actions.

Category ID:

power-management
Status:

Blocking
Purpose:

This test will check your lid sensor can detect lid close/open, and the DUT (Device Under Test) will suspend when the lid is closed
Steps:

  1. Press “Enter” to start the test 2. Close the lid (Please close the lid within 10 seconds) 3. Wait 5 seconds with the lid closed 4. Open the lid
Verification:

Did the system suspend when the lid was closed, and resume back when the lid was opened? Note: Systemd will not react to lid events if the DUT was just started or resumed. Please make sure the DUT has been running for long enough before running this test.
Plugin:

user-interact-verify
power-management/post-cold-reboot

Post cold reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the cold reboot
Plugin:

shell
power-management/post-warm-reboot

Post warm reboot service check

Category ID:

power-management
Status:

Blocking
Purpose:

Check there are no failed services after the warm reboot
Plugin:

shell
power-management/rtc

Test that RTC functions properly (if present)

Category ID:

power-management
Status:

Blocking
Purpose:

Verify that the Real-time clock (RTC) device functions properly, if present.
After-suspend:

True
Environment variable:

RTC_DEVICE_FILE
User:

root
Plugin:

shell
power-management/tickless_idle

Verify the tickless idle feature configuration in the kernel.

Category ID:

power-management
Status:

Blocking
Purpose:

Check to see if CONFIG_NO_HZ is set in the kernel (this is just a simple regression check)
After-suspend:

True
Plugin:

shell
power-management/warm-reboot

Warm reboot

Category ID:

power-management
Status:

Blocking
Purpose:

This tests reboots the system using the reboot command
User:

root
Plugin:

shell

Suspend tests

The following test units are covered in this category:

suspend/audio_after_suspend_auto

Verify mixer settings consistency after system suspend.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that mixer settings after suspend are the same as before suspend.
Plugin:

shell
suspend/audio_before_suspend

Record mixer settings before suspending.

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/cpu_after_suspend_auto

Ensure all CPUs are online post-resumption.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that all CPUs are online after resuming.
Plugin:

shell
suspend/cpu_before_suspend

Verify that all the CPUs are online before suspending

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/display_after_suspend

Verify if the display functions normally after a suspend and resume cycle.

Category ID:

suspend
Status:

Blocking
Purpose:

This test will check that the display is correct after suspend and resume
Steps:

  1. Check that your display does not show up visual artifacts after resuming.
Verification:

Does the display work normally after resuming from suspend?
Plugin:

manual
suspend/memory_after_suspend_auto

Ensure all memory is accessible after waking from suspend mode.

Category ID:

suspend
Status:

Blocking
Purpose:

Verify that all memory is available after resuming from suspend.
Plugin:

shell
suspend/memory_before_suspend

Dumps memory info to a file for comparison after suspend

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
suspend/network_after_suspend_auto

Verify the network functionality after system resume.

Category ID:

suspend
Status:

Blocking
Purpose:

Test the network after resuming.
User:

root
Plugin:

shell
suspend/network_before_suspend

Record the current network before suspending.

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
suspend/oops_after_suspend

Run fwts oops tests post-suspension to ensure system stability.

Category ID:

suspend
Status:

Blocking
Purpose:

Run Firmware Test Suite (fwts) oops tests after suspend.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
suspend/resolution_after_suspend

Verify resolution consistency after resuming from suspension.

Category ID:

suspend
Status:

Blocking
Purpose:

Test to see that we have the same resolution after resuming as before.
Plugin:

shell
suspend/resolution_before_suspend

Record the current resolution before suspending.

Category ID:

suspend
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell

Touchpad tests

Capacitive touchpads are tested for single touch and multitouch. Functionality tested:

  • Single touch, including single tap and double tap

  • Scrolling feature (horizontal and vertical) should work either with the edge scrolling option or the 2 finger scrolling option.

The following test units are covered in this category:

touchpad/basic

<missing summary>

Category ID:

touchpad
Status:

Blocking
Purpose:

Touchpad manual verification
Steps:

  1. Make sure that touchpad is enabled. 2. Move cursor using the touchpad.
Verification:

Did the cursor move?
After-suspend:

True
Plugin:

manual
touchpad/continuous-move

Verify that the touchpad supports continuous movement without interruption.

Category ID:

touchpad
Status:

Blocking
Purpose:

Verify continuous movement of the touchpad
Steps:

  1. Select “Test” when ready and continuously move your cursor within the borders of the displayed test window. You’ll need to keep moving your finger on the touchpad for 10 seconds.
Verification:

Does the mouse cursor move without interruption?
After-suspend:

True
Plugin:

user-interact
touchpad/detected-as-mouse

Check if the touchpad is recognized as a mouse by the system.

Category ID:

touchpad
Status:

Blocking
Purpose:

This test will check if your touchpad was detected as a mouse.
After-suspend:

True
Plugin:

shell
touchpad/drag-and-drop

Test the drag and drop functionality on the touchpad.

Category ID:

touchpad
Status:

Blocking
Purpose:

Determine that the drag and drop function is working as expected.
Steps:

  1. Press ‘PrtScn’ key to take a screenshot 2. Tap ‘Files’ on the dock (launcher bar) to open HomePictures folder 3. Double tap and hold the Screenshot* file 4. Drag the selected file to the Home folder and release the finger from touchpad.
Verification:

Does drag and drop work on the Touchpad?
After-suspend:

True
Plugin:

manual
touchpad/multitouch

Test the touchpad’s 2-finger scroll functionality.

Category ID:

touchpad
Status:

Blocking
Purpose:

Touchpad 2-finger scroll verification
Steps:

  1. Press “Enter” when ready and place your cursor within the borders of the displayed test window. 2. Verify that the scroll events are detected by moving two fingers in all four directions along the touchpad.
Verification:

Were scroll events detected for all four directions?
After-suspend:

True
Plugin:

user-interact
touchpad/multitouch-rightclick

Test the right-click function of touchpad with a multi-touch gesture.

Category ID:

touchpad
Status:

Blocking
Purpose:

Determine that the right-click function is working as expected.
Steps:

  1. Open a file folder. 2. Hover cursor over file in folder. 3. Perform a 2-touch tap.
Verification:

Did the right-click pop-up menu appear?
After-suspend:

True
Plugin:

manual
touchpad/singletouch-selection

<missing summary>

Category ID:

touchpad
Status:

Blocking
Purpose:

Determine that the selection window function is working as expected.
Steps:

  1. Open a file folder 2. Double tap and drag the cursor across several file.
Verification:

Did a selection window open and were several files selected?
After-suspend:

True
Plugin:

manual

Touchscreen tests

Touch screens are tested for single touch and multitouch. Functionality tested:

  • Single touch, including single tap and double tap

  • For multitouch touch screens with more than two finger support, at least 4 finger gestures must be recognized by the OS. Gestures tested include:

    • 2 finger expand/pinch zoom

    • 3 finger touch-tap

    • 4 finger touch-tap

The following test units are covered in this category:

touchscreen/drag-n-drop

Assess touchscreen functionality for drag & drop tasks.

Category ID:

touchscreen
Status:

Blocking
Purpose:

Check touchscreen drag & drop
Steps:

  1. Press ‘PrtScn’ key to create a screenshot 2. Tap ‘Files’ on dock (launcher bar) to open Home folder 3. Double Tap ‘Pictures’ folder to open Pictures folder 4. Tap and hold the Screenshot* files in the Pictures folder 5. Drag and drop the Screenshot* files to Home folder
Verification:

Does drag and drop work?
After-suspend:

True
Plugin:

manual
touchscreen/evdev/2-touch-tap-product_slug

Validate proper detection of a 2-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
After-suspend:

True
From template:

touchscreen/evdev/2-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/3-touch-tap-product_slug

Validate proper detection of a 3-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
After-suspend:

True
From template:

touchscreen/evdev/3-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/4-touch-tap-product_slug

Validate the detection of a 4-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
After-suspend:

True
From template:

touchscreen/evdev/4-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/evdev/single-touch-tap-product_slug

Validate the detection of a single-touch tap on touchscreen devices.

Unit type:

template
Category ID:

touchscreen
Status:

Blocking
After-suspend:

True
From template:

touchscreen/evdev/single-touch-tap-product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHSCREEN’
touchscreen/multitouch-zoom

Check touchscreen pinch gesture for zoom

Category ID:

touchscreen
Status:

Blocking
Purpose:

Check touchscreen pinch gesture for zoom
Steps:

  1. Commence the test 2. Using 2 fingers, resize the blue square until it turns green, then release it.
Verification:

Did the blue square change size following the gesture?
After-suspend:

True
Plugin:

user-interact-verify

TPM 2.0 (Trusted Platform Module)

The following test units are covered in this category:

clevis-encrypt-tpm2/detect-ecc-capabilities

Ensure the TPM has required capabilities for clevis ECC test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/detect-rsa-capabilities

Ensure the TPM has required capabilities for clevis RSA test

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/ecc

clevis encrypt/decrypt key ecc

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell
clevis-encrypt-tpm2/precheck

clevis encrypt/decrypt precheck

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
clevis-encrypt-tpm2/rsa

clevis encrypt/decrypt key rsa

Category ID:

tpm2
Status:

Blocking
Purpose:

<missing purpose>
Description:

<missing description>
User:

root
Plugin:

shell

USB tests

USB 2.0

USB storage devices must work on all available USB ports. USB Human Interface Devices (HID), specifically keyboard or mouse, should be working properly on any USB port.

USB 3.2

USB storage devices must work on all available USB ports. USB Human Interface Devices (HID), specifically keyboard or mouse, should be working properly on any USB port.

USB Type-C

USB Type-C supports various types of devices (e.g. Video, Power) through the use of adapters or peripherals. The following adapters/peripherals should work:

  • Storage devices

  • Keyboard or mouse (basic functionality)

  • When DisplayPort over USB Type-C is advertised:

    • Display hot plugging and the following display are required to work: mirrored, extended, internal only, external only.

    • Audio output needs to be undistorted over this port.

USB4

USB controllers are tested using storage devices in all available USB ports. And the USB human interface device, keyboard or mouse, should be working properly with any USB port.

  • Storage devices

  • Keyboard or mouse (basic functionality)

  • For USB4 controllers that support Thunderbolt, Thunderbolt functionality will be tested

The following test units are covered in this category:

usb-c/c-to-a-adapter/hid

USB HID work on USB Type-C port using a “USB Type-C to Type-A” adapter

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port using a “USB Type-C to Type-A” adapter
Steps:

  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port using a “USB Type-C to Type-A” adapter 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text
Verification:

Did the device work as expected?
After-suspend:

True
Plugin:

manual
usb-c/c-to-a-adapter/insert

Verify that the system detects a USB 3 storage device using a “Type-C to Type-A” adapter.

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3 storage device in a USB Type-C connector using a “Type-C to Type-A” adapter. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test 2. Connect a USB 3 storage device to a USB Type-C port using a “Type-C to Type-A” adapter. 3. Do not unplug the device after the test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/c-to-a-adapter/remove

Check the detection of a USB 3 storage device removal, connected via a “Type-C to Type-A” adapter.

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a USB 3 storage device connected to a USB Type-C port using a “USB Type-C to Type-A” adapter.
Steps:

  1. Begin the test 2. Disconnect a USB 3 storage device from a USB Type-C port using a “Type-C to Type-A” adapter
Verification:

The verification of this test is automated. Do not modify the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/c-to-a-adapter/storage-automated

Automate the execution process post usb-c/c-to-a-adapter/insert test.

Category ID:

usb
Status:

Blocking
Purpose:

This test is automated and executes after the usb-c/c-to-a-adapter/insert test is run.
After-suspend:

True
User:

root
Plugin:

shell
usb-c/hid

USB HID work on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that you can use a USB HID device plugged in a USB Type-C port
Steps:

  1. Enable either a USB mouse or keyboard by plugging it in the USB Type-C port 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, switch to another tty and type some text
Verification:

Did the device work as expected?
After-suspend:

True
Plugin:

manual
usb-c/insert

USB 3.0 storage device insertion detected on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the insertion of a USB 3.0 storage device in a USB Type-C connector. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Commence the test. 2. Connect a USB 3.0 storage device to a USB Type-C port. 3. Do not unplug the device after the test.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/remove

USB 3.0 storage device removal detected on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that the system correctly detects the removal of a USB 3.0 storage device connected to a USB Type-C port.
Steps:

  1. Commence the test 2. Connect a USB 3.0 storage device to a USB Type-C port 3. Disconnect the USB 3.0 storage device from the USB Type-C port
Verification:

The system should automatically detect the removal of the USB 3.0 storage device. The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb-c/storage-automated

USB 3.0 storage device read & write works on USB Type-C port

Category ID:

usb
Status:

Blocking
Purpose:

This test is automated and executes after the usb-c/insert test is run.
After-suspend:

True
User:

root
Plugin:

shell
usb/HID

Verify USB HID devices functionality by performing specified actions.

Category ID:

usb
Status:

Blocking
Purpose:

This test will check that you can use a USB HID device
Steps:

  1. Enable either a USB mouse or keyboard 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 3. For keyboards, commence the test to launch a small tool. Type some text and close the tool.
Verification:

Did the device work as expected?
After-suspend:

True
Plugin:

user-interact-verify
usb/detect

Display USB devices attached to SUT

Category ID:

usb
Status:

Blocking
Purpose:

Detects and shows USB devices attached to this system.
After-suspend:

True
Plugin:

shell
usb/insert

USB 2.0 storage device insertion detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect USB 2.0 storage when inserted. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Press continue 2. Wait until the message “INSERT NOW” is printed on the screen 3. Connect USB 2.0 storage device
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb/remove

USB 2.0 storage device removal detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect removal of a USB 2.0 storage device
Steps:

  1. Press continue 2. Wait until the message “REMOVE NOW” is printed on the screen 3. Disconnect USB 2.0 storage device
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb/storage-automated

USB 2.0 storage device read & write works

Category ID:

usb
Status:

Blocking
Purpose:

Check system can read/write to USB 2.0 storage correctly
After-suspend:

True
User:

root
Plugin:

shell
usb3/insert

USB 3.0 storage device insertion detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect insertion of a USB 3.0 storage device. NOTE: Make sure the USB storage device has a partition before starting the test.
Steps:

  1. Press continue. 2. Wait until the message “INSERT NOW” is displayed on the screen. 3. Connect USB 3.0 storage device.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb3/remove

USB 3.0 storage device removal detected

Category ID:

usb
Status:

Blocking
Purpose:

Check system can detect removal of a USB 3.0 storage device
Steps:

  1. Press continue 2. Wait until the message “REMOVE NOW” is printed on the screen 3. Disconnect USB 3.0 storage device
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
usb3/storage-automated

USB 3.0 storage device read & write works

Category ID:

usb
Status:

Blocking
Purpose:

Check system can read/write to USB 3.0 storage devices correctly
After-suspend:

True
User:

root
Plugin:

shell

Wireless networking tests

Wi-Fi interfaces are tested for connection to access points configured for 802.11 b/g/n/ac/ax (Wi-Fi 6) protocols.

The following test units are covered in this category:

wireless/wireless_connection_open_ac_nm_interface

Connect to unencrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_ax_nm_interface

Connect to unencrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_be_nm_interface

Connect to unencrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_bg_nm_interface

Connect to an unencrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_open_n_nm_interface

Connect to an unencrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_open_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_ax_nm_interface

Connect to WPA3-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa3_be_nm_interface

Connect to WPA3-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa3_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ac_nm_interface

Connect to WPA-encrypted 802.11ac Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ac_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_ax_nm_interface

Connect to WPA-encrypted 802.11ax Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_ax_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_be_nm_interface

Connect to WPA-encrypted 802.11be Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_be_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_bg_nm_interface

Connect to WPA-encrypted 802.11b/g Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_bg_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_connection_wpa_n_nm_interface

Connect to a WPA-encrypted 802.11n Wi-Fi network on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_connection_wpa_n_nm_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/wireless_scanning_interface

Test system can discover Wi-Fi networks on {{ interface }}

Unit type:

template
Category ID:

wireless
Status:

Blocking
After-suspend:

True
From template:

wireless/wireless_scanning_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Non-blocking

Advanced Configuration and Power Interface

The following test units are covered in this category:

acpi/oem_osi

Test ACPI OEM _OSI strings

Category ID:

acpi
Status:

Non-blocking
Purpose:

This checks if the deprecated OEM _OSI strings are still used by checking the ACPI DSDT and SSDT tables
User:

root
Plugin:

shell

Audio tests

The following test units are covered in this category:

audio/alsa_info_attachment

Attach audio hardware data collection log to the results.

Category ID:

audio
Status:

Non-blocking
Purpose:

Attaches the audio hardware data collection log to the results.
Plugin:

attachment
audio/alsa_info_collect

Collect audio-related system information for simulation and detailed testing.

Category ID:

audio
Status:

Non-blocking
Purpose:

Collect audio-related system information. This data can be used to simulate this computer’s audio subsystem and perform more detailed tests under a controlled environment.
User:

root
Plugin:

shell
audio/alsa_record_playback_automated

Test playback and recording functionality on the default audio input and output.

Category ID:

audio
Status:

Non-blocking
Purpose:

Play back a sound on the default output and listen for it on the default input. This test is intended mostly for laptops, where the default output will be the internal speakers and the default input will be the internal microphone.
After-suspend:

True
Plugin:

shell
audio/channels

Verify that all audio channels are functioning correctly by hearing a voice clearly from them.

Category ID:

audio
Status:

Non-blocking
Purpose:

Check that the various audio channels are working properly
Steps:

  1. Commence the test
Verification:

You should clearly hear a voice from the different audio channels
After-suspend:

True
Plugin:

user-interact-verify
audio/detect_sinks

Ensure audio sinks are available for detection.

Category ID:

audio
Status:

Non-blocking
Purpose:

Test to detect if there are available sinks
After-suspend:

True
Plugin:

shell
audio/external-linein

Verify external line-in connection functionality by recording and playback testing.

Category ID:

audio
Status:

Non-blocking
Purpose:

Check that external line-in connection works correctly
Steps:

  1. Use a cable to connect the line-in port to an external line-out source. 2. Open system sound preferences, ‘Input’ tab, select ‘Line-in’ on the connector list. Commence the test. 3. After a few seconds, your recording will be played back to you.
Verification:

Did you hear your recording?
After-suspend:

True
Plugin:

user-interact-verify
audio/external-lineout

Verify external line out connection functionality by inserting a cable to speakers, selecting ‘Line Out’ in system sound preferences, and testing sound channels.

Category ID:

audio
Status:

Non-blocking
Purpose:

Check that external line out connection works correctly
Steps:

  1. Insert cable to speakers (with built-in amplifiers) on the line out port 2. Open system sound preferences, ‘Output’ tab, select ‘Line Out’ on the connector list. Commence the test 3. On the system sound preferences, click ‘Test Sound’ to check left and right channel
Verification:

  1. Do you see internal speakers? The internal speakers should be replaced by external speakers from the Line out port, if any 2. Do you hear the sound in the internal speakers? The internal speakers should be muted automatically 3. Do you hear the sound coming out on the corresponding channel by external speakers from the Line out port?
After-suspend:

True
Plugin:

manual
audio/list_devices

Check if audio devices can be detected.

Category ID:

audio
Status:

Non-blocking
Purpose:

Test to detect audio devices
Plugin:

shell
audio/valid-sof-firmware-sig

Ensure SOF firmware signature is valid.

Category ID:

audio
Status:

Non-blocking
Purpose:

Validate SOF firmware signature.
Plugin:

shell

Benchmarks tests

The following test units are covered in this category:

benchmarks/disk/hdparm-cache-read_name

Cached read timing benchmark of {name} using hdparm

Unit type:

template
Category ID:

benchmarks
Status:

Non-blocking
After-suspend:

True
From template:

benchmarks/disk/hdparm-cache-read_name
Template resource:

device
Template filter:

device.category == ‘DISK’
benchmarks/disk/hdparm-read_name

Raw read timing benchmark of {name} using hdparm

Unit type:

template
Category ID:

benchmarks
Status:

Non-blocking
After-suspend:

True
From template:

benchmarks/disk/hdparm-read_name
Template resource:

device
Template filter:

device.category == ‘DISK’

Camera tests

The following test units are covered in this category:

camera/camera-quality-image_name

Attach the image used for the BRISQUE score for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Description:

This test will attach the image used for the BRISQUE score.
After-suspend:

True
From template:

camera/camera-quality-image_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’
camera/camera-quality_name

Webcam BRISQUE score for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
After-suspend:

True
From template:

camera/camera-quality_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’
camera/multiple-resolution-images-attachment_name

Attach an image from the multiple resolution images test for {product_slug}

Unit type:

template
Category ID:

camera
Status:

Non-blocking
Description:

This test will attach one of the images used for the multiple resolution images test.
After-suspend:

True
From template:

camera/multiple-resolution-images-attachment_name
Template resource:

device
Template filter:

device.category == ‘CAPTURE’ and device.name != ‘’

CPU tests

The following test units are covered in this category:

cpu/clocktest

Tests the CPU for clock jitter

Category ID:

cpu
Status:

Non-blocking
Purpose:

Runs a test for clock jitter on SMP machines.
After-suspend:

True
Plugin:

shell
cpu/cstates_results.log

Attach C-States test log

Category ID:

cpu
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission.
Plugin:

attachment
cpu/maxfreq_test-log-attach

Attach CPU max frequency log

Category ID:

cpu
Status:

Non-blocking
Purpose:

Attaches the log generated by cpu/maxfreq_test to the results submission.
Plugin:

attachment
cpu/scaling_test-log-attach

Attach CPU scaling capabilities log

Category ID:

cpu
Status:

Non-blocking
Purpose:

Attaches the log generated by cpu/scaling_test to the results submission.
Plugin:

attachment

Disk tests

The following test units are covered in this category:

disk/apste_support_on_name

Check support for Autonomous Power State Transition on {name}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
Description:

<missing description>
After-suspend:

True
From template:

disk/apste_support_on_name
Template resource:

device
Template filter:

device.driver == ‘nvme’ and device.category == ‘OTHER’
disk/check-software-raid

Validate the configuration of software RAID devices are expected

Category ID:

disk
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Examine the system to detect Software RAID devices are created and the RAID mode are expected the SOFTWARE_RAID_LEVEL variable is needed for this tests. e.g. SOFTWARE_RAID_LEVEL=”raid0 raid1 raid5”
Environment variable:

SOFTWARE_RAID_LEVEL
User:

root
Plugin:

shell
disk/hdd-parking

Test the system’s hard drive protection mechanism.

Category ID:

disk
Status:

Non-blocking
Purpose:

This test checks that a system’s drive protection mechanism (free fall sensor in laptops) works properly.
Steps:

  1. Click on Test 2. Move the system under test around, ensuring it is raised and lowered at some point.
Verification:

The verification of this test is automated. Do not change the automatically selected result.
After-suspend:

True
User:

root
Plugin:

user-interact
disk/stats_name

Disk statistics for {product_slug}

Unit type:

template
Category ID:

disk
Status:

Non-blocking
After-suspend:

True
From template:

disk/stats_name
Template resource:

device
Template filter:

device.category == ‘DISK’ and device.name != ‘’
thunderbolt3/daisy-chain

Daisy-chain testing for Thunderbolt 3 storage and display device

Category ID:

disk
Status:

Non-blocking
Purpose:

This test will check if your system can support daisy-chaining of a storage and a monitor over Thunderbolt 3 port
Steps:

  1. Connect your Thunderbolt monitor to your system 2. Connect and mount your Thunderbolt HDD to another Thunderbolt 3 port of the monitor (you can do this with HDD first as well) 3. Click ‘Test’ to perform the storage test on the Thunderbolt HDD
Verification:

  1. The verification for storage is automated, please select the result combined with the result for the display. 2. Was the desktop displayed correctly on the Thunderbolt-connected screen?
After-suspend:

True
User:

root
Plugin:

user-interact-verify

Ethernet Device tests

The following test units are covered in this category:

ethernet/iperf3_reverse_interface

Iperf3 stress testing for {interface} (reverse)

Unit type:

template
Category ID:

ethernet
Status:

Non-blocking
Environment variable:

LD_LIBRARY_PATH, TEST_TARGET_IPERF
From template:

ethernet/iperf3_reverse_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’
ethernet/iperf3_interface

Iperf3 stress testing for {interface}

Unit type:

template
Category ID:

ethernet
Status:

Non-blocking
Environment variable:

LD_LIBRARY_PATH, TEST_TARGET_IPERF
From template:

ethernet/iperf3_interface
Template resource:

device
Template filter:

device.category == ‘NETWORK’ and device.interface != ‘UNKNOWN’

Fingerprint reader tests

The following test units are covered in this category:

fingerprint/delete

Remove existing fingerprint signatures

Category ID:

fingerprint
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

shell
fingerprint/detect

Check if at least one fingerprint reader is detected

Category ID:

fingerprint
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
User:

root
Plugin:

shell
fingerprint/enroll

Enroll a fingerprint

Category ID:

fingerprint
Status:

Non-blocking
Purpose:

Enroll the user’s right index finger into the database
Steps:

  1. Press Enter 2. Swipe your right index finger five times (wait 3s for the Enrolling message)
Verification:

The verification of this test is automated.
After-suspend:

True
User:

root
Plugin:

user-interact
fingerprint/unlock

Fingerprint unlock screen

Category ID:

fingerprint
Status:

Non-blocking
Purpose:

This test will verify that a fingerprint reader can be used to unlock a locked system.
Steps:

  1. Press Enter to lock the screen 2. Press any key or move the mouse. 3. A window should appear that provides the ability to unlock either typing your password or using fingerprint authentication. 4. Use the fingerprint reader to unlock. 5. Your screen should be unlocked.
Verification:

Did the authentication procedure work correctly?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
fingerprint/verify-match

Fingerprint positive match

Category ID:

fingerprint
Status:

Non-blocking
Purpose:

The verification must match when using the enrolled finger
Steps:

  1. Press Enter 2. Swipe your enrolled finger on the reader (3 attempts max)
Verification:

The verification of this test is automated.
After-suspend:

True
User:

root
Plugin:

user-interact
fingerprint/verify-no-match

Fingerprint negative match

Category ID:

fingerprint
Status:

Non-blocking
Purpose:

The verification must not match when using a wrong finger
Steps:

  1. Press Enter 2. Swipe an unenrolled finger on the reader
Verification:

The verification of this test is automated.
After-suspend:

True
User:

root
Plugin:

user-interact

Firmware tests

The following test units are covered in this category:

firmware/fwts_desktop_diagnosis

Run FWTS QA-concerned desktop-specific diagnosis tests.

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Run Firmware Test Suite (fwts) QA-concerned desktop-specific diagnosis tests.
Environment variable:

PLAINBOX_SESSION_SHARE
User:

root
Plugin:

shell
firmware/fwts_desktop_diagnosis_results.log.gz

Attach FWTS desktop diagnosis log to submission

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the FWTS desktop diagnosis results log to the submission
Plugin:

attachment
firmware/fwts_dump_acpi_attachment.gz

Collect the ACPI tables dump from FWTS

Category ID:

firmware
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

attachment
firmware/fwupdmgr_get_devices

Collect the device firmware update information

Category ID:

firmware
Status:

Non-blocking
Purpose:

Attach information about the devices, as reported by the fwupdmgr
Plugin:

attachment

Gathers information about the DUT

The following test units are covered in this category:

cpuinfo

Collect information about the CPU

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Gets CPU resource info from /proc/cpuinfo
User:

root
Plugin:

resource
dpkg

Collect information about dpkg version

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Gets info on the version of dpkg installed
Plugin:

resource
interface

Collect information about interfaces

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of interface declarations on the device
Plugin:

resource
lsb

Collect information about installed system (lsb-release)

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates release info based on /etc/lsb-release
Plugin:

resource
meminfo

Collect information about system memory (/proc/meminfo)

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates resource info based on /proc/meminfo
Plugin:

resource
package

Collect information about installed software packages

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of packages
Plugin:

resource
snap

Collect information about installed snap packages

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Generates a list of snap packages
Plugin:

resource
uname

Collect information about the running kernel

Category ID:

information_gathering
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Creates resource info from uname output
Plugin:

resource

Hotkey tests

  • The function keys from external keyboards shipped with the AIO system are supposed to work.

  • RGB Keyboard backlight. Backlight of the internal keyboard needs to be turned on and off when pressing the keyboard backlight key.

The following test units are covered in this category:

keys/fn-lock

<missing summary>

Category ID:

keys
Status:

Non-blocking
Purpose:

This test will test the Fn Lock key
Steps:

  1. Press the Fn+Fn Lock key. 2. Press the Fn+any hotkey.
Verification:

Do the keys work as expected?
After-suspend:

True
Plugin:

manual

Informational tests

The following test units are covered in this category:

acpi_sleep_attachment

Attach the contents of /proc/acpi/sleep for further analysis.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the contents of /proc/acpi/sleep if it exists.
Plugin:

attachment
codecs_attachment

Attach a report of installed codecs for Intel HDA.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a report of installed codecs for Intel HDA
Plugin:

attachment
cpuinfo_attachment

Attach a copy of /proc/cpuinfo

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a report of CPU information
Plugin:

attachment
dkms_info_attachment

Attaches json dumps of installed dkms package information.

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches json dumps of installed dkms package information.
Plugin:

attachment
dmesg_attachment

Attach a copy of dmesg or the current dmesg buffer to the test results.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a copy of /var/log/dmesg or the current dmesg buffer to the test results
User:

root
Plugin:

attachment
dmi_attachment

Attach a copy of /sys/class/dmi/id/*

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches info on DMI
Plugin:

attachment
dmidecode_attachment

Attach output of dmidecode

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches dmidecode output
User:

root
Plugin:

attachment
efi_attachment

Attaches firmware version info

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the firmware version
User:

root
Plugin:

attachment
info/buildstamp

Attaches the buildstamp identifier for the OS

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the buildstamp identifier for the OS
Plugin:

attachment
info/disk_partitions

Attaches info about disk partitions

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches information about disk partitions
User:

root
Plugin:

attachment
info/hdparm_name.txt

Attaches info from hdparm about {name}

Unit type:

template
Category ID:

info
Status:

Non-blocking
Description:

<missing description>
From template:

info/hdparm_name.txt
Template resource:

device
Template filter:

device.category == ‘DISK’
info/secure-boot-check

Check secure boot state

Category ID:

info
Status:

Non-blocking
Purpose:

Output whether secure boot is enabled or disabled
Plugin:

attachment
info/systemd-analyze

System boot-up performance statistics

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

shell
info/systemd-analyze-critical-chain

Print the tree of the time-critical chain of SystemD

Category ID:

info
Status:

Non-blocking
Purpose:

This job prints a tree of the time-critical chain of SystemD units.
Plugin:

shell
info/touchpad_driver

Gather touchpad name, driver name, and driver version information.

Category ID:

info
Status:

Non-blocking
Purpose:

Returns the name, driver name and driver version of any touchpad discovered on the system.
Plugin:

attachment
installer_debug.gz

Attach the installer’s debug log for diagnostic purposes.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the installer debug log if it exists.
User:

root
Plugin:

attachment
kernel_cmdline_attachment

Attach a copy of /proc/cmdline

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the kernel command line used to boot
Plugin:

attachment
lsmod_attachment

Attach a list of currently running kernel modules

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a list of the currently running kernel modules.
Plugin:

attachment
lspci_attachment

Attach a list of PCI devices

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches very verbose lspci output.
Plugin:

attachment
lspci_standard_config_attachment

Attach PCI configuration space hex dump

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a hex dump of the standard part of the PCI configuration space for all PCI devices.
Plugin:

attachment
lstopo_visual_attachment

Attach the output of lstopo command to present system topology.

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches the system topology as presented by the lstopo command
Plugin:

attachment
lsusb_attachment

Attach output of lsusb

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches a list of detected USB devices.
After-suspend:

True
User:

root
Plugin:

attachment
manifest

Hardware Manifest

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

This job loads the hardware manifest and exposes it as a resource.
Plugin:

resource
meminfo_attachment

Attach copy of /proc/meminfo

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches info on system memory as seen in /proc/meminfo.
Plugin:

attachment
modinfo_attachment

Attach modinfo information

Category ID:

info
Status:

Non-blocking
Purpose:

Attaches modinfo information for all currently loaded modules
Plugin:

attachment
modprobe_attachment

Attach the contents of /etc/modprobe.*

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of the various modprobe conf files.
User:

root
Plugin:

attachment
modules_attachment

Attach the contents of /etc/modules

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of the /etc/modules file.
Plugin:

attachment
recovery_info_attachment

Attach the recovery partition versions

Category ID:

info
Status:

Non-blocking
Purpose:

Attach the recovery partition version image_version is the preinstalled OS image version bto_version is only for dell_recovery Example: image_version: somerville-trusty-amd64-20140620-0 bto_version: A00_dell-bto-trusty-houston-15-A11-iso-20141203-0.iso
User:

root
Plugin:

attachment
sysctl_attachment

Attach sysctl configuration files.

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches the contents of various sysctl config files.
User:

root
Plugin:

attachment
sysfs_attachment

Attach detailed sysfs property output from udev

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches a report of sysfs attributes.
Plugin:

attachment
udev_attachment

Attach dump of udev database

Category ID:

info
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

Attaches a dump of the udev database showing system hardware information.
Plugin:

attachment

Input Devices tests

The following test units are covered in this category:

input/fixed_screen_orientation_on_product___index__

Check whether screen orientation is fixed on {product}

Unit type:

template
Category ID:

input
Status:

Non-blocking
After-suspend:

True
From template:

input/fixed_screen_orientation_on_product___index__
Template resource:

dmi
Template filter:

dmi.category == ‘CHASSIS’ and dmi.product not in (‘Tablet’, ‘Convertible’, ‘Detachable’)

LED tests

The following test units are covered in this category:

led/fn

Test the Fn key LED functionality by activating/deactivating the Fn keys locking.

Category ID:

led
Status:

Non-blocking
Purpose:

This test will test the Fn key LED.
Steps:

  1. Press the Fn+Fn Lock key to activate/deactivate Fn keys locking. 2. The Fn key LED should be switched on/off every time the key is pressed.
Verification:

Did the Fn key LED light as expected?
After-suspend:

True
Plugin:

manual

Media Card tests

The following test units are covered in this category:

Miscellaneous tests

The following test units are covered in this category:

install/apt-get-gets-updates

Ensure apt can access repositories and get updates without installing them, to aid in recovery from broken updates.

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Tests to see that apt can access repositories and get updates (does not install updates). This is done to confirm that you could recover from an incomplete or broken update.
User:

root
Plugin:

shell
miscellanea/chvt

Verify the system’s ability to switch between a virtual terminal and the X session.

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

This test will check that the system can switch to a virtual terminal and back to X
Steps:

  1. Click “Test” to switch to another virtual terminal and then back to X
Verification:

Did your screen change temporarily to a text console and then switch back to your current session?
After-suspend:

True
User:

root
Plugin:

user-interact-verify
miscellanea/device_check

Device Check

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Device check
Steps:

  1. Commence the test 2. Compare items on System Manifest to the devices known to udev
Verification:

Do the devices reported by udev match the devices on the Manifest?
Plugin:

user-interact-verify
miscellanea/oops_results.log

Attach the FWTS oops results for submission.

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

Attaches the FWTS oops results log to the submission
Plugin:

attachment
miscellanea/submission-resources

Check that data for a complete result are present

Category ID:

miscellanea
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

A meta-job that verifies the data necessary for a complete result submission are present. Failure indicates that the results are incomplete and may be rejected.
Plugin:

shell

Mobile broadband tests

The following test units are covered in this category:

mobilebroadband/cdma_connection

Ensure a CDMA based modem establishes and maintains a mobile broadband connection.

Category ID:

mobilebroadband
Status:

Non-blocking
Purpose:

Creates a mobile broadband connection for a CDMA based modem and checks the connection to ensure it’s working.
After-suspend:

True
Environment variable:

CDMA_CONN_NAME, CDMA_PASSWORD, CDMA_USERNAME
User:

root
Plugin:

shell
mobilebroadband/gsm_connection

Establish and verify GSM mobile broadband connection.

Category ID:

mobilebroadband
Status:

Non-blocking
Purpose:

Creates a mobile broadband connection for a GSM based modem and checks the connection to ensure it’s working.
After-suspend:

True
Environment variable:

GSM_APN, GSM_CONN_NAME, GSM_PASSWORD, GSM_USERNAME
User:

root
Plugin:

shell

Power Management tests

Suspend/Resume time

On average, resume time needs to be less than 5 seconds, with an overall suspend/resume time of less than 10 seconds.

The following test units are covered in this category:

power-management/fwts_wakealarm-log-attach

Attach and display fwts wakealarm test log.

Category ID:

power-management
Status:

Non-blocking
Purpose:

Attach log from fwts wakealarm test
Plugin:

attachment
power-management/light_sensor

Test the functionality of the Ambient Light Sensor by checking if sensor values and screen backlight change when covered.

Category ID:

power-management
Status:

Non-blocking
Purpose:

This test will check your Ambient Light Sensor work, if you don’t have it, please skip this test.
Steps:

  1. Make sure “Automatic brightness” is ON in Power settings. 2. Locate the Ambient Light Sensor, which should be around the Camera. 3. Cover your hand over the Ambient Light Sensor. 4. When the backlight dims, press Enter to start testing. 5. Wait until the message “Has ambient light sensor” is printed on the screen and wave your hand slowly during testing.
Verification:

Did the Ambient Light Sensor values change when you _shook_ your hands over the sensor? Did the Screen backlight also change?
After-suspend:

True
Plugin:

user-interact-verify

Snapd

The following test units are covered in this category:

snappy/snap-install

Test the snap install command is working

Category ID:

snapd
Status:

Non-blocking
Purpose:

The store should contain the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap. The test makes sure this can be downloaded and installed on the system.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-list

Test that the snap list command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

If snap list command is working then should at least find the ubuntu-core package.
Plugin:

shell
snappy/snap-refresh-automated

Test whether the snap refresh command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

The test will install the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap from the stable channel and then refreshes it to the edge channel and compares the revision before and after the refresh.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-remove

Test the snap remove command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

After having installed the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap, check it can be removed.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-reupdate-automated

Test the snap refresh command works after blacklisting.

Category ID:

snapd
Status:

Non-blocking
Purpose:

Checks that the {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap can be refreshed after removal of the blacklisted revision.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/snap-revert-automated

Test the snap revert command is working.

Category ID:

snapd
Status:

Non-blocking
Purpose:

Checks if the edge channel {{ __checkbox_env__.get(“TEST_SNAP”, “test-snapd-tools”) }} snap is reverted back to the one from stable.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-snaps-confinement

Test all the snaps’ confinement

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test all the snaps’ confinement, devmode, revision. Make sure the confinement is “strict”, devmode is “False”, and revision should not be sideloaded.
Plugin:

shell
snappy/test-store-config-store

Test that image is using the correct snappy store configuration.

Unit type:

template
Category ID:

snapd
Status:

Non-blocking
Environment variable:

STORE_ID
From template:

snappy/test-store-config-store
Template resource:

com.canonical.certification::model_assertion
snappy/test-store-install-beta

Snappy install command - beta channel store

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test the snappy install command is able to install and remove snap in beta channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-store-install-edge

Snappy install command - edge channel store

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test the snappy install command is able to install a snap in the edge channel store.
Environment variable:

CHECKBOX_RUNTIME, SNAPD_POLL_INTERVAL, SNAPD_TASK_TIMEOUT, TEST_SNAP
User:

root
Plugin:

shell
snappy/test-system-confinement

Test if the system confinement is strict

Category ID:

snapd
Status:

Non-blocking
Purpose:

Test if the system confinement is “strict” If not, list the missing features
Plugin:

shell

Suspend tests

The following test units are covered in this category:

suspend/oops_results_after_suspend.log

Attach FWTS oops results log post-suspend.

Category ID:

suspend
Status:

Non-blocking
Purpose:

Attaches the FWTS oops results log to the submission after suspend
Plugin:

attachment
suspend/suspend-single-log-attach

Attaches the log from the single suspend/resume test

Category ID:

suspend
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
Plugin:

attachment
suspend/suspend-time-check

Ensure time to suspend/resume is under threshold

Category ID:

suspend
Status:

Non-blocking
Purpose:

Checks the sleep times to ensure that a machine suspends and resumes within a given threshold
Plugin:

shell

Touchpad tests

Resistive touchpads are tested for single touch and multitouch. The experience of a resistive touchpad is allowed to be worse than a capacitive touchpad (needs higher FTF - force to fire - to active cursor moving).

Functionality tested:

  • Single touch, including single tap and double tap

  • Scrolling feature (horizontal and vertical) should work either with the edge scrolling option or the 2 finger scrolling option.

The following test units are covered in this category:

touchpad/palm-rejection

Evaluate touchpad’s palm rejection feature by ensuring it ignores palm touches.

Category ID:

touchpad
Status:

Non-blocking
Purpose:

This test checks if the touchpad ignores palm touches.
Steps:

Select “Test” and follow the instructions on the screen.
Verification:

The cursor should not have moved.
After-suspend:

True
Plugin:

user-interact
touchpad/palm-rejection-firmware-labeling_product_slug

Check palm rejection firmware/labeling for touchpads

Unit type:

template
Category ID:

touchpad
Status:

Non-blocking
After-suspend:

True
From template:

touchpad/palm-rejection-firmware-labeling_product_slug
Template resource:

device
Template filter:

device.category == ‘TOUCHPAD’ and device.driver == ‘hid-multitouch’

Touchscreen tests

The following test units are covered in this category:

touchscreen/multitouch-rotate

Check touchscreen pinch gesture for rotate

Category ID:

touchscreen
Status:

Non-blocking
Purpose:

Check touchscreen pinch gesture for rotate
Steps:

  1. Commence the test 2. Using 2 fingers, rotate the blue square until it turns green, then release it.
Verification:

Did the blue square rotate following the gesture?
After-suspend:

True
Plugin:

user-interact-verify

TPM 2.0 (Trusted Platform Module)

TPM 2.0 functionality will be tested using clevis encrypt and decrypt commands with both RSA and ECC keys with the following options:

  • hash algorithm: sha256

  • pcr algorithm bank to use for policy: sha256

  • pcr ids: 0 and 1 only

The following test units are covered in this category:

tpm2/fwts-event-log-dump

Dump the contents of the TPM Event Log

Category ID:

tpm2
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

The information in the TPM Event Log can be useful in debugging problems with TPM command support and adherance to standards. This can be of particular help when checking whether a device can support Ubuntu Core Full Disk Encryption.
User:

root
Plugin:

attachment

USB tests

The following test units are covered in this category:

usb-c/c-to-ethernet-adapter-insert

Check if USB Type-C to Ethernet adapter works

Category ID:

usb
Status:

Non-blocking
Purpose:

This test will check if system detects network interface of the Type-C to ethernet adapter.
Steps:

  1. Prepare USB Type-C to Ethernet adapter 2. Start the test 3. When the message “INSERT NOW” is shown, plug in the adapter to Type-C port
Verification:

<missing verification>
After-suspend:

True
Plugin:

user-interact
usb/storage-detect

Detect storage partitions on a device on the USB bus

Category ID:

usb
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell

Wireless networking tests

  • Wi-Fi interfaces are tested for connection to access points configured for Wi-Fi 6E protocol.

  • network-manager hotspot mode.

The following test units are covered in this category:

wireless/check_iwlwifi_microcode_crash_interface

Check there have been no iwlwifi crashes

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
After-suspend:

True
From template:

wireless/check_iwlwifi_microcode_crash_interface
Template resource:

device
Template filter:

device.driver == ‘iwlwifi’
wireless/detect

Detect if at least one Wireless LAN device is detected

Category ID:

wireless
Status:

Non-blocking
Purpose:

<missing purpose>
Description:

<missing description>
After-suspend:

True
Plugin:

shell
wireless/nm_connection_restore_interface

Restore any NetworkManager 802.11 configurations after testing

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
Description:

<missing description>
After-suspend:

True
From template:

wireless/nm_connection_restore_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’
wireless/nm_connection_save_interface

Save any NetworkManager 802.11 configurations prior to testing

Unit type:

template
Category ID:

wireless
Status:

Non-blocking
Description:

<missing description>
After-suspend:

True
From template:

wireless/nm_connection_save_interface
Template resource:

device
Template filter:

device.category == ‘WIRELESS’ and device.interface != ‘UNKNOWN’

Appendix A. FWTS tests

As part of the certification process, we run a series of firmware tests that are part of the Canonical Firmware Test Suite. In general, any HIGH or CRITICAL error found in the fwts log can cause potential errors in the system and should be looked at by OEMs/ODMs.

Category

Test Item

Description

Information

acpidump

Check ACPI table acpidump.

Information

version

Gather kernel system information.

ACPI

acpitables

ACPI table settings confidence checks.

ACPI

apicinstance

Check for single instance of APIC/MADT table.

ACPI

hpet_check

High Precision Event Timer configuration test.

ACPI

mcfg

MCFG PCI Express* memory mapped config space.

ACPI

method

ACPI DSDT Method Semantic Tests.

CPU

mpcheck

Check Multi Processor tables.

CPU

msr

CPU MSR consistency check.

CPU

mtrr

MTRR validation.

System

apicedge

APIC Edge/Level Check.

System

klog

Scan kernel log for errors and warnings.