Generic Test Plans for Commercial Devices

This document lists the tests to be run during certification of IoT devices.

Benchmarks tests

  • Cached read timing benchmark of sda using hdparm

  • Raw read timing benchmark of sda using hdparm

Bluetooth

  • Basic BNEP Socket - Success

  • Bluetooth input device works

  • Bluetooth - BlueZ Self Tests

CPU

  • Validate that the Vector Floating Point Unit is running on ARM device

  • Tests the CPU for clock jitter

  • Run C-States tests

  • Attach C-States test log

  • Test that the CPU can run at its max frequency

  • Attach CPU max frequency log

  • Test offlining of each CPU core

  • Test the CPU scaling capabilities

  • Attach CPU scaling capabilities log

  • Check CPU topology for accuracy between proc and sysfs

Disk

  • Gathers information about each disk detected

  • Disk stress_ng test

  • Disk performance test

  • Disk statistics

  • Disk I/O stress test

Ethernet

  • Ensure ethernet port is physically disconnected

  • Report info about available network devices

  • Ethernet port hotplug detection test

  • Ethernet port hotplug disconnect step

  • Can ping another machine over Ethernet port

  • Multi-NIC Iperf3 stress testing for NIC wwp0s20u4i6

IPDT

  • CPU Frequency Test

  • Integrated Memory Controller Test

  • Floating Point Test

  • Prime Number Generation Test

  • Platform Controller Hub Test

  • Sample Production Bit Checker

  • AVX Test

  • Brand String Test

  • Cache Test

  • Genuine Intel Test

  • MMX/SSE Test

  • Temperature Test

LEDs

  • Bluetooth LED behavior

  • Power LED behavior when powered

  • Power LED behavior when suspended

  • Serial ports LED behavior

Memory tests

  • Stress test of system memory

  • Check amount of memory reported by meminfo against DMI

Power and Power Management

  • Verify watchdog daemon is started during boot

  • Verify the watchdog keepalive daemon is up once watchdog daemon is killed

  • Load watchdog kernel module when boot

  • Test that the watchdog module can trigger a system reset

Real Time Clock (RTC)

  • Test that RTC functions properly (if present)

Serial Port

  • Serial debugging console is enabled and operational

Snappy Stress Tests

  • Run the stress-ng stressors for class cpu

  • Run the stress-ng stressors for class cpu-cache

  • Run the stress-ng stressors for class memory

  • Run the stress-ng stressors for class os

  • Run the stress-ng stressors for class pipe

  • Run the stress-ng stressors for class scheduler

  • Run the stress-ng stressors for class vm

Snappy Ubuntu Core

  • Automatically rollback after failed boot after upgrade

  • Re-apply the update after rollback using the “snappy” tool

  • Rollback system update using the “snappy” tool

  • SSH is enabled and operational

  • Test the snap install command is able to install hello-world snap.

  • Test that the snap list command is working.

  • Test the snap remove command is able to remove the hello-world snap.

  • Test that the snap find command is working.

  • Update the system using the “snappy” tool

  • WebDM (Device Manager) is enabled and operational

USB 3.0

  • USB 3.0 storage device insertion detected

  • USB 3.0 storage device removal detected

  • USB 3.0 storage device read & write works

Wi-Fi

  • Create 802.11a Wi-Fi Access Point

  • Create 802.11ad Wi-Fi Access Point

  • Create 802.11b Wi-Fi Access Point

  • Create 802.11g Wi-Fi Access Point

  • Connect to unencrypted 802.11ac Wi-Fi network

  • Connect to unencrypted 802.11b/g Wi-Fi network

  • Connect to unencrypted 802.11n Wi-Fi network

  • Connect to WPA-encrypted 802.11ac Wi-Fi network

  • Connect to WPA-encrypted 802.11b/g Wi-Fi network

  • Connect to WPA-encrypted 802.11n Wi-Fi network

  • System can discover Wi-Fi networks

Wireless Wide Area Network

  • Identify if WWAN module is missing

  • Verify a GSM broadband modem can create a data connection

  • Gather device info about WWAN modems

Zigbee

  • Identify if Zigbee module is missing

  • Get product identification information of the Zigbee device

  • Read the current licence key assigned to the Zigbee device

  • Scan for Zigbee PANs and print details of any found

I2C

  • Check number of detected I2C bus

  • Check if any I2C device detected

TPM2

  • Start the resource manager daemon

  • tpm2_takeownership

  • tpm2 NV tools

  • tpm2_listpcrs

  • tpm2_getrandom

  • tpm2_load

  • tpm2_loadexternal

  • tpm2_evictcontrol

  • tpm2_hash

  • tpm2_quote

  • tpm2_unseal

  • tpm2_akparse

  • tpm2_certify

  • tpm2_getpubak

  • tpm2_makecredential

  • tpm2_activecredential

  • tpm2_readpublic

  • tpm2_rsaencrypt

  • tpm2_rsadecrypt

  • tpm2_encryptdecrypt

  • tpm2_sign

  • tpm2_verifysignature