Internet of Things Certified Hardware Coverage for Ubuntu Core 18 / Ubuntu 18.04¶
Introduction¶
This document lists the coverage for certification of Internet of Things (IoT) devices with Ubuntu images. IoT devices can be certified with the following image types:
Ubuntu Core 18
Ubuntu Server 18.04
Ubuntu Desktop 18.04
The guide applies to devices submitted to Canonical through one of the following programmes:
IoT Devices Enablement Programme with Certification
IoT ODM Partner Programme
For each test job, one of the following certification statuses is specified:
- Blocking
Features that are required for certification. If any of the blocking tests fails, the certification will fail.
- Non-blocking
Features that are tested but not mandatory for certification. Failure in non-blocking tests will not prevent certification. However, a note will be added to the certificate to inform potential customers or users.
Note
Only categories of hardware are tested and not specific types of hardware. For example, tests are run to verify USB controllers work, but the type of peripheral(s) used during those tests are not specified.
Coverage is flexible based on customer requirements (for example, if a device’s use cases don’t require LEDs, then LEDs can be non-blocking)
Full test descriptions can be found in Canonical certification site for partners:
Blocking¶
Firmware¶
Ubuntu Core Series 16 is installed using the factory default BIOS or UEFI, with the default options (including SecureBoot, if that’s the default setting). Firmware needs to be compliant with Canonical Firmware Test Suite (FWTS).
It is recommended that after running Canonical fwts with the list of tests defined in the Appendix A, ideally, no CRITICAL or HIGH failures should be reported, but those are not automatically certification blockers.
Processors¶
ia32 (x86), x86_64 and ARM processors are tested to ensure proper functionality. We will test specific features as:
CPU’s performance states (frequency up and down in runtime)
CPU’s sleep states (cpu on and off in runtime)
Running CPU at its maximum frequency
We will also include a general stress test performed for 120 minutes to verify that the system can handle a sustained high load for a period of time. This test uses the tool “stress” available in the Universe repositories.
For Intel CPU’s, the IPDT (Intel Processor Diagnostic Tool) test suite will be run. The diagnostic checks for brand identification, verifies the processor operating frequency, tests specific processor features, and performs a stress test on the processor.
Memory¶
Proper detection of the amount of memory installed is required (the amount of memory installed is the memory seen by the OS).
Internal Hard Drives¶
All internal hard drives are tested to be properly detected. On all of them, an in-house performance test is run. All disks need to report at least a read performance of 15MB/s.
Networking¶
Ethernet. Connections are tested for functionality, but not for performance.
WWAN connections (3G/4G) - GSM - CDMA - LTE
Wireless: - 802.11 b/g/n/ac connection
USB controllers¶
USB controllers are tested using storage devices in all available USB ports. And the USB human interface device, keyboard or mouse, should be working properly with any USB port.
USB 2.0
USB 3.0 SuperSpeed mode
Keyboard or mouse (basic functionality)
Mediacard readers¶
Mediacard readers are tested for read and write for the following type or cards:
SD
SDHC
Bluetooth 4.0¶
Bluetooth LE (Smart and Smart Ready) is tested for device scanning and pairing. Apart from pairing, several profiles are specifically tested and required:
File transfer (OBEX)
Audio (A2DP)
HID Over GATT Profile (HOGP), Low-Energy keyboard or mouse with basic functionality
Advanced Configuration and Power Interface¶
Boot/Reboot
Both cold boot and soft boot are tested and required to work.
I2C¶
All devices attached to the I2C bus must be found and able to be read, This includes:
Temperature sensors
Humidity sensors
Accelerometers
LEDs¶
When LEDs exist, they will be tested by following some basic expectations here. The actual behavior may vary depending on the hardware design. To ensure that the behavior is working as expected, please be sure to test against specifications obtained from OEM, as each OEM may have different defined behavior for LEDs. The following LEDs are tested:
Power
Serial Port LEDs (indicating activity)
Thunderbolt 2.0 or 1.0¶
Audio output needs to be undistorted over this port.
Storage devices with hot plugging capability needs to work on it with BIOS set to “No security” option.
Display hot plugging and different modes (mirror, extended, just internal, just external) are required to work.
Daisy-chain should work with a storage device and a monitor chained together.
USB 3.1 (Type C)¶
The following adapters/peripherals should work:
Type-C to Type-A adapter - Storage devices - Keyboard or mouse (basic functionality)
In addition, USB type-C must work in host mode.
TPM 2.0¶
TPM 2.0 should be able to be activated from the BIOS.
For TPM 2.0, all tpm2-tools commands have to work and pass the upstream integrations tests (NV, Attestation, Key management, Encryption, Signing and Utilities tests)
GPIO¶
We test the functionality of individual GPIO lines when the associated controller driver in the kernel implements a GPIO Sysfs Interface via the gpiolib implementers framework. In such cases, the GPIO system may be tested in two ways:
Direct: - GPIO controllers are exposed through sysfs - GPIO lines are accessible by the user
Indirect: - Communication with device connected via GPIO
Serial Ports¶
Tests are carried out on ports that provide access via the Linux tty layer. The exact tests performed depend on the physical characteristics of the driver/receiver hardware. The possible tests include:
Ensure expected number of devices are available
Looped tests: - RS232 Ports: perform loopback test to ensure RX/TX - RS422/485 Ports: connect together to ensure RX/TX
Machine to Machine tests: confirm that a connection can be made to another PC device and RX/TX is operational
Audio¶
Output sound needs to be undistorted between 0%-100%. Output lines tested:
Internal speakers
3.5mm headphones
HDMI audio output
DisplayPort audio output
Input needs to be recorded undistorted between 0%-100%. Input lines tested:
Internal microphone
3.5mm microphone
Plug detection: when a new audio line input or output is plugged in the system, it needs to be recognized.
Non-blocking¶
Mediacard readers¶
Mediacard readers are also tested for read and write for the following type or cards:
MMC (before and after suspend)
CANBus¶
CANBus is tested to ensure that the adapter is present and can be communicated with via CANBus configuration commands. It is also tested that data can be sent over the channel using serial commands.
IoT Communication Protocols¶
The following protocols will be tested to ensure devices can be found and commands can be issued:
Zigbee
Z-wave
Advanced Configuration and Power Interface¶
Suspend/Resume (S3)
A 30 cycle suspend/resume stress test is performed using the fwts S3 test. The test is passed if all 30 cycles complete without failure. Any errors reported in the fwts log for the 30 cycle suspend/resume stress test are informational only and do not affect the outcome of the test, however, we do recommend examining and fixing any failures noted, as they indicate firmware non-compliance with standards.
Apart from the stress test, a single cycle suspend/resume is performed, if it’s a hybrid graphic system, suspend and graphic related functionalities are required to work flawlessly on both Performance mode and Power Saving mode, and the following features and devices are tested and need to work after suspend:
CPU
Memory
Networking (Wifi, Ethernet)
Audio
Bluetooth
USB controllers
Input devices
Mediacards
Hibernate/Resume (S4)
A general 30 cycle hibernate/resume stress test is performed. This test is performed using the fwts S4 test. No Critical/High failures in the fwts log are allowed.
Watchdog Timer
A test will be performed to verify that any kernel module needed for watchdog timers are loaded and working as expected.
LEDs¶
Cloud LED
Wireless LED
Bluetooth LED
WWAN LED
USB 3.1 (Type C)¶
USB Type C (USB3.1) are tested using various adapters/peripherals , as the new USB Type C interface supports more types of devices (i.e. Video, Power, etc). The following adapters/peripherals should work.
Type-C to Display Port - Display hot plugging and different modes (mirror, extended, just internal, just external) are required to work. - Audio output needs to be undistorted over this port.
Storage devices
Keyboard or mouse (basic functionality)
Thunderbolt 3¶
Audio output needs to be undistorted over this port.
Storage devices with hot plugging capability needs to work on it with BIOS set to “No security” option.
Display hot plugging and different modes (mirror, extended, just internal, just external) are required to work.
Daisy-chain should work with a storage device and a monitor chained together.
Q & A¶
Does changing the speed of processors require a new certificate?
No. Only changing the CPU family would require retesting and issuing a new certificate.
Appendix A. FWTS tests¶
As part of the certification process, we run a series of firmware tests that are part of the Canonical Firmware Test Suite. In general, any HIGH or CRITICAL error found in the fwts log can cause potential errors in the system and should be looked at by OEMs/ODMs.
Category |
Test Item |
Description |
---|---|---|
Information |
acpidump |
Check ACPI table acpidump. |
Information |
version |
Gather kernel system information. |
ACPI |
acpitables |
ACPI table settings confidence checks. |
ACPI |
apicinstance |
Check for single instance of APIC/MADT table. |
ACPI |
hpet_check |
High Precision Event Timer configuration test. |
ACPI |
mcfg |
MCFG PCI Express* memory mapped config space. |
ACPI |
method |
ACPI DSDT Method Semantic Tests. |
CPU |
mpcheck |
Check Multi Processor tables. |
CPU |
msr |
CPU MSR consistency check. |
CPU |
mtrr |
MTRR validation. |
System |
apicedge |
APIC Edge/Level Check. |
System |
klog |
Scan kernel log for errors and warnings. |