General Policies ================ Comprehensive Server Certification ---------------------------------- In order to ensure the best user experience possible when running Ubuntu on Partner Servers, Canonical requires that hardware partners sufficiently test the list of Vendor Approved Options that are applicable to a given Server Model. As we understand this can be a very long list of devices, your Partner Engineer will work with you to determine a minimal amount of testing necessary to maximise coverage of options across all server models. This means that not every single device will need to be tested, and none will need to be tested more than once or twice. For this, we require a spec sheet that includes a listing of any Vendor Approved Options for each Server Model to be certified so that we can help determine the matrix of tests needed. Hardware Partners can send Spec Sheets directly to your Partner Engineer, or can provide a URL to online copies of Spec Sheets. As this testing is required to help guarantee the quality of Ubuntu on your hardware, and to ensure that end users have the best experience possible when deploying your hardware into an Ubuntu environment in their datacenters, failure to perform sufficient testing of options may result delayed or denied certification requests. Devices Specific Information ---------------------------- Boot Device ''''''''''' The Server should be configured so that the only item in the boot order is a single network device configured for Network Booting. If it is impossible to remove HDD/SDD/NVMe devices from the boot menu, the Network Boot device should always be in the first position to ensure the machine ALWAYS boots from Network. MAAS will instruct the server, during the network boot, to boot from onboard storage if necessary. CPUs and RAM '''''''''''' Any CPU from a given CPU Family (e.g. Cascade Lake, Ice Lake, Rome, Milan) will count for the entire CPU family. You do not need to re-test for each CPU unless the CPU is from a different family. It is recommended that you do test different models of CPU across a given server line to help broaden the scope of testing. Any DIMM size may be tested, though we strongly encourage you to use the largest DIMM size available. Jumps in DIMM size do not require additional testing. DCPMMs (NVDIMM devices) ''''''''''''''''''''''' Systems that support Intel's Optane DataCenter Persistent Memory Modules must include those DCPMM devices. For convenience, DPCMMs can be configured in mixed mode where supported in a mix of at least 30% RAM and 70% Storage. (That percentage will vary based on the amount of DCPMM storage installed). Where Mixed Mode is not supported, then testers will need to test the DCPMM devices in both Memory Mode and App Direct (Storage) Mode, which requires reconfiguring the DCPMM devices, then recommissioning in MAAS and re-running the appropriate test command (one of test-memory or test-storage). When configuring the DCPMMs for storage a percentage should be dedicated to all block store modes: fsdax, sector, and raw. As with any other storage device, each DCPMM storage volume should be properly partitioned, formatted, and mounted prior to testing. HDD, SSD, NVMe '''''''''''''' For HDDs and SSDs, only one of each supported interface type needs to be tested. You should use the largest HDD or SSD of each type where possible. Systems that support NVMe devices should include NVMes where possible as well. RAID Controllers '''''''''''''''' ALL RAID controllers must be tested. Exceptions can be made to this for very minor variances in RAID Controller Models. An example of this exception would be two SAS RAID Controllers where they are essentially identical except for the addition or deletion of a Cache Battery. Your Partner Engineer or the Certification Team will work with you to determine if a RAID Controller is exempt from testing. This will be handled as we build the test matrix for all Vendor Approved Options that apply to each Server Model. HBA ''' HBAs must be tested. We will work with you to determine which particular ones from the list of Vendor Approved Options will need to be tested and which can be risk assumed. CNA and Network Devices ''''''''''''''''''''''' CNAs and NICs must be tested, CNAs should be tested in Network mode. Canonical reserves the right to require further CNA testing in other modes as necessary (for instance a CNA may need to be tested as an iSCSI initiator as well as a Network card). For Network Devices, all ports must be configured and tested on appropriate network segments (e.g. a 100Gb port must be connected to a network segment that is at least 100Gb and the iperf target must also be at least 100Gb). GPGPUs '''''' GPGPU testing is now supported for nVidia GPGPUs. This is a separate test and requires the installation of nVidia's CUDA libraries and drivers, a system reboot, and running a separate test suite. All GPGPU models should be tested, at this time there is no allowance for "representative" samples on GPGPU devices. Please refer to Appendix G - Setting Up and Testing a GPGPU of the :doc:`../Self-Test_Guide/Self-Test_Guide` for more information. Anything Else ''''''''''''' Any device not explicitly outlined above must be tested. If you have any questions about what should or should not be tested, please contact your Partner Engineer. Application of Test Results for Vendor Approved Options ------------------------------------------------------- Once a Vendor Approved Option has been tested in ANY Model of a Partner's Server Line, that Vendor Approved Option will be considered certified for the full Server Line. Thus, if a Server Line has 10 Models, and the Vendor sells 1 (one) model of 100Gb Network Controller, once that controller has been validated in Model 1, it will also be automatically considered certified for Models 2 - 10. This does not apply to Blade systems. A PCIe card tested in a rack or tower server chassis does not remove the need to test a Mezzanine card with the same chipset in a Blade or Compute Sled style system. Display of Status of Vendor Approved Options -------------------------------------------- Once a Model is certified it will be publicly listed on the `Ubuntu Certification Website`_ described later in this document. In addition to the Model, all Vendor Approved Options that apply to the Certified Model will be listed alongside that Model. Each Vendor Approved Option will show it's status (e.g. Certified, Untested, Unsupported, etc) and at least the Ubuntu Version or Kernel Version that the Option was tested against. .. _Ubuntu Certification Website: https://ubuntu.com/certified/server This will give customers a more complete view of what Models and Options are supported by Ubuntu Server. This will also make creating BOMs for projects easier as there will no longer be any question if the components in a given BOM have been certified. Inadequate Test Coverage of Vendor Approved Options --------------------------------------------------- We do not expect the Partner to test all Vendor Approved Options, which is why your Partner Engineer will work with you to create a test matrix and highlight the minimum devices to be tested to provide the maximum coverage of all Options for the Partner Server Model. Likewise, testing of Vendor Approved Options can be spread out over time and across Certification tests for several servers. Thus if you have a list of 20 Vendor Approved Options and 10 Server Models, you could spread that test effort out so that each Server Model only features 2 Options. However, as alluded to above, if adequate coverage is not seen within a reasonable amount of time, we reserve the right to delay or reject certificates, and possibly revoke existing certifications as a last resort. OS and Kernel Versions ---------------------- Certifications are available for the two most recent LTS versions of Ubuntu Server. At this time, this includes **22.04 LTS** and **24.04 LTS**. Certification is never granted for Interim releases of Ubuntu Server (non-LTS versions such as 22.10, 23.04, or 23.10). Certification Testing should be performed using the LTS release and GA kernel initially (e.g. 24.04 LTS and the ga-24.04 kernel option in MAAS). When hardware cannot pass certification because of hardware support issues with the GA kernel, testers may use the most recent HWE kernel option (e.g. 24.04 LTS and the hwe-24.04 kernel option in MAAS) to perform testing. Certification is valid from the certified kernel onward including all subsequent kernel updates and HWE kernel releases for that LTS. In other words, if a system is certified using 22.04 LTS and the 5.15 GA kernel, that system is certified for the 5.15 (22.04 GA, 22.04.1 GA), 5.19 (22.04.2 HWE), 6.2 (22.04.3 HWE), 6.5 (22.04.4 HWE) and eventually 6.8 (22.04.4 HWE) kernels that comprise the 22.04 LTS and LTS Point Release family. If a system is certified using 22.04 LTS and the HWE kernel, then the certification is valid from that HWE kernel version onward. Thus if the system was certified using 22.04.3 LTS and the 6.2 HWE kernel, the system is considered certified for the 22.04.3 LTS HWE Kernel version 6.2, the 22.04.4 LTS HWE Kernel version 6.5, and the 22.04.4 LTS HWE Kernel version 6.8. Any exceptions to this policy will be decided on a case by case basis before the certification can be issued. Package Versions ---------------- Installation should be performed using the Ubuntu images and kernels provided by the default MAAS image stream hosted at https://maas.io Deployed OSs for Certification should *not* be updated with current package versions unless explicitly instructed to by the Server Certification Team. Testing should always start with the GA version of Ubuntu unless the HWE kernel is necessary to provide support for newer hardware. Doing certification primarily on the GA release will ensure the longest support lifetime for customers, while the HWE release will ensure customers are able to use the newest hardware. Certification should always be performed using the most recent version of the Certification Suite. This is installed separately after the OS Version has been installed on the SUT. Installation is usually performed automatically as part of the deployment process. Certification Lifetime ---------------------- .. figure:: ../images/LTS_Certification_Schedule.png :alt: This image shows a graphical representation of the typical Certification support schedule from one LTS to another. :width: 100% Figure 1: The typical Ubuntu Server Certification LTS Support Schedule. This is also available for download and closer examination from the Certification Portal: https://certification.canonical.com/server/lifecycle/ Certifications are valid from the point release they are issued against until the end of the lifetime for that LTS. For example, a system certified on Ubuntu 22.04.2 LTS will be considered certified for 22.04.2, 22.04.3 and 22.04.4 but will *not* be considered certified for 24.04 LTS or subsequent LTS releases. Those subsequent LTS releases will constitute a new LTS release family and will require separate certification of each Model and Vendor Approved Option. Models ------ For the purpose of Ubuntu Server Certification, we encourage the testing and certification of a Partner's entire server line, including user selectable Vendor Approved Options. For each Server Model, we will list all Vendor Approved Components that were tested. We will also list the supported/certified status of each of those components and that information will be plainly visible on the public certification website for each model the Vendor Approved Components are applicable. In order to ensure coverage of as many component options as possible, the Certification team will work with the Partner to decide on a test matrix to cover a model and its full breadth of component options. Changes to the Test Suite ------------------------- Suite Changes ''''''''''''' The Certification Suite is constantly evolving as new testing methods are employed, as technology changes, and as bugs are discovered and resolved within the Suite. Changes to existing test cases in a given LTS will *not* change the test requirements, and will likely only change the method used to test. Newly introduced tests are considered a Suite change and will not block current LTS certifications. For example, the KVM test has moved from using qemu directly to using LXD to manage the virtual machine. As this only changes how the test is conducted, the VM test is still considered a blocking test Conversely, the addition of a TPM 2.0 test would constitute a suite change and would *not* gate current LTS certifications, but MAY become a blocker for future LTS releases. Likewise, tests specifically for new technologies will not be blockers on the current LTS, but could become blockers on the next LTS. Test Requirements Changes ''''''''''''''''''''''''' The requirements for Certification are considered fluid up to the day the LTS is released. At that point, certification requirements are locked in and will not change for the life of that LTS. Any new test cases will be introduced as Non-Blocking items and will not gate certifications. Note that this only applies to additions to the requirements. Requirements can be eased (tests removed) at any time and *will* be applicable to all certifications going forward. An example of this would be the removal of the requirement for floppy disk testing as such devices are not in use any longer. Progression of New Tests '''''''''''''''''''''''' As the Suite *is* constantly evolving, there is a natural progression for tests that is applied throughout the development cycle. Any new test is introduced as a Non-Blocking item. This implies that the test must be run, but will *not* gate the certification effort for the current LTS. As we approach the next LTS, Non-Blocking tests are re-evaluated for promotion to Blocking (Required) tests and likewise, Blocking tests are evaluated for demotion to Non-Blocking or removal altogether. As a more concrete example, let's suppose a new Storage I/O stress test is introduced after 24.04 LTS is released but before 26.04 LTS. That new test would be introduced as a Non-Blocking test, thus any failures would *not* gate 24.04 certifications. This "Break-In" period is a chance to review and improve the test as well as gather data from various testing scenarios to determine its viability later on. As we approach 26.04, we would re-evaluate this new Storage Stress test. If it is seen as important and reliable enough, then it will be promoted to Blocking for 26.04 and be required to pass for all 26.04 certifications. Also keep in mind that even though this test would now be required for 26.04, it will still remain a Non-Blocking item for 24.04 certifications. Changes to Certification Policies ''''''''''''''''''''''''''''''''' The policies for Certification are subject to change at any time for any reason. That said, we make every effort to minimize policy changes and make modifications only where necessary for changing business needs. Virtualization -------------- Ubuntu as Host '''''''''''''' For all Servers that support virtualization, the KVM and LXD tests must be run with Ubuntu as the host OS. This will launch an Ubuntu guest and validate that virtualization functions. Certification does not apply to using any other operating systems as a virtualization guest OS. Ubuntu as Guest ''''''''''''''' In special situations, we will provide Certification of Ubuntu as a guest OS on a different host OS. These certifications are provided on a case-by-case basis and must be agreed upon by both Canonical and the Partner. Please discuss this with your Partner Engineer if you need to certify Ubuntu as a guest on your hypervisor. Virtual Machine Requirements '''''''''''''''''''''''''''' Guests or VMs created for the purpose of certifying Ubuntu as Guest on a non-Ubuntu host OS should have a minimum of 4 GiB RAM and at least 100 GiB of disk space to ensure the tests run successfully. Guests should also have at least one virtual NIC that can successfully ping the MAAS server and iperf target. Certifications of this type will use the “virtual-machine-full” test plan, which is a subset of the full server suite defined by the “server-full” test plan. KVM testing is generally not required for certification of Ubuntu as Guest scenarios as nested virtualization (e.g. running KVM inside a VM is considered an advanced/non-standard configuration.) This exception may not apply to certain special situations that are business goal dependent. That determination will be made by the Certification Team. System on Chip Certification ---------------------------- The Server Hardware Certification Team also provides System on Chip Certification Services for companies that produce SoCs meant to be used in server systems built by OEM/ODMs down the road. Application ''''''''''' SoC Certification applies *only* to Systems on Chip and reference boards that showcase those SoCs. It does not apply to production server systems based on SoCs. Additionally there is no inheritance upstream. So though an SoC may be certified by an SoC vendor like Ampere or HiSilicon, OEM/ODMs who build servers based on that SoC cannot also claim certification for their server. Server Certification Application '''''''''''''''''''''''''''''''' ARM64 SoC based servers can ONLY be certified if the SoC in use is also SoC certified. This is due to the need for enablement and ongoing maintenance of code specific to numerous SoCs. Also, SoC certification does not imply Server certification and Server certification does not imply SoC certification. Requirements '''''''''''' SoC certification is best thought of as a subset of Server Certification. The tools and test cases are the same, but SoCs have less stringent requirements for certification. For example, SoCs can have non-functional blocks at the time of Certification. The implication is that while an SoC may have non-functional blocks (such as USB3), those blocks will and should be enabled by the time an OEM/ODM is creating a server based on that SoC. Note that once a server product based on a certified SoC is presented for Server Certification, it must adhere to the more stringent Server Certification rules. In other words, once the SoC becomes a server, all hardware must work, with the only exception being non-accessible/non-included blocks. A compute card with no externally accessible USB ports, for example, will not need to pass the USB tests. Additionally, SoC certification does not imply any level of support beyond basic functionality where Server Certification does imply a level of support including Ubuntu Advantage and other avenues. Exceptions '''''''''' As noted above, SoC Certification is a subset of Server Certification with less stringent rules. Thus exceptions can be made for items that are non-functional at the time of SoC Certification. When these are encountered, the certification will have a note attached indicating what items are not considered certified and are non-functional or untested. Performance / Benchmark Testing ------------------------------- Canonical does not perform performance or benchmark testing as part of certification. Any benchmark or stress tools utilized are used strictly with the goal of introducing significant load to the system. It is the responsibility of the Partner to properly benchmark their own hardware with Ubuntu installed. Third Party Testing ------------------- Third Party Testing means testing hardware on behalf of another company. This happens when an OEM produces a system that is sold to a reseller who re-brands the hardware and sells at retail under the reseller's name and marketing model. Third Party Testing for Certification is ONLY allowed on a case-by-case basis and must be agreed upon by Canonical, the OEM who will be doing the testing and the Reseller (who may also be an OEM) who will be selling the hardware at retail. Any system tested in this manner MUST be readily identifiable as being the Reseller's system. See :ref:`system-identification` for more information Public Web Site --------------- All published Certificates are accessible via our public certification website found at * http://ubuntu.com/certified/server * http://ubuntu.com/certified/soc Public certificates will include Make/Model, release and pertinent hardware information including certification/supported status of any Vendor Approved Options applicable to the certified Make/Model. Public certificates will *not* include any Pass/Fail test information, private system data or other details that are not meant to be publicly accessible. Private Web Site ---------------- The private certification portal can be found at: * https://certification.canonical.com This site is often referred to as C3. Access to C3 is available only to Canonical employees and designated employees of partners participating in the Programme. The private site will provide the Partner with a history of all certified and registered models and a history of all submitted test results and all certificates. People with access must have an account on Launchpad (https://launchpad.net) and their account must be added to the appropriate access group by the Partner's PE or a member of the Certification team. Documentation ------------- All documentation for the Certification Programme is available on C3 in both PDF and HTML versions.