Appendix B - Re-Testing and Installing Updated Tests
====================================================

Occasionally, a test will fail, necessitating re-testing a feature. For
instance, if a USB flash drive is defective or improperly prepared, the
relevant USB tests will fail. Another common source of problems is
network tests, which can fail because of busy LANs, flaky switches, bad
cables, and so on. When this happens, you must re-run the relevant
test(s). Broadly speaking, there are two ways to re-run tests: via a
limited test script and by installing updated test scripts.

Running a Limited Test Script
-----------------------------

In addition to the ``certify-ubuntu-server`` test script, several others are
provided with the Server Test Suite:

- If you're testing a System-on-Chip (SoC) rather than a production
  server, you should run ``certify-ubuntu-server-soc``.

- If you're testing a virtual machine, you should run
  ``certify-ubuntu-server-vm``.

- The ``test-firmware`` command runs firmware tests.

- The ``test-functional`` command runs functional tests.

- The ``test-gpgpu`` command runs tests on nVidia GPGPUs. (See
  :doc:`appendix-g-setup-gpgpu` for important information
  related to GPGPU testing.)

- The ``test-memory`` command runs memory tests.

- The ``test-network`` command runs network tests.

- The ``test-network-underspeed`` command runs the network tests with the speed
  check disabled. This is helpful in situations where a network device reports
  an incorrect maximum speed.

- The ``test-nvdimm`` command will run the memory and storage focused tests and
  some NVDIMM health checks. This launcher is intended for testing NVDIMMs
  configured in mixed mode. (See :ref:`configure-dcpmm` for
  more information.)

- The ``test-storage`` command runs tests of storage devices.

- The ``test-stress`` command runs CPU, RAM and storage stress tests.

- The ``test-usb`` command runs tests of USB ports.

- The ``test-virtualization`` command runs virtualization (KVM and
  LXD) tests.

If you're testing NVDIMMs alone, you should note that ``test-nvdimm`` will run
both memory and storage stress tests and thus will take a while to run.  If
your NVDIMMs are configured only in memory or storage mode you can save some
time by using the ``test-memory`` or ``test-storage`` launchers respectively.

If you're testing Ubuntu 20.04, change the version number in commands that
include it. Consult your Partner Engineer if you need help
deciding which of these tests to run.

When the test run completes, submit the test result in the same way you
would for a complete test run. You can then request a certificate based on
the main results (the one with the most passed tests) and refer to the
secondary set of results in the certificate notes. This procedure ensures
that all the necessary data will be present on C3.