[AETHER-3277] Add documentation for System Tests
Change-Id: Ic0a0116a28255ae9c634da7e2169492da4d5666c
diff --git a/index.rst b/index.rst
index c73e54f..a5aedcb 100644
--- a/index.rst
+++ b/index.rst
@@ -105,6 +105,7 @@
testing/about_system_tests
testing/aether-roc-tests
+ testing/system-tests
testing/acceptance_specification
.. toctree::
diff --git a/testing/system-tests.rst b/testing/system-tests.rst
new file mode 100644
index 0000000..0c64a5d
--- /dev/null
+++ b/testing/system-tests.rst
@@ -0,0 +1,161 @@
+System Testing
+==============
+
+The Aether system is tested using Robot Framework.
+The tests are located inside the aether-system-tests repository and they are run nightly using a Jenkins job.
+
+Test Structure
+--------------
+
+At a high level, functional tests are ran by:
+
+- Configuring ROC with various slices required for tests using pre-defined input data.
+- ROC updates using pre-defined input data with patch updates built with the test framework.
+- Control plane and data plane validated using NG40.
+
+5G
+---
+Functional testing includes multiple slice creations, enable/disable of device groups,
+QoS validations, rate limiting tests (at UE, slice, application), application filtering tests, container restart tests.
+
+4G
+---
+Functional testing includes multiple slice creations, enable/disable of device groups,
+QoS validations, rate limiting tests (at UE, slice, application), application filtering tests, container restart tests.
+
+Development Prerequisites
+-------------------------
+
+Deploying each component is done with the use of Helm.
+Instructions can be found on `this page <https://docs.onosproject.org/onos-docs/docs/content/developers/deploy_with_helm/>`_.
+
+The following steps are needed to deploy the Aether system:
+
+- If applicable, remove ROC, CRDs, and other resources. Fetch Kubernetes config.
+- Deploy SD-Core components
+- Deploy SD-Fabric components
+- Deploy ROC
+- Wait for pods to be up
+- Sync with SD-Core adapter
+
+Running Tests
+-------------
+
+System tests are executed in the following Jenkins views:
+
+`Automated Test Jobs (Nightly) <https://jenkins.aetherproject.org/view/Aether%20System%20Tests/>`_
+
+Tests are executed in the following order after deployment:
+
+- Functional
+- Failure/Restart
+- Cleanup
+
+Test Setup
+----------
+
+.. code-block:: shell
+
+ # Set up virtual environment
+ cd aether-system-tests/
+ make ast-venv
+ source ast-venv/bin/activate; set -u;
+
+Running Tests (4G)
+------------------
+
+Functional, Failure/Restart, and Cleanup are executed similarly.
+Replace `${TEST_FILE}` with the test file to execute.
+This runs all tests in the suite. Add `-i ${TEST_TAG}` to robot arguments to run a specific test case.
+
+.. code-block:: shell
+
+ cd ${WORKSPACE_DIR}
+ robot -d ${WORKSPACE_DIR}/${TEST_TYPE}/robotlogs \
+ --debugfile ${WORKSPACE_DIR}/functionality/robotlogs/${TEST_FILE}_debug \
+ -o ${log_xml} \
+ -l ${log_html} \
+ -r ${report_html} \
+ -e notready \
+ -v ACC_CONTEXT:${accContext} \
+ -v AMP_CONTEXT:${rocContext} \
+ -v ACE_CONTEXT:${aceContext} \
+ -v UPF_NAMESPACE:${upfNamespace} \
+ -v UPF_NAMESPACE2:${upfNamespace2} \
+ -v UPF_NAMESPACE3:${upfNamespace3} \
+ -v PFCP_NAMESPACE:${pfcpNamespace} \
+ -v CORE_TYPE:${coreType} \
+ -v UPF_TYPE:${upfType} \
+ -v UPF_TYPE_NAME:${upfTypeName}\
+ -v NG40_HOST:${ng40Host} \
+ -v NG40_USER:${ng40User} \
+ -v NG40RAN_DIR:${ng40RanDir} \
+ -v NG40TEST_DIR:${ng40TestDir} \
+ -v CSV_DIR:${ng40CsvDir} \
+ -v PCAP_DIR:${WORKSPACE_DIR}/functionality/${pcapDir} \
+ -v NG40_LOG_DIR:${WORKSPACE_DIR}/functionality/${ng40LogDir} \
+ -v CDR_LOG_DIR:${WORKSPACE_DIR}/functionality/${cdrLogDir} \
+ -v CONTAINER_LOG_DIR:${WORKSPACE_DIR}/functionality/${containerLogDir} \
+ -v POD_NAME:${pod} \
+ aether-system-tests/system-tests/tests/4G/functional/${TEST_FILE}.robot || true
+
+Variables
+---------
+
+| `${WORKSPACE_DIR}` - Current workspace directory for Aether System Tests. Used for output logs and PCAPs.
+| `${TEST_TYPE}` - Directories for logs for each test type Ex: functional, failure, cleanup.
+| `${accContext}` - SD-Core context.
+| `${rocContext}` - ROC context.
+| `${aceContext}` - SD-Fabric and UPF context.
+| `${upfNamespace}` - UPF namespace used in tests. Ex: `aether-sdcore-upf1`.
+| `${pfcpNamespace}` - PFCP namespace used in tests. Ex: `tost`.
+| `${coreType}` - 4G or 5G.
+| `${upfType}` - UPF type used in tests. Ex: `bess`. Can also be `${None}` or empty.
+| `${upfTypeName}` - UPF type name used in tests.
+ This affects which input files are used. Ex: `p4`.
+ Can also be `${None}` or empty.
+| `${ng40Host}` - IP address for NG40 host.
+| `${ng40User}` - NG40 User.
+| `${ng40RanDir}` - NG40. RAN directory.
+| `${ng40CsvDir}` - NG40 CSV directory.
+| `${ng40TestDir}` - NG40 tests directory.
+| `${pcapDir}` - Saved PCAP directory.
+| `${cdrLogDir}` - NG40 CDR directory.
+| `${containerLogDir}` - Log directory for each deployed component.
+| `${pod}` - Pod name used in tests. This affects which input files are used. Ex: `qa`, `qa2`.
+
+Running Tests (5G)
+------------------
+5G tests are executed similar to 4G, except using the tests located in the 5G directory.
+
+.. code-block:: shell
+
+ cd ${WORKSPACE_DIR}
+ robot -d ${WORKSPACE_DIR}/${TEST_TYPE}/robotlogs \
+ --debugfile ${WORKSPACE_DIR}/functionality/robotlogs/${TEST_FILE}_debug \
+ -o ${log_xml} \
+ -l ${log_html} \
+ -r ${report_html} \
+ -e notready \
+ -v ACC_CONTEXT:${accContext} \
+ -v AMP_CONTEXT:${rocContext} \
+ -v ACE_CONTEXT:${aceContext} \
+ -v UPF_NAMESPACE:${upfNamespace} \
+ -v UPF_NAMESPACE2:${upfNamespace2} \
+ -v UPF_NAMESPACE3:${upfNamespace3} \
+ -v PFCP_NAMESPACE:${pfcpNamespace} \
+ -v CORE_TYPE:${coreType} \
+ -v UPF_TYPE:${upfType} \
+ -v UPF_TYPE_NAME:${upfTypeName}\
+ -v NG40_HOST:${ng40Host} \
+ -v NG40_USER:${ng40User} \
+ -v NG40RAN_DIR:${ng40RanDir} \
+ -v NG40TEST_DIR:${ng40TestDir} \
+ -v CSV_DIR:${ng40CsvDir} \
+ -v PCAP_DIR:${WORKSPACE_DIR}/functionality/${pcapDir} \
+ -v NG40_LOG_DIR:${WORKSPACE_DIR}/functionality/${ng40LogDir} \
+ -v CDR_LOG_DIR:${WORKSPACE_DIR}/functionality/${cdrLogDir} \
+ -v CONTAINER_LOG_DIR:${WORKSPACE_DIR}/functionality/${containerLogDir} \
+ -v POD_NAME:${pod} \
+ aether-system-tests/system-tests/tests/5G/functional/${TEST_FILE}.robot || true
+