Fix bullet points in sd-core testing doc
Change-Id: Ice4c84e6005257f1694287817f5b9ab6bf287b55
diff --git a/testing/sdcore_testing.rst b/testing/sdcore_testing.rst
index 52e4578..535a1ad 100644
--- a/testing/sdcore_testing.rst
+++ b/testing/sdcore_testing.rst
@@ -52,18 +52,16 @@
The test cases are atomic testing units and can be combined to build test
suites. The following test suites have been built so far:
-* ``functionality test suite`` verifies basic functionality of the
- mobile core. It runs test case #1 to #8 including ``4G_SCALE`` which attaches
- 5 UEs with 1/s attach rate
-
-* ``scalability test suite`` tests the system by scale and verifies
- system stability. It runs ``4G_SCALE`` which attaches a large number of UEs
- with high attach rate (16k UEs with 100/s rate on dev pod, and 10k UEs with
- 10/s rate on staging pod)
-
-* ``performance test suite`` measures performance of the control and
- data plane. It runs ``4G_SCALE`` multiple times with different attach rates
- to understand how the system performs under different loads.
+1. ``functionality test suite`` verifies basic functionality of the
+ mobile core. It runs test case #1 to #8 including ``4G_SCALE`` which attaches
+ 5 UEs with 1/s attach rate
+2. ``scalability test suite`` tests the system by scale and verifies
+ system stability. It runs ``4G_SCALE`` which attaches a large number of UEs
+ with high attach rate (16k UEs with 100/s rate on dev pod, and 10k UEs with
+ 10/s rate on staging pod)
+3. ``performance test suite`` measures performance of the control and
+ data plane. It runs ``4G_SCALE`` multiple times with different attach rates
+ to understand how the system performs under different loads.
Robot Framework
~~~~~~~~~~~~~~~
@@ -81,13 +79,13 @@
The following integration tests are implemented at the moment:
-* Subscriber Attach with HSS Restart
-* Subscriber Attach with MME Restart
-* Subscriber Attach with SPGWC Restart
-* Subscriber Attach with PFCP Agent Restart
+1. Subscriber Attach with HSS Restart
+2. Subscriber Attach with MME Restart
+3. Subscriber Attach with SPGWC Restart
+4. Subscriber Attach with PFCP Agent Restart
.. Note::
- More integration tests are being developed as part of Robot Framework
+ More integration tests are being developed as part of Robot Framework
Test Schedules
--------------
@@ -101,59 +99,52 @@
SD-Core nightly tests are a set of jobs managed by Aether Jenkins.
All four test suites we mentioned above are scheduled to run nightly.
-* ``functionality job (func)`` runs NG40 test cases included in the
- functionality suite and verifies all tests pass.
-
-* ``scalability job (scale)`` runs the scalability test suite and reports
- the number of successful/failed attaches, detaches and pings.
-
-* ``performance job (perf)`` runs the performance test suite and reports
- SCTP heartbeat RTT, GTP ICMP RTT and call setup latency numbers.
+1. ``functionality job (func)`` runs NG40 test cases included in the
+ functionality suite and verifies all tests pass.
+2. ``scalability job (scale)`` runs the scalability test suite and reports
+ the number of successful/failed attaches, detaches and pings.
+3. ``performance job (perf)`` runs the performance test suite and reports
+ SCTP heartbeat RTT, GTP ICMP RTT and call setup latency numbers.
And all these jobs can be scheduled on any of the Aether PODs including
``dev`` pod, ``staging`` pod and ``qa`` pod. By combining the test type and
test pod the following Jenkins jobs are generated:
-* ``dev`` pod: `func_dev`, `scale_dev`, `perf_dev`, `integ_dev`
-* ``staging`` pod: `func_staging`, `scale_staging`, `perf_staging`, `integ_staging`
-* ``qa`` pod: `func_qa`, `scale_qa`, `perf_qa`, `integ_qa`
+1. ``dev`` pod: `func_dev`, `scale_dev`, `perf_dev`, `integ_dev`
+2. ``staging`` pod: `func_staging`, `scale_staging`, `perf_staging`, `integ_staging`
+3. ``qa`` pod: `func_qa`, `scale_qa`, `perf_qa`, `integ_qa`
Job structure
^^^^^^^^^^^^^
Take `scale_dev` job as an example. It runs the following downstream jobs:
-* `omec_deploy_dev`: this job re-deploys the dev pod with latest OMEC images.
+1. `omec_deploy_dev`: this job re-deploys the dev pod with latest OMEC images.
.. Note::
- only the dev pod job triggers a deployment downstream job. No
- re-deployment is performed on the staging and qa pod before the tests
+ only the dev pod job triggers a deployment downstream job. No
+ re-deployment is performed on the staging and qa pod before the tests
-* `ng40-test_dev`: this job executes the scalability test suite.
-
-* `archive-artifacts_dev`: this job collects and uploads k8s and container logs.
-
-* `post-results_dev`: this job collects the NG40 test logs/pcaps and pushes the
- test data to database. It also generates plots using Rscript for func and
- scale tests
+2. `ng40-test_dev`: this job executes the scalability test suite.
+3. `archive-artifacts_dev`: this job collects and uploads k8s and container logs.
+4. `post-results_dev`: this job collects the NG40 test logs/pcaps and pushes the
+ test data to database. It also generates plots using Rscript for func and
+ scale tests
The integration tests are written using Robot Framework so have a slightly
different Jenkins Job structure. Take `integ_dev` as an example. It runs the
following downstream jobs:
-* `omec_deploy_dev`: this job executes the scalability test suite.
-
-* `robotframework-test_dev`: this job is similar to `ng40-test_dev` with the
- exception that instead of directly executing NG40 commands it calls robot
- framework to exectue the test cases and publishes the test results using
- `RobotPublisher` Jenkins plugin. The robot results will also be copied to
- the upstream job and published there.
-
-* `archive-artifacts_dev`: this job collects and uploads k8s and container logs.
-
-* `post-results_dev`: this job collects the NG40 test logs/pcaps and pushes the
- test data to database. It also generates plots using Rscript for func and
- scale tests
+1. `omec_deploy_dev`: this job executes the scalability test suite.
+2. `robotframework-test_dev`: this job is similar to `ng40-test_dev` with the
+ exception that instead of directly executing NG40 commands it calls robot
+ framework to exectue the test cases and publishes the test results using
+ `RobotPublisher` Jenkins plugin. The robot results will also be copied to
+ the upstream job and published there.
+3. `archive-artifacts_dev`: this job collects and uploads k8s and container logs.
+4. `post-results_dev`: this job collects the NG40 test logs/pcaps and pushes the
+ test data to database. It also generates plots using Rscript for func and
+ scale tests
Patchset Tests
~~~~~~~~~~~~~~
@@ -165,9 +156,9 @@
``Nucleus``, ``upf-epc`` and ``spgw`` (private). OMEC CI includes the following
verifications:
-* ONF CLA verification
-* License verifications (FOSSA/Reuse)
-* NG40 tests
+1. ONF CLA verification
+2. License verifications (FOSSA/Reuse)
+3. NG40 tests
These verifications are automatically triggered by submitted or updated PR to
the repos above. They can also be triggered manually by commenting ``retest
@@ -177,23 +168,17 @@
The NG40 verifications are a set of jobs running on both opencord Jenkins and
Aether Jenkins (private). The jobs run on opencord Jenkins include
-* `omec_c3po_container_remote <https://jenkins.opencord.org/job/omec_c3po_container_remote/>`_ (public)
-
-* `omec_Nucleus_container_remote <https://jenkins.opencord.org/job/omec_Nucleus_container_remote/>`_ (public)
-
-* `omec_upf-epc_container_remote <https://jenkins.opencord.org/job/omec_upf-epc_container_remote/>`_ (public)
-
-* `omec_spgw_container_remote` (private, under member-only folder)
+1. `omec_c3po_container_remote <https://jenkins.opencord.org/job/omec_c3po_container_remote/>`_ (public)
+2. `omec_Nucleus_container_remote <https://jenkins.opencord.org/job/omec_Nucleus_container_remote/>`_ (public)
+3. `omec_upf-epc_container_remote <https://jenkins.opencord.org/job/omec_upf-epc_container_remote/>`_ (public)
+4. `omec_spgw_container_remote` (private, under member-only folder)
And the jobs run on Aether Jenkins include
-* `c3po_premerge_dev`
-
-* `Nucleus_premerge_dev`
-
-* `upf-epc_premerge_dev`
-
-* `spgw_premerge_dev`
+1. `c3po_premerge_dev`
+2. `Nucleus_premerge_dev`
+3. `upf-epc_premerge_dev`
+4. `spgw_premerge_dev`
Job structure
^^^^^^^^^^^^^
@@ -205,15 +190,12 @@
The private c3po job runs the following downstream jobs sequentially:
-* `docker-publish-github_c3po`: this job downloads the c3po PR, runs docker
- build and publishes the c3po docker images to `Aether registry`.
-
-* `omec_deploy_dev`: this job deploys the images built from previous job onto
- the omec dev pod.
-
-* `ng40-test_dev`: this job executes the functionality test suite.
-
-* `archive-artifacts_dev`: this job collects and uploads k8s and container logs.
+1. `docker-publish-github_c3po`: this job downloads the c3po PR, runs docker
+ build and publishes the c3po docker images to `Aether registry`.
+2. `omec_deploy_dev`: this job deploys the images built from previous job onto
+ the omec dev pod.
+3. `ng40-test_dev`: this job executes the functionality test suite.
+4. `archive-artifacts_dev`: this job collects and uploads k8s and container logs.
After all the downstream jobs are finished, the upstream job (`c3po_premerge_dev`)
copies artifacts including k8s/container/NG40 logs and pcap files from
@@ -231,25 +213,22 @@
The following jobs are triggered as post-merge jobs when PRs are merged to
OMEC repos:
-* `docker-publish-github-merge_c3po`
-
-* `docker-publish-github-merge_Nucleus`
-
-* `docker-publish-github-merge_upf-epc`
-
-* `docker-publish-github-merge_spgw`
+1. `docker-publish-github-merge_c3po`
+2. `docker-publish-github-merge_Nucleus`
+3. `docker-publish-github-merge_upf-epc`
+4. `docker-publish-github-merge_spgw`
Again take the c3po job as an example. The post-merge job (`docker-publish-github-merge_c3po`)
runs the following downstream jobs sequentially:
-* `docker-publish-github_c3po`: this is the same job as the one in pre-merge
- section. It checks out the latest c3po code, runs docker build and
- publishes the c3po docker images to `docker hub <https://hub.docker.com/u/omecproject>`__.
+1. `docker-publish-github_c3po`: this is the same job as the one in pre-merge
+ section. It checks out the latest c3po code, runs docker build and
+ publishes the c3po docker images to `docker hub <https://hub.docker.com/u/omecproject>`__.
.. Note::
- the spgw images are published to Aether registry instead of docker hub
+ the spgw images are published to Aether registry instead of docker hub
-* `c3po_postrelease`: this job submits a patchset to aether-pod-configs repo
- for updating the CD pipeline with images published in the job above.
+2. `c3po_postrelease`: this job submits a patchset to aether-pod-configs repo
+ for updating the CD pipeline with images published in the job above.
Post-merge jobs for other OMEC repos share the same structure.