Merge "[VOL-4597] Added Voltha Test Automation Documentation"
diff --git a/_static/Voltha_Test_Automation_Jenkins.png b/_static/Voltha_Test_Automation_Jenkins.png
new file mode 100644
index 0000000..68c7382
--- /dev/null
+++ b/_static/Voltha_Test_Automation_Jenkins.png
Binary files differ
diff --git a/index.rst b/index.rst
index 821b99d..5908500 100644
--- a/index.rst
+++ b/index.rst
@@ -168,6 +168,7 @@
VOLTHA system tests <voltha-system-tests/README.md>
testing/index.rst
testing/memory_usage_pprof.rst
+ testing/voltha_test_automation.rst
.. toctree::
:maxdepth: 1
diff --git a/testing/voltha_test_automation.rst b/testing/voltha_test_automation.rst
new file mode 100644
index 0000000..a95d37b
--- /dev/null
+++ b/testing/voltha_test_automation.rst
@@ -0,0 +1,106 @@
+Voltha Test Automation
+======================
+
+This page shows how the Voltha tests are automated to run on `OpenCORD Jenkins <https://jenkins.opencord.org/>`_.
+
+Tests of different workflows, i.e. ATT, DT and TT, with various configurations based on either BBSim deployment or
+actual Physical POD run on Jenkins periodically.
+
+.. figure:: ../_static/Voltha_Test_Automation_Jenkins.png
+ :alt: Voltha OpenCord Jenkins
+ :width: 70%
+ :align: center
+
+ Voltha OpenCord Jenkins
+
+Similar set of jobs are kept under similar tabs. Like,
+all the master build jobs are placed under `VOLTHA-2.X-Tests <https://jenkins.opencord.org/view/VOLTHA-2.X-Tests/>`_ tab,
+jobs related to the voltha-2.8 LTS release are in VOLTHA-2.8 tab.
+
+Likewise, tests that collect measurements on Voltha performance at scale are placed under `voltha-scale-measurements <https://jenkins.opencord.org/view/voltha-scale-measurements/>`_,
+and the Voltha Soak testing jobs are kept under the tab named `voltha-soak <https://jenkins.opencord.org/view/voltha-soak/>`_.
+
+Jenkins also host `verification jobs <https://jenkins.opencord.org/view/VOLTHA%202.x%20verify/>`_ that trigger automatically whenever a patch is pushed or updated on Gerrit for the review.
+
+
+Job Templates
+^^^^^^^^^^^^^
+
+Templates for the jobs are defined in `ci-management <https://github.com/opencord/ci-management>`__ repo.
+
+
+BBSim Jobs
+^^^^^^^^^^
+
+OpenCORD Jenkins host a large number of BBSim based regression jobs that run periodically
+with different deployment configurations to verify Voltha control plane functionality
+for different operator workflows. Such as,
+
+- `periodic-voltha-test-bbsim <https://jenkins.opencord.org/job/periodic-voltha-test-bbsim/>`_ & `periodic-voltha-multiple-olts-test-bbsim <https://jenkins.opencord.org/job/periodic-voltha-multiple-olts-test-bbsim/>`_ : tests Voltha functional scenarios, recovery and error scenarios, ONOS HA scenarios etc. for ATT
+- `periodic-voltha-multi-uni-test-bbsim <https://jenkins.opencord.org/job/periodic-voltha-multi-uni-test-bbsim/>`_ & `periodic-voltha-multi-uni-multiple-olts-test-bbsim <https://jenkins.opencord.org/job/periodic-voltha-multi-uni-multiple-olts-test-bbsim/>`_ : tests Voltha functionality with multiple UNIs enabled, and with single & multiple OLT (BBSim) deployments
+- `periodic-voltha-dt-test-bbsim <https://jenkins.opencord.org/job/periodic-voltha-dt-test-bbsim/>`_ : tests functional and failure scenarios for DT workflow
+- `periodic-voltha-unitag-subscriber-tt-test-bbsim <https://jenkins.opencord.org/job/periodic-voltha-unitag-subscriber-tt-test-bbsim/>`_ : tests Voltha functionality with UNI-Tag Subscriber provisioning enabled for TT workflow
+- `periodic-voltha-tt-maclearner-sanity-test-bbsim <https://jenkins.opencord.org/job/periodic-voltha-tt-maclearner-sanity-test-bbsim/>`_ : tests Voltha functionality with MAC Learner enabled for TT workflow
+- `periodic-software-upgrade-test-bbsim <https://jenkins.opencord.org/job/periodic-software-upgrade-test-bbsim/>`_ : tests ONOS Applications, Voltha Components and ONU Software Upgrades
+- `periodic-voltha-openonu-go-test-bbsim <https://jenkins.opencord.org/job/periodic-voltha-openonu-go-test-bbsim/>`_ & `periodic-voltha-multiple-olts-openonu-go-test-bbsim <https://jenkins.opencord.org/job/periodic-voltha-multiple-olts-openonu-go-test-bbsim/>`_ : tests state for ONU Go adapter with different operator workflows, different constellations and tech profiles
+- `periodic-voltha-pm-data-test-bbsim <https://jenkins.opencord.org/job/periodic-voltha-pm-data-test-bbsim/>`_ & `periodic-voltha-multiple-olts-pm-data-test-bbsim <https://jenkins.opencord.org/job/periodic-voltha-multiple-olts-pm-data-test-bbsim/>`_ : tests Open ONU Go adapter PM data
+- `periodic-voltha-etcd-test <https://jenkins.opencord.org/job/periodic-voltha-etcd-test/>`_ : tests Voltha etcd functionality
+- `periodic-voltha-test-DMI <https://jenkins.opencord.org/job/periodic-voltha-test-DMI/>`_ : tests device management interface functionality
+
+Similar to the above mentioned jobs which are based on **master**,
+Jenkins has regression jobs for other Voltha releases as well, like votlha-2.8 and voltha-2.9.
+
+The template for the jobs that run on BBSim based deployment is defined `here <https://github.com/opencord/ci-management/blob/master/jjb/voltha-e2e.yaml>`__.
+
+BBSim based deployment is also used to test Voltha at scale.
+The scale jobs run the tests that collect measurements on Voltha performance.
+
+Scale jobs do the deployment with various combinations
+like single or multiple voltha stacks with one or more BBSim based OLTs,
+multiple PONs and ONUs thus testing 4000+ subscribers for different workflows.
+More details about scale jobs can be found `here <https://jenkins.opencord.org/view/voltha-scale-measurements/>`__.
+
+
+Hardware (or Physical POD) Jobs
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+OpenCord Jenkins host various regression jobs that runs on the ONF physical pods
+situated at different geographical locations.
+Hardware jobs test both control and dataplane functionality of Voltha.
+
+These hardware jobs are mostly defined in a combination of a build job and a test job.
+The template for the jobs that run on physical pod is defined `here <https://github.com/opencord/ci-management/blob/master/jjb/voltha-test/voltha-nightly-jobs.yaml>`__.
+
+Some of the configurations that are currently being tested on ONF's Physical Pods are:
+
+- Default Technology Profile with master or LTS release build for ATT
+- 1T4GEM profile with master or LTS release build for ATT
+- 1T4GEM-unencrypted profile with master or LTS release build for ATT
+- 1T8GEM Technology Profile with master or LTS release build for DT
+- 1T8GEM Technology Profile with master or LTS release build for DT with GPON based OLT setup
+- 1T8GEM Technology Profile with master or LTS release build for DT with multiple OLTs setup
+- Different service based Technology Profiles with master or LTS release build for TT
+- Different service based profiles with master or LTS release build for TT with multiple UNIs enabled
+
+
+Per-Patchset Verification Jobs
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The verification jobs gets triggered automatically whenever a patch is pushed or updated on Gerrit, e.g.
+
+- verify_{project}_licensed : This job verifies that there is valid license/copyright on files.
+- verify_{project}_tag-collision : This job checks for changes to version files,
+ and that they don't duplicate tags already in the git repo.
+- verify_{project}_unit-test : This job runs the Makefile based unit tests.
+- verify_{project}_sanity-test : This job verifies that basic Voltha functionality is not broken
+ for any of the operator workflows.
+
+Apart from system triggered jobs, there are some verification jobs which can be triggered by the user
+by providing apt comment in the Gerrit patch, e.g.
+
+- hardware test functional tests : This comment triggers the run on a dedicated physical pod
+ and executes the entire functional test suite for ATT.
+- hardware test functional tests DT : This comment triggers the functional suite tests for DT.
+
+The templates for the above mentioned and other similar jobs can be found in the
+`ci-management <https://github.com/opencord/ci-management>`__ repo.
