| Jon Hall | 7bd8d0f | 2021-04-12 14:32:16 -0700 | [diff] [blame] | 1 | .. |
| 2 | SPDX-FileCopyrightText: © 2021 Open Networking Foundation <support@opennetworking.org> |
| 3 | SPDX-License-Identifier: Apache-2.0 |
| 4 | |
| 5 | Fabric Testing |
| 6 | ============== |
| 7 | |
| 8 | |
| 9 | Test Framework |
| 10 | -------------- |
| 11 | |
| 12 | We use `TestON`_ to connect to and manipulate different test components using CLI or REST |
| 13 | API. These tests use a common set of library functions for Segment Routing testing to |
| 14 | make writing new tests easier. For Aether, we are porting our existing set of Segment |
| 15 | Routing functionality tests written for Mininet to using Stratum hardware switches. |
| 16 | |
| 17 | We use Jenkins to schedule and trigger our tests which run against either the QA Pod or |
| 18 | the Staging Pod. |
| 19 | |
| 20 | Resources: |
| 21 | ^^^^^^^^^^ |
| 22 | |
| 23 | #. `ONOS System Test Guide`_ |
| 24 | #. `Segment Routing Funcationality Test Plan`_ |
| 25 | |
| 26 | Integration Testing |
| 27 | ------------------- |
| 28 | |
| 29 | We are in the process of moving existing functionality tests to written for OVS or BMv2 |
| 30 | software switches running in Mininet to running on hardware switches, starting with |
| 31 | bridging and routing functionality. These tests will run nightly on the QA Pod. |
| 32 | |
| 33 | Failure/Recovery Performance Testing |
| 34 | ------------------------------------ |
| 35 | |
| 36 | We run several experiments every night on the Staging Pod and measure the performance of |
| 37 | Aether during the experiment. |
| 38 | The test job can be found in Aether Jenkins under `tost-teston-staging-nightly`. |
| 39 | |
| 40 | #. Port Down/Up |
| 41 | |
| 42 | * **Summary**: Uses onos CLI `portstate` command to send gNMI request to disable/enable |
| 43 | a port on a leaf leading to a spine. There are two versions of this test, one on the |
| 44 | access leaf and one on the upstream leaf. |
| 45 | |
| 46 | * **Test code**: |
| 47 | |
| 48 | * `Access Leaf Port Failure/Recovery Test`_ |
| 49 | |
| 50 | * `Upstream Leaf Port Failure/Recovery Test`_ |
| 51 | |
| 52 | #. Stratum Agent Stop/Start |
| 53 | |
| 54 | * **Summary**: Uses Kubernetes to delete/restart the stratum pod on a spine |
| 55 | |
| 56 | * **Test code**: |
| 57 | |
| 58 | * `Stratum Restart Test`_ |
| 59 | |
| 60 | #. ONL Shutdown/Startup |
| 61 | |
| 62 | * **Summary**: Restart ONL on a spine |
| 63 | |
| 64 | * **Test code**: |
| 65 | |
| 66 | * `ONL Reboot Test`_ |
| 67 | |
| 68 | #. ONOS Node Reboot |
| 69 | |
| 70 | * **Summary**: Restart each ONOS node in the cluster one by one |
| 71 | |
| 72 | * **Test code**: To be implemented |
| 73 | |
| 74 | * `Rolling ONOS Restart Test`_ |
| 75 | |
| 76 | .. _TestON: https://github.com/opennetworkinglab/OnosSystemTest |
| 77 | .. _ONOS System Test Guide: https://wiki.onosproject.org/display/ONOS/System+Testing+Guide |
| 78 | .. _Segment Routing Funcationality Test Plan: https://wiki.opencord.org/display/CORD/Test+Plan+-+Fabric+Control |
| 79 | .. _Access Leaf Port Failure/Recovery Test: https://github.com/opennetworkinglab/OnosSystemTest/tree/master/TestON/tests/USECASE/SegmentRouting/SRStaging/SReNBLeafSpinePortstateFailure |
| 80 | .. _Upstream Leaf Port Failure/Recovery Test: https://github.com/opennetworkinglab/OnosSystemTest/tree/master/TestON/tests/USECASE/SegmentRouting/SRStaging/SRupstreamLeafSpinePortstateFailure |
| 81 | .. _Stratum Restart Test: https://github.com/opennetworkinglab/OnosSystemTest/tree/master/TestON/tests/USECASE/SegmentRouting/SRStaging/SRstratumRestart |
| 82 | .. _ONL Reboot Test: https://github.com/opennetworkinglab/OnosSystemTest/tree/master/TestON/tests/USECASE/SegmentRouting/SRStaging/SRONLReboot |
| 83 | .. _Rolling ONOS Restart Test: https://github.com/opennetworkinglab/OnosSystemTest/tree/master/TestON/tests/USECASE/SegmentRouting/SRStaging/SRrollingRestart |