Gitiles
Code Review Sign In
gerrit.opencord.org / voltha-system-tests / a1f7c5983b9a25e7e3600ded47c905667931dc4d
commita1f7c5983b9a25e7e3600ded47c905667931dc4d[log] [tgz]
authorSuchitra Vemuri <suchitra@opennetworking.org>Sat Sep 28 19:21:50 2019 -0700
committerSuchitra Vemuri <suchitra@opennetworking.org>Sat Sep 28 19:21:50 2019 -0700
tree49efe633aec3567b99afa7abb2af36b2cad5512d
parent65e5fbba0759cc1daadaae257a5972133b70c3e7 [diff]
update oltuplink port input

Change-Id: I1d836c3f8d79b39577ce03173011c8d82088653a
1 file changed
tree: 49efe633aec3567b99afa7abb2af36b2cad5512d
  1. .gitignore
  2. .gitreview
  3. Jenkinsfile-voltha-build
  4. Jenkinsfile-voltha-test
  5. Makefile
  6. README.md
  7. VERSION
  8. libraries/
  9. requirements.txt
  10. tests/
  11. variables/
README.md

VOLTHA System Tests

Automated test-suites to validate the stability/functionality of VOLTHA. Tests that reside in here should be written in Robot Framework and Python.

Intended use includes:

  • Functional testing
  • Integration and Acceptance testing
  • Sanity and Regression testing
  • Scale Testing using BBSIM
  • Failure scenario testing

Prerequisites

  • Python virtualenv

  • voltctl - a command line tool to access VOLTHA. Reference - voltctl

  • kubectl - a command line tool to access your Kubernetes Clusers. Reference

  • voltctl and kubectl must be properly configured on your system prior to any test executions. The kind-voltha environment will install and configure these tools for you; see below.

Directory is structured as follows:

├── tests
  └── sanity/           // basic tests that should always pass. Will be used as gating-patchsets
  └── functional/       // feature/functionality tests that should be implemented as new features get developed
└── libraries           // shared test keywords (functions) across various test suites
└── variables           // shared variables across various test suites

Setting up a test environment

An easy way to bring up VOLTHA + BBSim for testing is by using kind-voltha. To set up a minimal environment, first install Docker and the Go programming language. Then run the following commands:

git clone https://github.com/ciena/kind-voltha
cd kind-voltha
EXTRA_HELM_FLAGS="--set defaults.image_tag=voltha-2.1” TYPE=minimal WITH_RADIUS=y WITH_BBSIM=y INSTALL_ONOS_APPS=y CONFIG_SADIS=y ./voltha up
source minimal-env.sh

The defaults.image_tag value above is used to specify which VOLTHA branch images to pull from Docker Hub.

Running the sanity tests

Assuming that you have brought up VOLTHA as described above, to run the the sanity tests:

git clone https://github.com/opencord/voltha-system-tests
make -C voltha-system-tests sanity-kind

This test execution will generate three report files in voltha-system-tests/tests/sanity (output.xml, report.html, log.html). View the report.html page in a browser to analyze the results.

Test variables

The make sanity-kind target is equivalent to the following:

ROBOT_PORT_ARGS="-v ONOS_REST_PORT:8181 -v ONOS_SSH_PORT:8101" \
ROBOT_TEST_ARGS="--exclude notready --critical sanity" \
ROBOT_MISC_ARGS="-v num_onus:1" \
make sanity

If you are running the tests in another environment, you can run make sanity with the arguments appropriate for your environment. Look at variables.robot for a list of variables that you may need to override.

Adding to the tests

Most additions should be done by adding keywords to the libraries, then calling these keywords from the tests. Consult a guide on how to write good Robot framework tests.

Make sure that make lint passes, which runs robotframework-lint on any new code that is created.

WIP:

  • Containerizing test environment so these tests can be run independent of the system.