[VOL-2386]
Creating a base structure for the docs
Inserting basic content
Change-Id: Ie3c3417e95065021aeed79985f7d6262418b08f8
diff --git a/.gitignore b/.gitignore
index 60b433c..c19dd4e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -11,3 +11,6 @@
voltha-openonu-adapter
voltha-protos
voltha-system-tests
+
+# IDEs
+.idea
\ No newline at end of file
diff --git a/.rstcheck.cfg b/.rstcheck.cfg
new file mode 100644
index 0000000..c758ac4
--- /dev/null
+++ b/.rstcheck.cfg
@@ -0,0 +1,3 @@
+[rstcheck]
+ignore_directives=code
+report=warning
\ No newline at end of file
diff --git a/index.rst b/index.rst
index 7dcc158..bd1a3da 100644
--- a/index.rst
+++ b/index.rst
@@ -3,11 +3,22 @@
.. image:: images/VOLTHA_LOGO.png
-This documents the VOLTHA project
+Welcome to the VOLTHA project documentation.
.. toctree::
:maxdepth: 1
- :caption: VOLTHA Core
+ :caption: Overview
+
+ overview/deploy-physical.rst
+ overview/deploy-virtual.rst
+ overview/development_loop.rst
+ overview/troubleshooting.rst
+ overview/known_issues.rst
+ overview/contribute.rst
+
+.. toctree::
+ :maxdepth: 1
+ :caption: VOLTHA Core (outdated)
voltha-go/README.md
voltha-go/quickstart.md
@@ -18,13 +29,13 @@
.. toctree::
:maxdepth: 1
- :caption: Adapters
+ :caption: Adapters (outdated)
voltha-openolt-adapter/README.md
.. toctree::
:maxdepth: 1
- :caption: VOLTHA-protos
+ :caption: VOLTHA-protos (outdated)
voltha-protos/README.md
@@ -48,4 +59,3 @@
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
-
diff --git a/overview/contribute.rst b/overview/contribute.rst
new file mode 100644
index 0000000..0d4c382
--- /dev/null
+++ b/overview/contribute.rst
@@ -0,0 +1,21 @@
+How to contribute
+=================
+
+Hello and welcome!
+
+If you want to contribute to VOLTHA, please don't hesitate to contact us!
+The best ways to do so are:
+
+- `Slack <https://slackin.opencord.org>`_
+- `Mailing List <https://groups.google.com/a/opencord.org/forum/#!forum/voltha-discuss>`_
+
+Ongoing Brigades
+----------------
+
+- `BBSim and Scale <https://jira.opencord.org/secure/Dashboard.jspa?selectPageId=10902>`_
+- `Stabilization <https://jira.opencord.org/secure/Dashboard.jspa?selectPageId=10801>`_
+- `BAL 3.x <https://jira.opencord.org/secure/Dashboard.jspa?selectPageId=10700>`_
+- `Multicast and Multi-TCONT <https://jira.opencord.org/secure/Dashboard.jspa?selectPageId=10800>`_
+- `POD Management <https://jira.opencord.org/secure/Dashboard.jspa?selectPageId=10901>`_
+- `ONOS FCAPS <https://jira.opencord.org/secure/Dashboard.jspa?selectPageId=10903>`_
+- `Test Automation <https://jira.opencord.org/secure/Dashboard.jspa?selectPageId=11400>`_
diff --git a/overview/deploy-physical.rst b/overview/deploy-physical.rst
new file mode 100644
index 0000000..510c9ae
--- /dev/null
+++ b/overview/deploy-physical.rst
@@ -0,0 +1,282 @@
+.. _deploy_physical:
+
+Deploy a physical VOLTHA POD
+============================
+
+Quickstart
+----------
+
+The quickstart assumes you POD is already correctly cabled, if not you can refer to :ref:`Physical_Layout`
+
+Requires:
+
+- a physical kubernetes cluster
+- `kind-voltha <https://github.com/ciena/kind-voltha>`_ cloned on your machine
+
+
+Enter the ``kind-voltha`` directory and execute this command (``kubectl`` must be installed and poiting to your cluster):
+
+.. code:: bash
+
+ DEPLOY_K8S=no WITH_RADIUS=y CONFIG_SADIS=y ./voltha up
+
+If you already have a ``radius`` server that you want to use, change the flag to ``WITH_RADIUS=n``
+and `configure ONOS accordingly <https://github.com/opencord/aaa>`_.
+
+TLDR;
+-----
+
+Below are the complete steps to install a physical cluster. It assumes ``kubectl`` and ``helm`` commands are already
+available.
+
+Configure Helm
+--------------
+
+Helm provide a capabilty to install and manage Kubernetes applications.
+VOLTHA’s default deployment mechanism utilized Helm. Before Helm can be
+used to deploy VOLTHA it must be initialized and the repositories that
+container the artifacts required to deploy VOLTHA must be added to Helm.
+
+.. code:: bash
+
+ helm repo add incubator https://kubernetes-charts-incubator.storage.googleapis.com
+ helm repo add stable https://kubernetes-charts.storage.googleapis.com
+ helm repo add onf https://charts.opencord.org
+ helm repo update
+
+.. _installation_steps:
+
+Install EtcdOperator
+--------------------
+
+ETCD Operator is a utility that allows applications to create and manage
+ETCD key/value clusters as Kubernetes resources. VOLTHA utilizes this
+utility to create its key/value store. *NOTE: it is not required that
+VOLTHA create its own datastore as VOLTHA can utilize and existing
+datastore, but for this example VOLTHA will creates its own datastore*
+
+.. code:: bash
+
+ helm install -f $TYPE-values.yaml --namespace voltha --name etcd-operator stable/etcd-operator
+
+Wait for operator pods
+~~~~~~~~~~~~~~~~~~~~~~
+
+Before continuing the Kubernetes pods associated with ETCD Operator must
+be in the ``Running`` state.
+
+.. code:: bash
+
+ kubectl get -n voltha pod
+
+Once all the pods are in the ``Running`` state the output, for a
+**full** deployment should be similar to the output below. For a
+**minimal** deployment there will only be a single pod, the
+``etcd-operator-etcd-operator-etcd-operator`` pod.
+
+.. code:: bash
+
+ NAME READY STATUS RESTARTS AGE
+ etcd-operator-etcd-operator-etcd-backup-operator-7897665cfq75w2 1/1 Running 0 2m
+ etcd-operator-etcd-operator-etcd-operator-7d579799f7-bjdnj 1/1 Running 0 2m
+ etcd-operator-etcd-operator-etcd-restore-operator-7d77d878wwcn7 1/1 Running 0 2m
+
+It is not just VOLTHA
+---------------------
+
+To demonstrate the capability of VOLTHA other *partner* applications are
+required, such as ONOS. The followins sections describe how to install
+and configure these *partner* applications.
+
+*NOTE: It is important to start ONOS before VOLTHA as if they are
+started in the reverse order ofagent sometimes does not connect to the
+SDN
+controller*\ `VOL-1764 <https://jira.opencord.org/browse/VOL-1764>`__.
+
+ONOS (OpenFlow Controller)
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+VOLTHA exposes an OLT and its connected ONUs as an OpenFlow switch. To
+control that virtual OpenFlow switch an OpenFlow controller is required.
+For most VOLTHA deployments that controller is ONOS with a set of ONOS
+applications installed. To install ONOS use the following Helm command:
+
+.. code:: bash
+
+ helm install -f $TYPE-values.yaml --name onos onf/onos
+
+Exposing ONOS Services
+^^^^^^^^^^^^^^^^^^^^^^
+
+.. code:: bash
+
+ screen -dmS onos-ui kubectl port-forward service/onos-ui 8181:8181
+ screen -dmS onos-ssh kubectl port-forward service/onos-ssh 8101:8101
+
+Configuring ONOS Applications
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Configuration files have been provided to configure aspects of the ONOS
+deployment. The following curl commands push those configurations to the
+ONOS instance. It is possible (likely) that ONOS won’t be immediately
+ready to accept REST requests, so the first ``curl`` command may need
+retried until ONOS is ready to accept REST connections.
+
+.. code:: bash
+
+ curl --fail -sSL --user karaf:karaf \
+ -X POST -H Content-Type:application/json \
+ http://127.0.0.1:8181/onos/v1/network/configuration/apps/org.opencord.kafka \
+ --data @onos-files/onos-kafka.json
+ curl --fail -sSL --user karaf:karaf \
+ -X POST -H Content-Type:application/json \
+ http://127.0.0.1:8181/onos/v1/network/configuration/apps/org.opencord.dhcpl2relay \
+ --data @onos-files/onos-dhcpl2relay.json
+ curl --fail -sSL --user karaf:karaf \
+ -X POST -H Content-Type:application/json \
+ http://127.0.0.1:8181/onos/v1/configuration/org.opencord.olt.impl.Olt \
+ --data @onos-files/olt-onos-olt-settings.json
+ curl --fail -sSL --user karaf:karaf \
+ -X POST -H Content-Type:application/json \
+ http://127.0.0.1:8181/onos/v1/configuration/org.onosproject.net.flow.impl.FlowRuleManager \
+ --data @onos-files/olt-onos-enableExtraneousRules.json
+
+SADIS Configuration
+^^^^^^^^^^^^^^^^^^^
+
+The ONOS applications leverage the *Subscriber and Device Information
+Store (SADIS)* when processing EAPOL and DHCP packets from VOLTHA
+controlled devices. In order for VOLTHA to function propperly SADIS
+entries must be configured into ONOS.
+
+The repository contains two example SADIS configuration that can be used
+with ONOS depending if you using VOLTHA with *tech profile* support
+(``onos-files/onos-sadis-no-tp.json``) or without *tech profile* support
+(``onos-files/onos-sadis-tp.json``). Either of these configurations can
+be pushed to ONOS using the following command:
+
+.. code:: bash
+
+ curl --fail -sSL --user karaf:karaf \
+ -X POST -H Content-Type:application/json \
+ http://127.0.0.1:8181/onos/v1/network/configuration/apps/org.opencord.sadis \
+ --data @<selected SADIS configuration file>
+
+Install VOLTHA Core
+-------------------
+
+VOLTHA has two main *parts*: core and adapters. The **core** provides
+the main logic for the VOLTHA application and the **adapters** contain
+logic to adapter vendor neutral operations to vendor specific devices.
+
+Before any adapters can be deployed the VOLTHA core must be installed
+and in the ``Running`` state. The following Helm command installs the
+core components of VOLTHA based on the desired deployment type.
+
+.. code:: bash
+
+ helm install -f $TYPE-values.yaml --set use_go=true --set defaults.log_level=WARN \
+ --namespace voltha --name voltha onf/voltha
+
+During the install of the core VOLTHA components some containers may
+“crash” or restart. This is normal as there are dependencies, such as
+the read/write cores cannot start until the ETCD cluster is established
+and so they crash until the ETCD cluster is operational. Eventually all
+the containers should be in a ``Running`` state as queried by the
+command:
+
+.. code:: bash
+
+ kubectl get -n voltha pod
+
+The output should be similar to the following with a different number of
+``etcd-operator`` and ``voltha-etcd-cluster`` pods depending on the
+deployment type.
+
+.. code:: bash
+
+ NAME READY STATUS RESTARTS AGE
+ etcd-operator-etcd-operator-etcd-operator-7d579799f7-xq6f2 1/1 Running 0 19m
+ ofagent-8ccb7f5fb-hwgfn 1/1 Running 0 4m
+ ro-core-564f5cdcc7-2pch8 1/1 Running 0 4m
+ rw-core1-7fbb878cdd-6npvr 1/1 Running 2 4m
+ rw-core2-7fbb878cdd-k7w9j 1/1 Running 3 4m
+ voltha-api-server-5f7c8b5b77-k6mrg 2/2 Running 0 4m
+ voltha-cli-server-5df4c95b7f-kcpdl 1/1 Running 0 4m
+ voltha-etcd-cluster-4rsqcvpwr4 1/1 Running 0 4m
+ voltha-kafka-0 1/1 Running 0 4m
+ voltha-zookeeper-0 1/1 Running 0 4m
+
+Install Adapters
+----------------
+
+The following commands install both the simulated OLT and ONU adapters
+as well as the adapters for an OpenOLT and OpenONU device.
+
+.. code:: bash
+
+ helm install -f $TYPE-values.yaml -set use_go=true --set defaults.log_level=WARN \
+ --namespace voltha --name sim onf/voltha-adapter-simulated
+ helm install -f $TYPE-values.yaml -set use_go=true --set defaults.log_level=WARN \
+ --namespace voltha --name open-olt onf/voltha-adapter-openolt
+ helm install -f $TYPE-values.yaml -set use_go=true --set defaults.log_level=WARN \
+ --namespace voltha --name open-onu onf/voltha-adapter-openonu
+
+Exposing VOLTHA Services
+------------------------
+
+At this point VOLTHA is deployed and from within the Kubernetes cluster
+the VOLTHA services can be reached. However, from outside the Kubernetes
+cluster the services cannot be reached.
+
+.. code:: bash
+
+ screen -dmS voltha-api kubectl port-forward -n voltha service/voltha-api 55555:55555
+ screen -dmS voltha-ssh kubectl port-forward -n voltha service/voltha-cli 5022:5022
+
+Install FreeRADIUS Service
+--------------------------
+
+.. code:: bash
+
+ helm install -f minimal-values.yaml --namespace voltha --name radius onf/freeradius
+
+Configure ``voltctl`` to Connect to VOLTHA
+------------------------------------------
+
+In order for ``voltctl`` to connect to the VOLTHA instance deplpoyed in
+the Kubernetes cluster it must know which IP address and port to use.
+This configuration can be persisted to a local config file using the
+following commands.
+
+.. code:: bash
+
+ mkdir -p $HOME/.volt
+ voltctl -a v2 -s localhost:55555 config > $HOME/.volt/config
+
+To test the connectivity you can query the version of the VOLTHA client
+and server.
+
+.. code:: bash
+
+ voltctl version
+
+The output should be similar to the following
+
+.. code:: bash
+
+ Client:
+ Version unknown-version
+ Go version: unknown-goversion
+ Vcs reference: unknown-vcsref
+ Vcs dirty: unknown-vcsdirty
+ Built: unknown-buildtime
+ OS/Arch: unknown-os/unknown-arch
+
+ Cluster:
+ Version 2.1.0-dev
+ Go version: 1.12.6
+ Vcs feference: 28f120f1f4751284cadccf73f2f559ce838dd0a5
+ Vcs dirty: false
+ Built: 2019-06-26T16:58:22Z
+ OS/Arch: linux/amd64
\ No newline at end of file
diff --git a/overview/deploy-virtual.rst b/overview/deploy-virtual.rst
new file mode 100644
index 0000000..dfec06d
--- /dev/null
+++ b/overview/deploy-virtual.rst
@@ -0,0 +1,85 @@
+Deploy a virtual VOLTHA POD
+===========================
+
+A guide to install a virtual POD. This is generally used to gain familiarity with the
+environment or to do development.
+
+Quickstart
+----------
+
+Requires:
+
+- ``docker`` and ``go`` installed on your machine
+- `kind-voltha <https://github.com/ciena/kind-voltha>`_ cloned on your machine
+
+.. code:: bash
+
+ TYPE=minimal WITH_RADIUS=y CONFIG_SADIS=y ONLY_ONE=y WITH_BBSIM=y ./voltha up
+
+TLDR;
+-----
+
+Below are the complete steps to install a physical cluster.
+Most of the steps are the same as :ref:`deploy_physical`.
+
+Create Kubernetes Cluster
+-------------------------
+
+Kind provides a command line control tool to easily create Kubernetes
+clusters using just a basic Docker envionrment. The following commands
+will create the desired deployment of Kubernetes and then configure your
+local copy of ``kubectl`` to connect to this cluster.
+
+.. code:: bash
+
+ kind create cluster --name=voltha-$TYPE --config $TYPE-cluster.cfg
+ export KUBECONFIG="$(kind get kubeconfig-path --name="voltha-$TYPE")"
+ kubectl cluster-info
+
+Initialize Helm
+---------------
+
+Helm provide a capabilty to install and manage Kubernetes applications.
+VOLTHA’s default deployment mechanism utilized Helm. Before Helm can be
+used to deploy VOLTHA it must be initialized and the repositories that
+container the artifacts required to deploy VOLTHA must be added to Helm.
+
+.. code:: bash
+
+ # Initialize Helm and add the required chart repositories
+ helm init
+ helm repo add incubator https://kubernetes-charts-incubator.storage.googleapis.com
+ helm repo add stable https://kubernetes-charts.storage.googleapis.com
+ helm repo add onf https://charts.opencord.org
+ helm repo update
+
+ # Create and k8s service account so that Helm can create pods
+ kubectl create serviceaccount --namespace kube-system tiller
+ kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
+ kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
+
+From this point follow the :ref:`physical VOLTHA POD installation instructions <installation_steps>`. Come back here once done.
+
+Install BBSIM (Broad Band OLT/ONU Simulator)
+--------------------------------------------
+
+BBSIM provides a simulation of a BB device. It can be useful for
+testing.
+
+.. code:: bash
+
+ helm install -f minimal-values.yaml --namespace voltha --name bbsim onf/bbsim
+
+Create BBSIM Device
+^^^^^^^^^^^^^^^^^^^
+
+.. code:: bash
+
+ voltctl device create -t openolt -H $(kubectl get -n voltha service/bbsim -o go-template='{{.spec.clusterIP}}'):50060
+
+Enable BBSIM Device
+^^^^^^^^^^^^^^^^^^^
+
+.. code:: bash
+
+ voltctl device enable $(voltctl device list --filter Type~openolt -q)
\ No newline at end of file
diff --git a/overview/development_loop.rst b/overview/development_loop.rst
new file mode 100644
index 0000000..581a1e7
--- /dev/null
+++ b/overview/development_loop.rst
@@ -0,0 +1,53 @@
+Development Loop
+================
+
+Start with `kind-voltha <https://github.com/ciena/kind-voltha>`_. installed.
+
+Make changes
+------------
+
+We assume you already have downloaded the repository you want to modify and your IDE is correctly set up.
+
+In this tutorial we are going to use ``voltha-go`` as an example.
+
+Make the required changes in the ``voltha-go`` repository (the process
+is the same for all the VOLTHA repositories) to the code and build the
+docker images and push them on your private dockerhub account:
+
+.. code:: bash
+
+ $ DOCKER_REGISTRY="matteoscandolo/" DOCKER_TAG="dev" make docker-build
+
+Then push them to your docker hub account:
+
+.. code:: bash
+
+ $ DOCKER_REGISTRY="matteoscandolo/" DOCKER_TAG="dev" make docker-push
+
+Deploy your changes on kind-voltha
+----------------------------------
+
+Create a copy of the `minimal-values.yaml` file:
+
+.. code:: bash
+
+ $ cp minimal-values.yaml dev-values.yaml
+
+And edit that file so that it contains the appropriate values for the images you want to try,
+for example uncomment and change these two lines (mind the indentation):
+
+.. code:: yaml
+
+ images:
+ ro_core:
+ repository: matteoscandolo/voltha-ro-core
+ tag: dev
+ rw_core:
+ repository: matteoscandolo/voltha-rw-core
+ tag: dev
+
+Then redeploy `kind-voltha` using that value file:
+
+.. code:: bash
+
+ $ DEPLOY_K8S=no ./voltha down && DEPLOY_K8S=no EXTRA_HELM_FLAGS="-f dev-values.yaml" ./voltha up
\ No newline at end of file
diff --git a/overview/known_issues.rst b/overview/known_issues.rst
new file mode 100644
index 0000000..ee67832
--- /dev/null
+++ b/overview/known_issues.rst
@@ -0,0 +1,4 @@
+Known Issues
+============
+
+Coming Soon
diff --git a/overview/physical_layout.rst b/overview/physical_layout.rst
new file mode 100644
index 0000000..978be8f
--- /dev/null
+++ b/overview/physical_layout.rst
@@ -0,0 +1,9 @@
+.. _Physical_Layout:
+
+Build a physical POD
+====================
+
+Coming Soon.
+
+For the time being you can refer to the `SEBA LAB Setup guide <https://guide.opencord.org/profiles/seba/lab-setup.html#configuring-a-lab-setup>`_,
+just remember that the ``AggSwitch`` is optional in a VOLTHA deployment.
\ No newline at end of file
diff --git a/overview/troubleshooting.rst b/overview/troubleshooting.rst
new file mode 100644
index 0000000..cb3b747
--- /dev/null
+++ b/overview/troubleshooting.rst
@@ -0,0 +1,84 @@
+Troubleshooting
+===============
+
+Here is a collection of useful commands and tools to troubleshoot VOLTHA.
+The project is still in a very early phase, so this section of the guide is more focus
+on how to collection information and packet dumps than it is on operations.
+
+Trace Packets across components
+-------------------------------
+
+Given that ``VOLTHA`` is a collection of microservices we strongly suggest the usage of a log aggregator like
+`stern <https://github.com/wercker/stern>`_. installed. You can then aggregate logs from multiple containers in a simple way,
+for example to track all the ``packet-ins`` and ``packet-outs``:
+
+.. code:: bash
+
+ $ stern -n voltha "voltha|adapter" | grep -E "packet-out|packet-in" > packets.trace
+
+Once you have captured the packets you need, you can see them in wireshark by transforming the logs with:
+
+.. code:: bash
+
+ sed -n 's/.*packet":"\(.*\)",.*$/\1/p' packets.trace | sed -e 's/.\{2\}/& /g' | sed -e 's/^/000000 /' > packets.hex
+
+And then in wireshark select ``File -> Import from Hex Dump``
+
+Or you can decode a single packet using this online tool: `https://hpd.gasmi.net <https://hpd.gasmi.net>`_
+
+To get all the EAPOL packets: ``cat packets.trace | grep 888e``
+
+or if you have `hl <https://github.com/mbornet-hl/hl>`__ installed you can highlight with:
+
+.. code:: bash
+
+ cat packets.trace | grep 888e | hl -m '.*packet-in.*' -g '.*packet-out.*'
+
+To get all the DHCP packets: ``cat packets.trace | grep 8100``
+
+or if you have `hl <https://github.com/mbornet-hl/hl>`__ installed you can highlight with:
+
+.. code:: bash
+
+ cat packets.trace | grep 8100 | hl -m '.*packet-in.*' -g '.*packet-out.*'
+
+OMCI Packet Dump
+----------------
+
+In order to see OMCI packets in Wireshark you'll need to install ``omci.lua`` and ``BinDecHex.lua`` dissectors
+`wiki.wireshark.org/Contrib <https://wiki.wireshark.org/Contrib#Protocol_Dissectors>`_.
+
+To install them on Ubuntu:
+
+.. code:: bash
+
+ mkdir -p $(HOME)/.local/lib/wireshark/plugins
+ cd $(HOME)/.local/lib/wireshark/plugins
+ wget "https://wiki.wireshark.org/Contrib?action=AttachFile&do=get&target=omci.lua"
+ wget "https://wiki.wireshark.org/Contrib?action=AttachFile&do=get&target=BinDecHex.lua"
+
+IMPORTANT - Apply `this <https://ask.wireshark.org/question/4557/bindechexlua-error-bad-argument-to-module-packageseeall/?answer=4573#post-id-4573>`_ fix to BinDecHex.lua.
+
+To capture the OMCI packets for all ONUs:
+
+.. code:: bash
+
+ cat openonu.logs | grep -E "receive_message|_send_next" | grep msg > omci.dump
+
+To capture the OMCI packets for a particular ONU:
+
+.. code:: bash
+
+ cat openonu.logs | grep -E "receive_message|_send_next" | grep msg | grep [deviceId] > omci.dump
+
+Once you have the ``omci.dump`` file you need to prepare it to be imported in wireshark with this command:
+
+.. code:: bash
+
+ sed -n "s/.*[omci_msg|msg]: b'\(.*\)',.*$/\1/p" omci.dump | sed -e 's/.\{2\}/& /g' | sed -e 's/^/000000 /' > omci.hex
+
+And then in wireshark:
+
+- select ``File -> Import from Hex Dump``
+- select ``Encapsulation Type -> Ethernet``
+- set ``Ethernet -> Ethertype (hex): 88b5``