add developer documentation
Change-Id: I95654c6260a327fe52bde2b372668540ac25bcd1
diff --git a/developer/roc.rst b/developer/roc.rst
new file mode 100644
index 0000000..a3b2f9b
--- /dev/null
+++ b/developer/roc.rst
@@ -0,0 +1,231 @@
+.. vim: syntax=rst
+
+Aether ROC Developer Guide
+==========================
+
+Background / Development Environment
+------------------------------------
+
+This document assumes familiarity with Kubernetes and Helm, and that a Kubernetes/Helm development
+environment has already been deployed in the developer’s work environment.
+This development environment can use any of a number of potential mechanisms -- including KinD, Kubeadm, etc.
+The Aether-in-a-Box script is one potential way to setup a development environment, but not the only way.
+As an alternative to the developer’s local machine, a remote environment can be set up, for example on
+cloud infrastructure such as cloudlab.
+
+Installing Prerequisites
+------------------------
+
+Atomix and onos-operator must be installed::
+
+ # create necessary namespaces
+ kubectl create namespace micro-onos
+
+ # install atomix
+ helm -n kube-system install atomix-controller atomix/atomix-controller
+ helm -n kube-system install atomix-raft-storage atomix/atomix-raft-storage
+
+ # install the onos operator
+ helm install -n kube-system onos-operator onosproject/onos-operator
+
+
+Verify that these services were installed properly.
+You should see pods for *atomix-controller*, *atomix-raft-storage-controller*,
+*onos-operator-config*, and *onos-operator-topo*.
+Execute these commands::
+
+ kubectl -n kube-system get pods | grep -i atomix
+ kubectl -n kube-system get pods | grep -i onos
+
+
+Create a values-override.yaml
+-----------------------------
+
+You’ll want to override several of the defaults in the ROC helm charts::
+
+ cat > values-override.yaml <<EOF
+ import:
+ onos-gui:
+ enabled: true
+
+ onos-gui:
+ ingress:
+ enabled: false
+
+ sdcore-adapter-v3:
+ prometheusEnabled: false
+
+ sdcore-exporter:
+ prometheusEnabled: false
+
+ onos-exporter:
+ prometheusEnabled: false
+
+ aether-roc-gui-v3:
+ ingress:
+ enabled: false
+ EOF
+
+Installing the Aether-Roc-Umbrella Helm chart
+---------------------------------------------
+
+Add the necessary helm repositories::
+
+ # obtain username and password from Michelle and/or ONF infra team
+ export repo_user=<username>
+ export repo_password=<password>
+ helm repo add sdran --username "$repo_user" --password "$repo_password" https://sdrancharts.onosproject.org
+
+Aether-Roc-Umbrella will bring up the ROC and its services::
+
+ helm -n micro-onos install aether-roc-umbrella sdran/aether-roc-umbrella -f values-override.yaml
+
+ kubectl wait pod -n micro-onos --for=condition=Ready -l type=config --timeout=300s
+
+
+Posting the mega-patch
+----------------------
+
+The ROC usually comes up in a blank state -- there are no Enterprises, UEs, or other artifacts present in it.
+The mega-patch is an example patch that populates the ROC with some sample enterprises, UEs, slices, etc.
+Execute the following::
+
+ # launch a port-forward for the API
+ # this will continue to run in the background
+ kubectl -n micro-onos port-forward service/aether-roc-api --address 0.0.0.0 8181:8181 &
+
+ git clone https://github.com/onosproject/aether-roc-api.git
+
+ # execute the mega-patch (it will post via CURL to localhost:8181)
+ bash ~/path/to/aether-roc-api/examples/MEGA_Patch.curl
+
+
+You may wish to customize the mega patch.
+For example, by default the patch configures the sdcore-adapter to push to sdcore-test-dummy.
+You could configure it to push to a live aether-in-a-box core by doing something like this::
+
+ sed -i 's^http://aether-roc-umbrella-sdcore-test-dummy/v1/config/5g^http://webui.omec.svc.cluster.local:9089/config^g' MEGA_Patch.curl
+
+ #apply the patch
+ ./MEGA_Patch.curl
+
+(Note that if your Aether-in-a-Box was installed on a different machine that port-forwarding may be necessary)
+
+
+Expected CURL output from a successful mega-patch post will be a UUID.
+You can also verify that the mega-patch was successful by going into the aether-roc-gui in a browser
+(see the section on useful port-forwards below). The GUI may open to a dashboard that is unpopulated -- you
+can use the dropdown menu (upper-right hand corner of the screen) to select an object such as VCS and you
+will see a list of VCS.
+
+ |ROCGUI|
+
+Uninstalling the Aether-Roc-Umbrella Helm chart
+-----------------------------------------------
+
+To tear things back down, usually as part of a developer loop prior to redeploying again, do the following::
+
+ helm -n micro-onos del aether-roc-umbrella
+
+If the uninstall hangs or if a subsequent reinstall hangs, it could be an issue with some of the CRDs
+not getting cleaned up. The following may be useful::
+
+ # fix stuck finalizers in operator CRDs
+
+ kubectl -n micro-onos patch entities connectivity-service-v2 --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'
+
+ kubectl -n micro-onos patch entities connectivity-service-v3 --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'
+
+ kubectl -n micro-onos patch kind aether --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'
+
+Useful port forwards
+--------------------
+
+Port forwarding is often necessary to allow access to ports inside of Kubernetes pods that use ClusterIP addressing.
+Note that you typically need to leave a port-forward running (you can put it in the background).
+Also, If you redeploy the ROC and/or if a pod crashes then you might have to restart a port-forward.
+The following port-forwards may be useful::
+
+ # aether-roc-api
+
+ kubectl -n micro-onos port-forward service/aether-roc-api --address 0.0.0.0 8181:8181
+
+ # aether-roc-gui
+
+ kubectl -n micro-onos port-forward service/aether-roc-gui --address 0.0.0.0 8183:80
+
+ # grafana
+
+ kubectl -n micro-onos port-forward service/aether-roc-umbrella-grafana --address 0.0.0.0 8187:80
+
+ # onos gui
+
+ kubectl -n micro-onos port-forward service/onos-gui --address 0.0.0.0 8182:80
+
+Aether-roc-api and aether-roc-gui are in our experience the most useful two port-forwards.
+Aether-roc-api is useful to be able to POST REST API requests.
+Aether-roc-gui is useful to be able to interactively browse the current configuration.
+
+Deploying using custom images
+-----------------------------
+
+Custom images may be used by editing the values-override.yaml file.
+For example, to deploy a custom sdcore-adapter::
+
+ sdcore-adapter-v3:
+
+ prometheusEnabled: false
+
+ image:
+
+ repository: my-private-repo/sdcore-adapter
+
+ tag: my-tag
+
+ pullPolicy: Always
+
+The above example assumes you have published a docker images at my-private-repo/sdcore-adapter:my-tag.
+My particular workflow is to deploy a local-docker registry and push my images to that.
+Please do not publish ONF images to a public repository unless the image is intended to be public.
+Several ONF repositories are private, and therefore their docker artifacts should also be private.
+
+There are alternatives to using a private docker repository.
+For example, if you are using kubadm, then you may be able to simply tag the image locally.
+If you’re using KinD, then you can push a local image to into the kind cluster::
+
+ kind load docker-image sdcore-adapter:my-tag
+
+Inspecting logs
+---------------
+
+Most of the relevant Kubernetes pods are in the micro-onos namespace.
+The names may change from deployment to deployment, so start by getting a list of pods::
+
+ kubectl -n micro-onos get pods
+
+Then you can inspect a specific pod/container::
+
+ kubectl -n micro-onos logs sdcore-adapter-v3-7468cc58dc-ktctz sdcore-adapter-v3
+
+Some exercises to get familiar
+------------------------------
+
+1) Deploy the ROC and POST the mega-patch, go into the aether-roc-GUI and click through the VCS, DeviceGroup, and
+other objects to see that they were created as expected.
+
+2) Examine the log of the sdcore-adapter-v3 container.
+It should be attempting to push the mega-patch’s changes.
+If you don’t have a core available, it may be failing the push, but you should see the attempts.
+
+3) Change an object in the GUI.
+Watch the sdcore-adapter-v3 log file and see that the adapter attempts to push the change.
+
+4) Try POSTing a change via the API.
+Observe the sdcore-adapter-v3 log file and see that the adapter attempts to push the change.
+
+5) Deploy a 5G Aether-in-a-Box (See sd-core developer guide), modify the mega-patch to specify the URL for the
+Aether-in-a-Box webui container, POST the mega-patch, and observe that the changes were correctly pushed via the
+sdcore-adapter-v3 into the sd-core’s webui container (webui container log will show configuration as it is
+received)
+
+.. |ROCGUI| image:: images/rocgui.png