developer/roc.rst - aether-docs - Gitiles

 .. vim: syntax=rst

 Aether ROC Developer Guide
 ==========================

 Background / Development Environment
 ------------------------------------

 This document assumes familiarity with Kubernetes and Helm, and that a Kubernetes/Helm development
 environment has already been deployed in the developer’s work environment.
 This development environment can use any of a number of potential mechanisms -- including KinD, Kubeadm, etc.
 The Aether-in-a-Box script is one potential way to setup a development environment, but not the only way.
 As an alternative to the developer’s local machine, a remote environment can be set up, for example on
 cloud infrastructure such as cloudlab.

 Installing Prerequisites
 ------------------------

 Atomix and onos-operator must be installed::

    # create necessary namespaces
    kubectl create namespace micro-onos

    # install atomix
    helm -n kube-system install atomix-controller atomix/atomix-controller
    helm -n kube-system install atomix-raft-storage atomix/atomix-raft-storage

    # install the onos operator
    helm install -n kube-system onos-operator onosproject/onos-operator


 Verify that these services were installed properly.
 You should see pods for *atomix-controller*, *atomix-raft-storage-controller*,
 *onos-operator-config*, and *onos-operator-topo*.
 Execute these commands::

    kubectl -n kube-system get pods | grep -i atomix
    kubectl -n kube-system get pods | grep -i onos


 Create a values-override.yaml
 -----------------------------

 You’ll want to override several of the defaults in the ROC helm charts::

    cat > values-override.yaml <<EOF
    import:
    onos-gui:
       enabled: true

    onos-gui:
    ingress:
       enabled: false

    sdcore-adapter-v3:
    prometheusEnabled: false

    sdcore-exporter:
    prometheusEnabled: false

    onos-exporter:
    prometheusEnabled: false

    aether-roc-gui-v3:
    ingress:
       enabled: false
    EOF

 Installing the Aether-Roc-Umbrella Helm chart
 ---------------------------------------------

 Add the necessary helm repositories::

    # obtain username and password from Michelle and/or ONF infra team
    export repo_user=<username>
    export repo_password=<password>
    helm repo add sdran --username "$repo_user" --password "$repo_password" https://sdrancharts.onosproject.org

 Aether-Roc-Umbrella will bring up the ROC and its services::

    helm -n micro-onos install aether-roc-umbrella sdran/aether-roc-umbrella -f values-override.yaml

    kubectl wait pod -n micro-onos --for=condition=Ready -l type=config --timeout=300s


 Posting the mega-patch
 ----------------------

 The ROC usually comes up in a blank state -- there are no Enterprises, UEs, or other artifacts present in it.
 The mega-patch is an example patch that populates the ROC with some sample enterprises, UEs, slices, etc.
 Execute the following::

    # launch a port-forward for the API
    # this will continue to run in the background
    kubectl -n micro-onos port-forward service/aether-roc-api   --address 0.0.0.0 8181:8181 &

    git clone https://github.com/onosproject/aether-roc-api.git

    # execute the mega-patch (it will post via CURL to localhost:8181)
    bash ~/path/to/aether-roc-api/examples/MEGA_Patch.curl


 You may wish to customize the mega patch.
 For example, by default the patch configures the sdcore-adapter to push to sdcore-test-dummy.
 You could configure it to push to a live aether-in-a-box core by doing something like this::

    sed -i 's^http://aether-roc-umbrella-sdcore-test-dummy/v1/config/5g^http://webui.omec.svc.cluster.local:9089/config^g' MEGA_Patch.curl

    #apply the patch
    ./MEGA_Patch.curl

 (Note that if your Aether-in-a-Box was installed on a different machine that port-forwarding may be necessary)


 Expected CURL output from a successful mega-patch post will be a UUID.
 You can also verify that the mega-patch was successful by going into the aether-roc-gui in a browser
 (see the section on useful port-forwards below). The GUI may open to a dashboard that is unpopulated -- you
 can use the dropdown menu (upper-right hand corner of the screen) to select an object such as VCS and you
 will see a list of VCS.

    |ROCGUI|

 Uninstalling the Aether-Roc-Umbrella Helm chart
 -----------------------------------------------

 To tear things back down, usually as part of a developer loop prior to redeploying again, do the following::

    helm -n micro-onos del aether-roc-umbrella

 If the uninstall hangs or if a subsequent reinstall hangs, it could be an issue with some of the CRDs
 not getting cleaned up. The following may be useful::

    # fix stuck finalizers in operator CRDs

    kubectl -n micro-onos patch entities connectivity-service-v2 --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'

    kubectl -n micro-onos patch entities connectivity-service-v3 --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'

    kubectl -n micro-onos patch kind aether --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'

 Useful port forwards
 --------------------

 Port forwarding is often necessary to allow access to ports inside of Kubernetes pods that use ClusterIP addressing.
 Note that you typically need to leave a port-forward running (you can put it in the background).
 Also, If you redeploy the ROC and/or if a pod crashes then you might have to restart a port-forward.
 The following port-forwards may be useful::

    # aether-roc-api

    kubectl -n micro-onos port-forward service/aether-roc-api --address 0.0.0.0 8181:8181

    # aether-roc-gui

    kubectl -n micro-onos port-forward service/aether-roc-gui --address 0.0.0.0 8183:80

    # grafana

    kubectl -n micro-onos port-forward service/aether-roc-umbrella-grafana --address 0.0.0.0 8187:80

    # onos gui

    kubectl -n micro-onos port-forward service/onos-gui --address 0.0.0.0 8182:80

 Aether-roc-api and aether-roc-gui are in our experience the most useful two port-forwards.
 Aether-roc-api is useful to be able to POST REST API requests.
 Aether-roc-gui is useful to be able to interactively browse the current configuration.

 Deploying using custom images
 -----------------------------

 Custom images may be used by editing the values-override.yaml file.
 For example, to deploy a custom sdcore-adapter::

    sdcore-adapter-v3:

    prometheusEnabled: false

    image:

    repository: my-private-repo/sdcore-adapter

    tag: my-tag

    pullPolicy: Always

 The above example assumes you have published a docker images at my-private-repo/sdcore-adapter:my-tag.
 My particular workflow is to deploy a local-docker registry and push my images to that.
 Please do not publish ONF images to a public repository unless the image is intended to be public.
 Several ONF repositories are private, and therefore their docker artifacts should also be private.

 There are alternatives to using a private docker repository.
 For example, if you are using kubadm, then you may be able to simply tag the image locally.
 If you’re using KinD, then you can push a local image to into the kind cluster::

    kind load docker-image sdcore-adapter:my-tag

 Inspecting logs
 ---------------

 Most of the relevant Kubernetes pods are in the micro-onos namespace.
 The names may change from deployment to deployment, so start by getting a list of pods::

    kubectl -n micro-onos get pods

 Then you can inspect a specific pod/container::

    kubectl -n micro-onos logs sdcore-adapter-v3-7468cc58dc-ktctz sdcore-adapter-v3

 Some exercises to get familiar
 ------------------------------

 1) Deploy the ROC and POST the mega-patch, go into the aether-roc-GUI and click through the VCS, DeviceGroup, and
 other objects to see that they were created as expected.

 2) Examine the log of the sdcore-adapter-v3 container.
 It should be attempting to push the mega-patch’s changes.
 If you don’t have a core available, it may be failing the push, but you should see the attempts.

 3) Change an object in the GUI.
 Watch the sdcore-adapter-v3 log file and see that the adapter attempts to push the change.

 4) Try POSTing a change via the API.
 Observe the sdcore-adapter-v3 log file and see that the adapter attempts to push the change.

 5) Deploy a 5G Aether-in-a-Box (See sd-core developer guide), modify the mega-patch to specify the URL for the
 Aether-in-a-Box webui container, POST the mega-patch, and observe that the changes were correctly pushed via the
 sdcore-adapter-v3 into the sd-core’s webui container (webui container log will show configuration as it is
 received)

 .. |ROCGUI| image:: images/rocgui.png
	.. vim: syntax=rst

	Aether ROC Developer Guide
	==========================

	Background / Development Environment
	------------------------------------

	This document assumes familiarity with Kubernetes and Helm, and that a Kubernetes/Helm development
	environment has already been deployed in the developer’s work environment.
	This development environment can use any of a number of potential mechanisms -- including KinD, Kubeadm, etc.
	The Aether-in-a-Box script is one potential way to setup a development environment, but not the only way.
	As an alternative to the developer’s local machine, a remote environment can be set up, for example on
	cloud infrastructure such as cloudlab.

	Installing Prerequisites
	------------------------

	Atomix and onos-operator must be installed::

	# create necessary namespaces
	kubectl create namespace micro-onos

	# install atomix
	helm -n kube-system install atomix-controller atomix/atomix-controller
	helm -n kube-system install atomix-raft-storage atomix/atomix-raft-storage

	# install the onos operator
	helm install -n kube-system onos-operator onosproject/onos-operator


	Verify that these services were installed properly.
	You should see pods for atomix-controller, atomix-raft-storage-controller,
	onos-operator-config, and onos-operator-topo.
	Execute these commands::

	kubectl -n kube-system get pods \| grep -i atomix
	kubectl -n kube-system get pods \| grep -i onos


	Create a values-override.yaml
	-----------------------------

	You’ll want to override several of the defaults in the ROC helm charts::

	cat > values-override.yaml <<EOF
	import:
	onos-gui:
	enabled: true

	onos-gui:
	ingress:
	enabled: false

	sdcore-adapter-v3:
	prometheusEnabled: false

	sdcore-exporter:
	prometheusEnabled: false

	onos-exporter:
	prometheusEnabled: false

	aether-roc-gui-v3:
	ingress:
	enabled: false
	EOF

	Installing the Aether-Roc-Umbrella Helm chart
	---------------------------------------------

	Add the necessary helm repositories::

	# obtain username and password from Michelle and/or ONF infra team
	export repo_user=<username>
	export repo_password=<password>
	helm repo add sdran --username "$repo_user" --password "$repo_password" https://sdrancharts.onosproject.org

	Aether-Roc-Umbrella will bring up the ROC and its services::

	helm -n micro-onos install aether-roc-umbrella sdran/aether-roc-umbrella -f values-override.yaml

	kubectl wait pod -n micro-onos --for=condition=Ready -l type=config --timeout=300s


	Posting the mega-patch
	----------------------

	The ROC usually comes up in a blank state -- there are no Enterprises, UEs, or other artifacts present in it.
	The mega-patch is an example patch that populates the ROC with some sample enterprises, UEs, slices, etc.
	Execute the following::

	# launch a port-forward for the API
	# this will continue to run in the background
	kubectl -n micro-onos port-forward service/aether-roc-api --address 0.0.0.0 8181:8181 &

	git clone https://github.com/onosproject/aether-roc-api.git

	# execute the mega-patch (it will post via CURL to localhost:8181)
	bash ~/path/to/aether-roc-api/examples/MEGA_Patch.curl


	You may wish to customize the mega patch.
	For example, by default the patch configures the sdcore-adapter to push to sdcore-test-dummy.
	You could configure it to push to a live aether-in-a-box core by doing something like this::

	sed -i 's^http://aether-roc-umbrella-sdcore-test-dummy/v1/config/5g^http://webui.omec.svc.cluster.local:9089/config^g' MEGA_Patch.curl

	#apply the patch
	./MEGA_Patch.curl

	(Note that if your Aether-in-a-Box was installed on a different machine that port-forwarding may be necessary)


	Expected CURL output from a successful mega-patch post will be a UUID.
	You can also verify that the mega-patch was successful by going into the aether-roc-gui in a browser
	(see the section on useful port-forwards below). The GUI may open to a dashboard that is unpopulated -- you
	can use the dropdown menu (upper-right hand corner of the screen) to select an object such as VCS and you
	will see a list of VCS.

	\|ROCGUI\|

	Uninstalling the Aether-Roc-Umbrella Helm chart
	-----------------------------------------------

	To tear things back down, usually as part of a developer loop prior to redeploying again, do the following::

	helm -n micro-onos del aether-roc-umbrella

	If the uninstall hangs or if a subsequent reinstall hangs, it could be an issue with some of the CRDs
	not getting cleaned up. The following may be useful::

	# fix stuck finalizers in operator CRDs

	kubectl -n micro-onos patch entities connectivity-service-v2 --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'

	kubectl -n micro-onos patch entities connectivity-service-v3 --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'

	kubectl -n micro-onos patch kind aether --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'

	Useful port forwards
	--------------------

	Port forwarding is often necessary to allow access to ports inside of Kubernetes pods that use ClusterIP addressing.
	Note that you typically need to leave a port-forward running (you can put it in the background).
	Also, If you redeploy the ROC and/or if a pod crashes then you might have to restart a port-forward.
	The following port-forwards may be useful::

	# aether-roc-api

	kubectl -n micro-onos port-forward service/aether-roc-api --address 0.0.0.0 8181:8181

	# aether-roc-gui

	kubectl -n micro-onos port-forward service/aether-roc-gui --address 0.0.0.0 8183:80

	# grafana

	kubectl -n micro-onos port-forward service/aether-roc-umbrella-grafana --address 0.0.0.0 8187:80

	# onos gui

	kubectl -n micro-onos port-forward service/onos-gui --address 0.0.0.0 8182:80

	Aether-roc-api and aether-roc-gui are in our experience the most useful two port-forwards.
	Aether-roc-api is useful to be able to POST REST API requests.
	Aether-roc-gui is useful to be able to interactively browse the current configuration.

	Deploying using custom images
	-----------------------------

	Custom images may be used by editing the values-override.yaml file.
	For example, to deploy a custom sdcore-adapter::

	sdcore-adapter-v3:

	prometheusEnabled: false

	image:

	repository: my-private-repo/sdcore-adapter

	tag: my-tag

	pullPolicy: Always

	The above example assumes you have published a docker images at my-private-repo/sdcore-adapter:my-tag.
	My particular workflow is to deploy a local-docker registry and push my images to that.
	Please do not publish ONF images to a public repository unless the image is intended to be public.
	Several ONF repositories are private, and therefore their docker artifacts should also be private.

	There are alternatives to using a private docker repository.
	For example, if you are using kubadm, then you may be able to simply tag the image locally.
	If you’re using KinD, then you can push a local image to into the kind cluster::

	kind load docker-image sdcore-adapter:my-tag

	Inspecting logs
	---------------

	Most of the relevant Kubernetes pods are in the micro-onos namespace.
	The names may change from deployment to deployment, so start by getting a list of pods::

	kubectl -n micro-onos get pods

	Then you can inspect a specific pod/container::

	kubectl -n micro-onos logs sdcore-adapter-v3-7468cc58dc-ktctz sdcore-adapter-v3

	Some exercises to get familiar
	------------------------------

	1) Deploy the ROC and POST the mega-patch, go into the aether-roc-GUI and click through the VCS, DeviceGroup, and
	other objects to see that they were created as expected.

	2) Examine the log of the sdcore-adapter-v3 container.
	It should be attempting to push the mega-patch’s changes.
	If you don’t have a core available, it may be failing the push, but you should see the attempts.

	3) Change an object in the GUI.
	Watch the sdcore-adapter-v3 log file and see that the adapter attempts to push the change.

	4) Try POSTing a change via the API.
	Observe the sdcore-adapter-v3 log file and see that the adapter attempts to push the change.

	5) Deploy a 5G Aether-in-a-Box (See sd-core developer guide), modify the mega-patch to specify the URL for the
	Aether-in-a-Box webui container, POST the mega-patch, and observe that the changes were correctly pushed via the
	sdcore-adapter-v3 into the sd-core’s webui container (webui container log will show configuration as it is
	received)

	.. \|ROCGUI\| image:: images/rocgui.png