Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 1 | .. vim: syntax=rst |
| 2 | |
| 3 | Aether ROC Developer Guide |
| 4 | ========================== |
| 5 | |
| 6 | Background / Development Environment |
| 7 | ------------------------------------ |
| 8 | |
| 9 | This document assumes familiarity with Kubernetes and Helm, and that a Kubernetes/Helm development |
| 10 | environment has already been deployed in the developer’s work environment. |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 11 | |
| 12 | This development environment can use any of a number of potential mechanisms -- including KinD, kubeadm, etc. |
| 13 | |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 14 | The Aether-in-a-Box script is one potential way to setup a development environment, but not the only way. |
| 15 | As an alternative to the developer’s local machine, a remote environment can be set up, for example on |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 16 | cloud infrastructure such as Cloudlab. |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 17 | |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 18 | .. note:: When ROC is deployed it is unsecured by default, with no Authentication or Authorization. |
| 19 | To secure ROC so that the Authentication and Authorization can be tested, follow the Securing ROC |
| 20 | guide below :ref:`securing_roc` |
| 21 | |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 22 | Installing Prerequisites |
| 23 | ------------------------ |
| 24 | |
| 25 | Atomix and onos-operator must be installed:: |
| 26 | |
| 27 | # create necessary namespaces |
| 28 | kubectl create namespace micro-onos |
| 29 | |
Andy Bavier | 4c42541 | 2021-08-27 14:39:38 -0700 | [diff] [blame] | 30 | # add repos |
| 31 | helm repo add atomix https://charts.atomix.io |
| 32 | helm repo add onosproject https://charts.onosproject.org |
| 33 | helm repo update |
| 34 | |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 35 | # install atomix |
Sean Condon | 70dcf70 | 2021-08-24 10:57:29 +0100 | [diff] [blame] | 36 | export ATOMIX_CONTROLLER_VERSION=0.6.8 |
Sean Condon | 257687f | 2021-08-23 11:13:20 +0100 | [diff] [blame] | 37 | helm -n kube-system install atomix-controller atomix/atomix-controller --version $ATOMIX_CONTROLLER_VERSION |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 38 | export ATOMIX_RAFT_VERSION=0.1.15 |
Sean Condon | 70dcf70 | 2021-08-24 10:57:29 +0100 | [diff] [blame] | 39 | helm -n kube-system install atomix-raft-storage atomix/atomix-raft-storage --version $ATOMIX_RAFT_VERSION |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 40 | |
| 41 | # install the onos operator |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 42 | ONOS_OPERATOR_VERSION=0.4.14 |
Sean Condon | 257687f | 2021-08-23 11:13:20 +0100 | [diff] [blame] | 43 | helm install -n kube-system onos-operator onosproject/onos-operator --version $ONOS_OPERATOR_VERSION |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 44 | |
Sean Condon | 257687f | 2021-08-23 11:13:20 +0100 | [diff] [blame] | 45 | .. note:: The ROC is sensitive to the versions of Atomix and onos-operator installed. The values |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 46 | shown above are correct for the 1.4.42- versions of the *aether-roc-umbrella*. |
Sean Condon | 257687f | 2021-08-23 11:13:20 +0100 | [diff] [blame] | 47 | |
| 48 | .. list-table:: ROC support component version matrix |
Sean Condon | 70dcf70 | 2021-08-24 10:57:29 +0100 | [diff] [blame] | 49 | :widths: 40 20 20 20 |
Sean Condon | 257687f | 2021-08-23 11:13:20 +0100 | [diff] [blame] | 50 | :header-rows: 1 |
| 51 | |
| 52 | * - ROC Version |
| 53 | - Atomix Controller |
| 54 | - Atomix Raft |
| 55 | - Onos Operator |
Sean Condon | 70dcf70 | 2021-08-24 10:57:29 +0100 | [diff] [blame] | 56 | * - 1.2.25-1.2.45 |
Sean Condon | 257687f | 2021-08-23 11:13:20 +0100 | [diff] [blame] | 57 | - 0.6.7 |
| 58 | - 0.1.8 |
| 59 | - 0.4.8 |
Zack Williams | 90578fa | 2021-11-04 11:57:23 -0700 | [diff] [blame] | 60 | * - 1.3.0-1.3.10 |
Sean Condon | 70dcf70 | 2021-08-24 10:57:29 +0100 | [diff] [blame] | 61 | - 0.6.8 |
| 62 | - 0.1.9 |
Sean Condon | 1df1fcf | 2021-09-20 09:45:39 +0100 | [diff] [blame] | 63 | - 0.4.10 |
Zack Williams | 90578fa | 2021-11-04 11:57:23 -0700 | [diff] [blame] | 64 | * - 1.3.11-,1.4.0- |
| 65 | - 0.6.8 |
| 66 | - 0.1.14 |
| 67 | - 0.4.12 |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 68 | * - 1.4.42- |
| 69 | - 0.6.8 |
| 70 | - 0.1.15 |
| 71 | - 0.4.14 |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 72 | |
| 73 | Verify that these services were installed properly. |
| 74 | You should see pods for *atomix-controller*, *atomix-raft-storage-controller*, |
| 75 | *onos-operator-config*, and *onos-operator-topo*. |
| 76 | Execute these commands:: |
| 77 | |
Sean Condon | 257687f | 2021-08-23 11:13:20 +0100 | [diff] [blame] | 78 | helm -n kube-system list |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 79 | kubectl -n kube-system get pods | grep -i atomix |
| 80 | kubectl -n kube-system get pods | grep -i onos |
| 81 | |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 82 | Create a values-override.yaml |
| 83 | ----------------------------- |
| 84 | |
| 85 | You’ll want to override several of the defaults in the ROC helm charts:: |
| 86 | |
| 87 | cat > values-override.yaml <<EOF |
| 88 | import: |
Scott Baker | b46a6ed | 2021-08-02 14:03:10 -0700 | [diff] [blame] | 89 | onos-gui: |
| 90 | enabled: true |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 91 | |
| 92 | onos-gui: |
Scott Baker | b46a6ed | 2021-08-02 14:03:10 -0700 | [diff] [blame] | 93 | ingress: |
| 94 | enabled: false |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 95 | |
| 96 | aether-roc-gui-v3: |
Scott Baker | b46a6ed | 2021-08-02 14:03:10 -0700 | [diff] [blame] | 97 | ingress: |
| 98 | enabled: false |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 99 | EOF |
| 100 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 101 | Installing the ``aether-roc-umbrella`` Helm chart |
| 102 | ------------------------------------------------- |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 103 | |
| 104 | Add the necessary helm repositories:: |
| 105 | |
| 106 | # obtain username and password from Michelle and/or ONF infra team |
| 107 | export repo_user=<username> |
| 108 | export repo_password=<password> |
Sean Condon | 1df1fcf | 2021-09-20 09:45:39 +0100 | [diff] [blame] | 109 | helm repo add aether --username "$repo_user" --password "$repo_password" https://charts.aetherproject.org |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 110 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 111 | ``aether-roc-umbrella`` will bring up the ROC and its services:: |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 112 | |
Sean Condon | 1df1fcf | 2021-09-20 09:45:39 +0100 | [diff] [blame] | 113 | helm -n micro-onos install aether-roc-umbrella aether/aether-roc-umbrella -f values-override.yaml |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 114 | |
| 115 | kubectl wait pod -n micro-onos --for=condition=Ready -l type=config --timeout=300s |
| 116 | |
| 117 | |
Sean Condon | f918f64 | 2021-08-04 14:32:53 +0100 | [diff] [blame] | 118 | .. _posting-the-mega-patch: |
| 119 | |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 120 | Posting the mega-patch |
| 121 | ---------------------- |
| 122 | |
| 123 | The ROC usually comes up in a blank state -- there are no Enterprises, UEs, or other artifacts present in it. |
| 124 | The mega-patch is an example patch that populates the ROC with some sample enterprises, UEs, slices, etc. |
| 125 | Execute the following:: |
| 126 | |
| 127 | # launch a port-forward for the API |
| 128 | # this will continue to run in the background |
| 129 | kubectl -n micro-onos port-forward service/aether-roc-api --address 0.0.0.0 8181:8181 & |
| 130 | |
| 131 | git clone https://github.com/onosproject/aether-roc-api.git |
| 132 | |
| 133 | # execute the mega-patch (it will post via CURL to localhost:8181) |
Zack Williams | 90578fa | 2021-11-04 11:57:23 -0700 | [diff] [blame] | 134 | bash ~/path/to/aether-roc-api/examples/MEGA_Patch_40.curl |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 135 | |
| 136 | |
| 137 | You may wish to customize the mega patch. |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 138 | |
| 139 | For example, by default the patch configures the ``sdcore-adapter`` to push to |
| 140 | ``sdcore-test-dummy``. |
| 141 | |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 142 | You could configure it to push to a live aether-in-a-box core by doing something like this:: |
| 143 | |
Zack Williams | 90578fa | 2021-11-04 11:57:23 -0700 | [diff] [blame] | 144 | sed -i 's^http://aether-roc-umbrella-sdcore-test-dummy/v1/config/5g^http://webui.omec.svc.cluster.local:9089/config^g' MEGA_Patch_40.curl |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 145 | |
| 146 | #apply the patch |
Zack Williams | 90578fa | 2021-11-04 11:57:23 -0700 | [diff] [blame] | 147 | ./MEGA_Patch_40.curl |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 148 | |
| 149 | (Note that if your Aether-in-a-Box was installed on a different machine that port-forwarding may be necessary) |
| 150 | |
| 151 | |
| 152 | Expected CURL output from a successful mega-patch post will be a UUID. |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 153 | |
| 154 | You can also verify that the mega-patch was successful by going into the |
| 155 | ``aether-roc-gui`` in a browser (see the section on useful port-forwards |
| 156 | below). The GUI may open to a dashboard that is unpopulated -- you can use the |
| 157 | dropdown menu (upper-right hand corner of the screen) to select an object such |
| 158 | as VCS and you will see a list of VCS. |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 159 | |
| 160 | |ROCGUI| |
| 161 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 162 | Uninstalling the ``aether-roc-umbrella`` Helm chart |
| 163 | --------------------------------------------------- |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 164 | |
| 165 | To tear things back down, usually as part of a developer loop prior to redeploying again, do the following:: |
| 166 | |
| 167 | helm -n micro-onos del aether-roc-umbrella |
| 168 | |
| 169 | If the uninstall hangs or if a subsequent reinstall hangs, it could be an issue with some of the CRDs |
| 170 | not getting cleaned up. The following may be useful:: |
| 171 | |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 172 | # fix stuck finalizers in operator CRDs |
| 173 | kubectl -n micro-onos patch entities connectivity-service-v4 --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]' && \ |
| 174 | kubectl -n micro-onos patch entities connectivity-service-v3 --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]' && \ |
| 175 | kubectl -n micro-onos patch entities plproxy-amp --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]' && \ |
| 176 | kubectl -n micro-onos patch entities plproxy-acc --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]' && \ |
| 177 | kubectl -n micro-onos patch kind plproxy --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]' && \ |
| 178 | kubectl -n micro-onos patch kind aether --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]' |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 179 | |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 180 | |
| 181 | Useful port forwards |
| 182 | -------------------- |
| 183 | |
| 184 | Port forwarding is often necessary to allow access to ports inside of Kubernetes pods that use ClusterIP addressing. |
| 185 | Note that you typically need to leave a port-forward running (you can put it in the background). |
| 186 | Also, If you redeploy the ROC and/or if a pod crashes then you might have to restart a port-forward. |
| 187 | The following port-forwards may be useful:: |
| 188 | |
| 189 | # aether-roc-api |
| 190 | |
| 191 | kubectl -n micro-onos port-forward service/aether-roc-api --address 0.0.0.0 8181:8181 |
| 192 | |
| 193 | # aether-roc-gui |
| 194 | |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 195 | kubectl -n micro-onos port-forward service/aether-roc-gui-v4 --address 0.0.0.0 8183:80 |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 196 | |
| 197 | # grafana |
| 198 | |
| 199 | kubectl -n micro-onos port-forward service/aether-roc-umbrella-grafana --address 0.0.0.0 8187:80 |
| 200 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 201 | ``aether-roc-api`` and ``aether-roc-gui`` are in our experience the most useful two port-forwards. |
| 202 | |
| 203 | ``aether-roc-api`` is useful to be able to POST REST API requests. |
| 204 | |
| 205 | ``aether-roc-gui`` is useful to be able to interactively browse the current configuration. |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 206 | |
Sean Condon | 257687f | 2021-08-23 11:13:20 +0100 | [diff] [blame] | 207 | .. note:: Internally the ``aether-roc-gui`` operates a Reverse Proxy on the ``aether-roc-api``. This |
| 208 | means that if you have done a ``port-forward`` to ``aether-roc-gui`` say on port ``8183`` there's no |
| 209 | need to do another on the ``aether-roc-api`` instead you can access the API on |
| 210 | ``http://localhost:8183/aether-roc-api`` |
| 211 | |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 212 | Deploying using custom images |
| 213 | ----------------------------- |
| 214 | |
| 215 | Custom images may be used by editing the values-override.yaml file. |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 216 | For example, to deploy a custom ``sdcore-adapter``:: |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 217 | |
| 218 | sdcore-adapter-v3: |
| 219 | |
| 220 | prometheusEnabled: false |
| 221 | |
| 222 | image: |
| 223 | |
| 224 | repository: my-private-repo/sdcore-adapter |
| 225 | |
| 226 | tag: my-tag |
| 227 | |
| 228 | pullPolicy: Always |
| 229 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 230 | The above example assumes you have published a docker images at ``my-private-repo/sdcore-adapter:my-tag``. |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 231 | My particular workflow is to deploy a local-docker registry and push my images to that. |
| 232 | Please do not publish ONF images to a public repository unless the image is intended to be public. |
| 233 | Several ONF repositories are private, and therefore their docker artifacts should also be private. |
| 234 | |
| 235 | There are alternatives to using a private docker repository. |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 236 | For example, if you are using kubeadm, then you may be able to simply tag the image locally. |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 237 | If you’re using KinD, then you can push a local image to into the kind cluster:: |
| 238 | |
| 239 | kind load docker-image sdcore-adapter:my-tag |
| 240 | |
Scott Baker | abcfc6e | 2021-09-08 22:37:51 -0700 | [diff] [blame] | 241 | Developing using a custom onos-config |
| 242 | ------------------------------------- |
| 243 | |
| 244 | The onos-operator is responsible for building model plugins at runtime. To do this, it needs source code |
| 245 | for onos-config that matches the onos-config image that is deployed. One way to do this is to fork the |
| 246 | onos-config repository and commit your onos-config changes to a personal repository, and then reference |
| 247 | that personal repository in the values.yaml. For example:: |
| 248 | |
| 249 | onos-config: |
| 250 | plugin: |
| 251 | compiler: |
| 252 | target: "github.com/mygithubaccount/onos-config@mytag" |
| 253 | image: |
| 254 | repository: mydockeraccount/onos-config |
| 255 | tag: mytag |
| 256 | pullPolicy: Always |
| 257 | |
| 258 | In the above example, the operator will pull the image from `mydockeraccount`, and it'll pull the |
| 259 | onos-config code from `mygithubaccount`. Using a personal docker account is not strictly necessary; |
| 260 | images can also be built and tagged entirely locally. |
| 261 | |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 262 | Inspecting logs |
| 263 | --------------- |
| 264 | |
| 265 | Most of the relevant Kubernetes pods are in the micro-onos namespace. |
| 266 | The names may change from deployment to deployment, so start by getting a list of pods:: |
| 267 | |
| 268 | kubectl -n micro-onos get pods |
| 269 | |
| 270 | Then you can inspect a specific pod/container:: |
| 271 | |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 272 | kubectl -n micro-onos logs deployment/sdcore-adapter-v4 |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 273 | |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 274 | .. _securing_roc: |
| 275 | |
| 276 | Securing ROC |
| 277 | ------------ |
| 278 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 279 | When deploying ROC with the ``aether-roc-umbrella`` chart, secure mode can be enabled by |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 280 | specifying an OpenID Connect (OIDC) issuer like:: |
| 281 | |
Andy Bavier | e86261e | 2021-09-21 11:05:18 -0700 | [diff] [blame] | 282 | helm -n micro-onos install aether-roc-umbrella aether/aether-roc-umbrella \ |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 283 | --set onos-config.openidc.issuer=http://k3u-keycloak:80/auth/realms/master \ |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 284 | --set aether-roc-api.openidc.issuer=http://k3u-keycloak:80/auth/realms/master \ |
| 285 | --set aether-roc-gui-v4.openidc.issuer=http://k3u-keycloak:5557/auth/realms/master \ |
| 286 | --set prom-label-proxy-acc.config.openidc.issuer=http://k3u-keycloak:80/auth/realms/master \ |
| 287 | --set prom-label-proxy-amp.config.openidc.issuer=http://k3u-keycloak:80/auth/realms/master |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 288 | |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 289 | The choice of OIDC issuer in this case is ``keycloak-389-umbrella``, or alternately ``dex-ldap-umbrella`` |
| 290 | (`deprecated <https://github.com/onosproject/onos-helm-charts/tree/master/dex-ldap-umbrella>`_). |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 291 | |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 292 | ``keycloak-389-umbrella`` |
| 293 | """"""""""""""""""""""""" |
| 294 | |
| 295 | Keycloak is an Open Source Identity and Access Management for Modern Applications and |
| 296 | Services. It can be used as an OIDC Issuer than can act as a front end to several authentication systems |
| 297 | e.g. LDAP, Crowd, Google, GitHub |
| 298 | |
| 299 | ``keycloak-389-umbrella`` is a Helm chart that combines a Keycloak server with an LDAP |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 300 | installation (389 Directory Server), and an LDAP administration tool. It can be deployed (with name ``k3u`` in to the |
| 301 | same cluster namespace as ``aether-roc-umbrella``:: |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 302 | |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 303 | helm -n micro-onos install k3u onosproject/keycloak-389-umbrella |
| 304 | |
| 305 | |
| 306 | .. note:: Its LDAP server is populated with 7 different users in the 2 example enterprises - *starbucks* and *acme*. |
| 307 | |
| 308 | To make the deployment available with the hostname ``k3u-keycloak`` requires: |
| 309 | |
| 310 | #. a port forward like ``kubectl -n micro-onos port-forward service/k3u-keycloak --address=0.0.0.0 5557:80`` |
| 311 | #. editing your ``/etc/hosts`` file (on the machine where your browser runs) so that the name ``k3u-keycloak`` points |
| 312 | to the IP address of the machine where the ``port-forward`` runs (usually ``localhost``). |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 313 | |
| 314 | When running it should be available at *http://k3u-keycloak:5557/auth/realms/master/.well-known/openid-configuration*. |
| 315 | |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 316 | .. note:: You can access the Keycloak management page from *http://k3u-keycloak:5557/auth/admin* but you must |
| 317 | login as `admin/changeme`. Because of the SSO feature of Keycloak this will affect your Aether ROC GUI login too. |
| 318 | To login as 2 separate users at the same time, use a private browser window for one. |
| 319 | |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 320 | See `keycloak-389-umbrella <https://github.com/onosproject/onos-helm-charts/tree/master/keycloak-389-umbrella#readme>`_ |
| 321 | for more details. |
| 322 | |
| 323 | In a production environment, the public Aether Keycloak (with its LDAP server populated with Aether users and groups) should be used. |
| 324 | See `public keycloak <https://keycloak.opennetworking.org/auth/realms/master/.well-known/openid-configuration>`_ for more details. |
| 325 | |
| 326 | .. note:: Your RBAC access to ROC will be limited by the groups you belong to in its LDAP store. |
| 327 | |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 328 | Role Based Access Control |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 329 | """"""""""""""""""""""""" |
| 330 | |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 331 | When secured, access to the configuration in ROC is limited by the **groups** that a user belongs to. |
| 332 | |
| 333 | * **AetherROCAdmin** - users in this group have full read **and** write access to all configuration. |
| 334 | * *<enterprise>* - users in a group the lowercase name of an enterprise, will have **read** access to that enterprise. |
| 335 | * **EnterpriseAdmin** - users in this group will have read **and** write access the enterprise they belong to. |
| 336 | |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 337 | For example in *keycloak-389-umbrella* the user *Daisy Duke* belongs to *starbucks* **and** |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 338 | *EnterpriseAdmin* and so has read **and** write access to items linked with *starbucks* enterprise. |
| 339 | |
| 340 | By comparison the user *Elmer Fudd* belongs only to *starbucks* group and so has only **read** access to items |
| 341 | linked with the *starbucks* enterprise. |
| 342 | |
| 343 | Requests to a Secure System |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 344 | """"""""""""""""""""""""""" |
| 345 | |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 346 | When configuration is retrieved or updated through *aether-config*, a Bearer Token in the |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 347 | form of a JSON Web Token (JWT) issued by the selected OIDC Issuer server must accompany |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 348 | the request as an Authorization Header. |
| 349 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 350 | This applies to both the REST interface of ``aether-roc-api`` **and** the *gnmi* interface of |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 351 | ``aether-config``. |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 352 | |
| 353 | In the Aether ROC, a Bearer Token can be generated by logging in and selecting API Key from the |
| 354 | menu. This pops up a window with a copy button, where the key can be copied. |
| 355 | |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 356 | Alternatively with Keycloak a Token may be requested programmatically through the Keycloak API:: |
| 357 | |
| 358 | curl --location --request POST 'http://k3u-keycloak:5557/auth/realms/master/protocol/openid-connect/token' \ |
| 359 | --header 'Content-Type: application/x-www-form-urlencoded' \ |
| 360 | --data-urlencode 'grant_type=password' \ |
| 361 | --data-urlencode 'client_id=aether-roc-gui' \ |
| 362 | --data-urlencode 'username=alicea' \ |
| 363 | --data-urlencode 'password=password' \ |
| 364 | --data-urlencode 'scope=openid profile email groups' | jq "access_token" |
| 365 | |
| 366 | |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 367 | The key will expire after 24 hours. |
| 368 | |
| 369 | .. image:: images/aether-roc-gui-copy-api-key.png |
| 370 | :width: 580 |
| 371 | :alt: Aether ROC GUI allows copying of API Key to clipboard |
| 372 | |
| 373 | Accessing the REST interface from a tool like Postman, should include this Auth token. |
| 374 | |
| 375 | .. image:: images/postman-auth-token.png |
| 376 | :width: 930 |
| 377 | :alt: Postman showing Authentication Token pasted in |
| 378 | |
| 379 | Logging |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 380 | """"""" |
| 381 | |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 382 | The logs of *aether-config* will contain the **username** and **timestamp** of |
| 383 | any **gnmi** call when security is enabled. |
| 384 | |
| 385 | .. image:: images/aether-config-log.png |
| 386 | :width: 887 |
| 387 | :alt: aether-config log message showing username and timestamp |
| 388 | |
Sean Condon | 435be9a | 2021-08-06 14:28:37 +0100 | [diff] [blame] | 389 | Accessing GUI from an external system |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 390 | """"""""""""""""""""""""""""""""""""" |
| 391 | |
Sean Condon | 435be9a | 2021-08-06 14:28:37 +0100 | [diff] [blame] | 392 | To access the ROC GUI from a computer outside the Cluster machine using *port-forwarding* then |
| 393 | it is necessary to: |
| 394 | |
| 395 | * Ensure that all *port-forward*'s have **--address=0.0.0.0** |
| 396 | * Add to the IP address of the cluster machine to the **/etc/hosts** of the outside computer as:: |
| 397 | |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 398 | <ip address of cluster> k3u-keycloak aether-roc-gui |
| 399 | * Verify that you can access the Keycloak server by its name *http://k3u-keycloak:5557/auth/realms/master/.well-known/openid-configuration* |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 400 | * Access the GUI through the hostname (rather than ip address) ``http://aether-roc-gui:8183`` |
Sean Condon | 435be9a | 2021-08-06 14:28:37 +0100 | [diff] [blame] | 401 | |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 402 | Troubleshooting Secure Access |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 403 | """"""""""""""""""""""""""""" |
| 404 | |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 405 | While every effort has been made to ensure that securing Aether is simple and effective, |
| 406 | some difficulties may arise. |
| 407 | |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 408 | One of the most important steps is to validate that the OIDC Issuer (Keycloak server) can be reached |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 409 | from the browser. The **well_known** URL should be available and show the important endpoints are correct. |
| 410 | |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 411 | .. image:: images/keycloak-389-umbrella-well-known.png |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 412 | :width: 580 |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 413 | :alt: Keycloak Well Known page |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 414 | |
| 415 | If logged out of the Browser when accessing the Aether ROC GUI, accessing any page of the application should |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 416 | redirect to the Keycloak login page. |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 417 | |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 418 | .. image:: images/keycloak-ldap-login-page.png |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 419 | :width: 493 |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 420 | :alt: Keycloak Login page |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 421 | |
| 422 | When logged in the User details can be seen by clicking the User's name in the drop down menu. |
| 423 | This shows the **groups** that the user belongs to, and can be used to debug RBAC issues. |
| 424 | |
| 425 | .. image:: images/aether-roc-gui-user-details.png |
| 426 | :width: 700 |
| 427 | :alt: User Details page |
| 428 | |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 429 | When you sign out of the ROC GUI, if you are not redirected to the Keycloak Login Page, |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 430 | you should check the Developer Console of the browser. The console should show the correct |
Sean Condon | 948aeaa | 2021-09-29 12:08:30 +0100 | [diff] [blame] | 431 | OIDC issuer (Keycloak server), and that Auth is enabled. |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 432 | |
| 433 | .. image:: images/aether-roc-gui-console-loggedin.png |
| 434 | :width: 418 |
| 435 | :alt: Browser Console showing correct configuration |
| 436 | |
Sean Condon | 8d55a1e | 2021-11-11 12:49:00 +0000 | [diff] [blame^] | 437 | Keycloak installation issues |
| 438 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 439 | |
| 440 | The ``fedorea-389ds`` pod may restart a couple of times before it finally reaches running. |
| 441 | There are 2 post install jobs that take some time to start. Overall allow 3 minutes for startup. |
| 442 | |
| 443 | Some users are finding that the Fedora pod will never reach a running state on resource |
| 444 | constrained machines. This issue is being investigated. |
| 445 | |
| 446 | |
Scott Baker | b46a6ed | 2021-08-02 14:03:10 -0700 | [diff] [blame] | 447 | ROC Data Model Conventions and Requirements |
| 448 | ------------------------------------------- |
| 449 | |
| 450 | The MEGA-Patch described above will bring up a fully compliant sample data model. |
| 451 | However, it may be useful to bring up your own data model, customized to a different |
| 452 | site of sites. This subsection documents conventions and requirements for the Aether |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 453 | modeling within the ROC. |
Scott Baker | b46a6ed | 2021-08-02 14:03:10 -0700 | [diff] [blame] | 454 | |
| 455 | The ROC models must be configured with the following: |
| 456 | |
| 457 | * A default enterprise with the id `defaultent`. |
| 458 | * A default ip-domain with the id `defaultent-defaultip`. |
| 459 | * A default site with the id `defaultent-defaultsite`. |
| 460 | This site should be linked to the `defaultent` enterprise. |
| 461 | * A default device group with the id `defaultent-defaultsite-default`. |
| 462 | This device group should be linked to the `defaultent-defaultip` ip-domain |
| 463 | and the `defaultent-defaultsite` site. |
| 464 | |
| 465 | Each Enterprise Site must be configured with a default device group and that default |
| 466 | device group's name must end in the suffix `-default`. For example, `acme-chicago-default`. |
| 467 | |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 468 | Some exercises to get familiar |
| 469 | ------------------------------ |
| 470 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 471 | 1. Deploy the ROC and POST the mega-patch, go into the ``aether-roc-gui`` and click |
| 472 | through the VCS, DeviceGroup, and other objects to see that they were |
| 473 | created as expected. |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 474 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 475 | 2. Examine the log of the ``sdcore-adapter-v3`` container. It should be |
| 476 | attempting to push the mega-patch’s changes. If you don’t have a core |
| 477 | available, it may be failing the push, but you should see the attempts. |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 478 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 479 | 3. Change an object in the GUI. Watch the ``sdcore-adapter-v3`` log file and |
| 480 | see that the adapter attempts to push the change. |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 481 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 482 | 4. Try POSTing a change via the API. Observe the ``sdcore-adapter-v3`` log |
| 483 | file and see that the adapter attempts to push the change. |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 484 | |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 485 | 5. Deploy a 5G Aether-in-a-Box (See :doc:`Setting Up Aether-in-a-Box |
| 486 | <aiab>`), modify the mega-patch to specify the URL for the Aether-in-a-Box |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame] | 487 | ``webui`` container, POST the mega-patch, and observe that the changes were |
| 488 | correctly pushed via the ``sdcore-adapter-v3`` into the ``sd-core``’s |
| 489 | ``webui`` container (``webui`` container log will show configuration as it |
| 490 | is received) |
Scott Baker | fab7c9e | 2021-07-29 17:12:16 -0700 | [diff] [blame] | 491 | |
| 492 | .. |ROCGUI| image:: images/rocgui.png |
Sean Condon | eb95cd6 | 2021-08-04 19:44:18 +0100 | [diff] [blame] | 493 | :width: 945 |
| 494 | :alt: ROC GUI showing list of VCS |