Hung-Wei Chiu | e49ef3e | 2021-10-04 14:13:36 -0700 | [diff] [blame] | 1 | .. |
| 2 | SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org> |
| 3 | SPDX-License-Identifier: Apache-2.0 |
| 4 | |
Charles Chan | caebcf3 | 2021-09-20 22:17:52 -0700 | [diff] [blame] | 5 | Deployment Guide |
| 6 | ================ |
Hung-Wei Chiu | e49ef3e | 2021-10-04 14:13:36 -0700 | [diff] [blame] | 7 | |
| 8 | |
| 9 | Provision Switches |
| 10 | ------------------ |
| 11 | |
| 12 | We follow Open Network Install Environment(ONIE) way to install Open Network Linux (ONL) image to switch. |
| 13 | To work with the SD-Fabric environment, we have customized the ONL image to support related packages and dependencies. |
| 14 | |
| 15 | Image source file can be found on ONF repository `opennetworkinglab/OpenNetworkLinux <https://github.com/opennetworkinglab/OpenNetworkLinux>`_. |
| 16 | You can also download pre-compiled artifacts from `Github Release page <https://github.com/opennetworkinglab/OpenNetworkLinux/releases>`_ |
| 17 | |
| 18 | |
| 19 | .. note:: |
| 20 | If you're not familiar with ONIE/ONL environment, please check `Getting Started <https://github.com/opencomputeproject/OpenNetworkLinux/blob/master/docs/GettingStarted.md>`_ to |
| 21 | see how to install the ONL image to an ONIE supported switch. |
| 22 | |
| 23 | Below is an example about how to install the ONL image. |
| 24 | |
| 25 | 1. Prepare a server which is accessible by the switch and then download the |
| 26 | pre-compiled installer from the release page. |
| 27 | |
| 28 | .. code-block:: |
| 29 | |
| 30 | wget https://github.com/opennetworkinglab/OpenNetworkLinux/releases/download/v1.4.3/ONL-onf-ONLPv2_ONL-OS_2021-07-16.2159-5195444_AMD64_INSTALLED_INSTALLER |
| 31 | python -m http.server 8080 |
| 32 | |
| 33 | 2. Reboot the switch to enter ONIE installation mode |
| 34 | |
| 35 | .. note:: |
| 36 | Please access the switch via BMC or serial console to keep connection during the installation. |
| 37 | |
| 38 | |
| 39 | .. code-block:: |
| 40 | |
| 41 | onl-onie-boot-mode rescue; reboot |
| 42 | |
| 43 | 3. Install ONL installer |
| 44 | |
| 45 | .. code-block:: |
| 46 | |
| 47 | onie-nos-install http://$SERVER_IP:8080/ONL-onf-ONLPv2_ONL-OS_2021-07-16.2159-5195444_AMD64_INSTALLED_INSTALLER |
| 48 | |
| 49 | 4. Setup switch IP and hostname after the installation. |
| 50 | |
| 51 | |
| 52 | Kubernetes Environment |
| 53 | ---------------------- |
| 54 | |
| 55 | Our `ONL <https://github.com/opennetworkinglab/OpenNetworkLinux>`_ version |
| 56 | includes all packages required by running the Kubernetes on top of it. |
| 57 | Once the Kubernetes is ready, the `Stratum <https://opennetworking.org/stratum/>`_ application will be deployed to the switch to manage it. |
| 58 | |
| 59 | Unlike server, switch has less CPU and memory resources and we should avoid |
| 60 | deploying unnecessary workloads into switch. |
| 61 | Besides, the Stratum application should only be deployed to all switches. |
| 62 | |
| 63 | To achieve the above goals, please apply the resources to your Kubernetes cluster. |
| 64 | |
| 65 | 1. Set up Label to all switch node, e.g ``node-role.kubernetes.io=switch`` |
| 66 | 2. Set up Taint with ``NoSchedule`` to all switch node, e.g ``node-role.kubernetes.io=switch:NoSchedule`` |
| 67 | 3. Properly configure the ``NodeSelector`` and ``Toleration`` when deploying Stratum via DaemonSet |
| 68 | |
| 69 | Example of a five nodes Kubernetes cluster, two switches and three servers |
| 70 | |
| 71 | .. code-block:: |
| 72 | |
| 73 | ╰─$ kubectl get node -o custom-columns=NAME:.metadata.name,TAINT:.spec.taints |
| 74 | NAME TAINT |
| 75 | compute1 <none> |
| 76 | compute2 <none> |
| 77 | compute3 <none> |
| 78 | leaf1 [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]] |
| 79 | leaf2 [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]] |
Hung-Wei Chiu | b0232a1 | 2021-10-11 11:17:54 -0700 | [diff] [blame^] | 80 | ╰─$ kubectl get nodes -lnode-role.kubernetes.io=switch |
Hung-Wei Chiu | e49ef3e | 2021-10-04 14:13:36 -0700 | [diff] [blame] | 81 | NAME STATUS ROLES AGE VERSION |
| 82 | leaf1 Ready worker 27d v1.18.8 |
| 83 | leaf2 Ready worker 27d v1.18.8 |
| 84 | |
| 85 | |
Hung-Wei Chiu | b0232a1 | 2021-10-11 11:17:54 -0700 | [diff] [blame^] | 86 | Please follow the :ref:`Install SD-Fabric section <install_sd_fabric>` to |
| 87 | check how to use Taint and NodeSelector during installation. |
| 88 | |
| 89 | |
Hung-Wei Chiu | e49ef3e | 2021-10-04 14:13:36 -0700 | [diff] [blame] | 90 | # TODO |
| 91 | Build Image |
| 92 | ----------- |
| 93 | |
| 94 | Fetch Images From Private Registry |
| 95 | ---------------------------------- |
| 96 | |
| 97 | Container images can be download from ONF self-hosted container registry but you have to gain the access token first. |
| 98 | |
| 99 | 1. Login to `Aether Harbor Registry <https://registry.aetherproject.org/harbor/sign-in?redirect_url=%2Fharbor%2Fprojects>`_ using your ONF Crowd credential, |
| 100 | 2. Select ``User Profile`` drop-down menu in the upper-right corner |
| 101 | 3. Generate the CLI secret and it's the secret token you have to access the container registry via CLI tool. |
| 102 | 4. Login to the container registry with your username and access token |
| 103 | by ``docker login command`` to ensure you can access it. |
| 104 | |
| 105 | .. code-block:: |
| 106 | |
Hung-Wei Chiu | b0232a1 | 2021-10-11 11:17:54 -0700 | [diff] [blame^] | 107 | ╰─$ docker login registry.aetherproject.org --username hwchiu |
Hung-Wei Chiu | e49ef3e | 2021-10-04 14:13:36 -0700 | [diff] [blame] | 108 | Password: |
| 109 | Login Succeeded |
| 110 | |
| 111 | |
Hung-Wei Chiu | b0232a1 | 2021-10-11 11:17:54 -0700 | [diff] [blame^] | 112 | Please follow the :ref:`Install SD-Fabric section <install_sd_fabric>` to |
| 113 | check how to use Taint and NodeSelector during installation. |
Hung-Wei Chiu | e49ef3e | 2021-10-04 14:13:36 -0700 | [diff] [blame] | 114 | |
Hung-Wei Chiu | b0232a1 | 2021-10-11 11:17:54 -0700 | [diff] [blame^] | 115 | .. _install_sd_fabric: |
Hung-Wei Chiu | e49ef3e | 2021-10-04 14:13:36 -0700 | [diff] [blame] | 116 | |
Hung-Wei Chiu | e49ef3e | 2021-10-04 14:13:36 -0700 | [diff] [blame] | 117 | Install SD-Fabric |
| 118 | ----------------- |
Hung-Wei Chiu | b0232a1 | 2021-10-11 11:17:54 -0700 | [diff] [blame^] | 119 | |
| 120 | To install SD-Fabric into your Kubernetes cluster, follow instructions |
| 121 | described on the `SD-Fabric Helm Chart Repository <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/HEAD/sdfabric/README.md>`_ |
| 122 | |
| 123 | A workflow to install the SD-Fabric should be look like |
| 124 | |
| 125 | 1. Clone the Helm Charts files from the `SD-Fabric Helm Chart Repository <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/HEAD/sdfabric/README.md>`_ |
| 126 | 2. Customize the value file based on your environment |
| 127 | 3. Deploy it via Helm command |
| 128 | |
| 129 | Below is an example how to install SD-Fabric |
| 130 | |
| 131 | .. note:: |
| 132 | Please ensure you have installed switch into Kubernetes cluster and have configured them with proper taints and label, |
| 133 | and have the permission to fetch the container image from the `Aether Harbor Registry <https://registry.aetherproject.org/harbor/sign-in?redirect_url=%2Fharbor%2Fprojects>`_. |
| 134 | |
| 135 | |
| 136 | 1. Use the git command to clone the `SD-Fabric Helm Chart Repository <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/HEAD/sdfabric/README.md>`_ |
| 137 | 2. Update dependencies |
| 138 | |
| 139 | .. code-block:: |
| 140 | |
| 141 | ╰─$ cd sdfabric |
| 142 | ╰─$ helm dep update 146 ↵ |
| 143 | Downloading onos-classic from repo https://charts.onosproject.org |
| 144 | Downloading stratum from repo https://charts.stratumproject.org/ |
| 145 | Deleting outdated charts |
| 146 | |
| 147 | 3. Prepare your value file, you can modify an existing ``values.yaml`` or use a standalone |
| 148 | value file, use the later approach in this example. |
| 149 | |
| 150 | .. code-block:: |
| 151 | |
| 152 | ╰─$ cat myvaules.yaml |
| 153 | image: |
| 154 | credential: |
| 155 | username: my_username |
| 156 | password: my_access_token |
| 157 | |
| 158 | onos-classic: |
| 159 | config: |
| 160 | netcfg: > |
| 161 | { |
| 162 | "devices": { |
| 163 | "device:leaf1": { |
| 164 | "segmentrouting": { |
| 165 | "ipv4NodeSid": 101, |
| 166 | "ipv4Loopback": "10.11.22.33", |
| 167 | "routerMac": "aa:bb:cc:dd:ee:ff", |
| 168 | "pairDeviceId" : "device:leaf2", |
| 169 | "pairLocalPort" : 260, |
| 170 | "isEdgeRouter": true, |
| 171 | "adjacencySids": [] |
| 172 | } |
| 173 | } |
| 174 | } |
| 175 | } |
| 176 | |
| 177 | |
| 178 | stratum: |
| 179 | nodeSelector: |
| 180 | node-role.kubernetes.io: switch |
| 181 | |
| 182 | tolerations: |
| 183 | - effect: NoSchedule |
| 184 | value: switch |
| 185 | key: node-role.kubernetes.io |
| 186 | |
| 187 | Please pay attention to the following fields. |
| 188 | |
| 189 | * Configure your ``registry username and password`` on image.credential |
| 190 | * Configure the ``label`` on stratum.nodeSelector |
| 191 | * Configure the ``taint`` on stratum.tolerations |
| 192 | * Configure the ``network configuration`` on onos-classic.config.netcfg |
| 193 | |
| 194 | 4. Verify your Configuration to ensure no YAML/Helm syntax errors |
| 195 | |
| 196 | .. code-block:: |
| 197 | |
| 198 | ╰─$ helm template -f myvaules.yaml . |
| 199 | |
| 200 | 5. Using the helm command to install it. (``helm version is above v3.2``) |
| 201 | Following command will install the release `sdfabric` to namespace `sdfabric`. |
| 202 | |
| 203 | .. code-block:: |
| 204 | |
| 205 | ╰─$ helm install -n sdfabric --create-namespace -f myvaules.yaml sdfabric . |
| 206 | NAME: sdfabric |
| 207 | LAST DEPLOYED: Mon Oct 11 11:12:59 2021 |
| 208 | NAMESPACE: sdfabric |
| 209 | STATUS: deployed |
| 210 | REVISION: 1 |
| 211 | TEST SUITE: None |
| 212 | ╰─$ helm -n sdfabric ls |
| 213 | NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION |
| 214 | sdfabric sdfabric 1 2021-10-11 11:12:59.178789 -0700 PDT deployed sdfabric-1.0.1 |