Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 1 | .. vim: syntax=rst |
| 2 | |
| 3 | Setting Up Aether-in-a-Box |
| 4 | ========================== |
| 5 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 6 | Aether-in-a-Box (AiaB) provides an easy way to deploy Aether's SD-CORE and ROC |
| 7 | components, and then run basic tests to validate the installation. |
| 8 | This guide describes the steps to set up AiaB. |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 9 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 10 | AiaB can be set up with a 4G or 5G SD-CORE. In either case, SD-CORE configuration |
| 11 | can be done with or without the ROC. The ROC |
| 12 | provides an interactive GUI for examining and changing the configuration, and is used to |
| 13 | manage the production Aether; it can be deployed to test the integration between |
| 14 | ROC and SD-CORE. If the ROC is not deployed, a simple tool called SimApp |
| 15 | is used to configure the required state in SD-CORE for testing core functionality. |
| 16 | |
| 17 | Helm charts are the primary method of installing the SD-CORE and ROC resources. |
| 18 | AiaB offers a great deal of flexibility regarding which Helm chart versions to install: |
| 19 | |
| 20 | * Local definitions of charts (for testing Helm chart changes) |
| 21 | * Latest published charts (for deploying a development version of Aether) |
| 22 | * Specified versions of charts (for deploying a specific Aether release) |
| 23 | |
| 24 | AiaB can be run on a bare metal machine or VM. System prerequisites: |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 25 | |
| 26 | * Ubuntu 18.04 |
| 27 | * Kernel 4.15 or later |
| 28 | * Haswell CPU or newer |
Andy Bavier | 7254767 | 2022-02-03 14:45:16 -0700 | [diff] [blame^] | 29 | * At least 4 CPUs and 12GB RAM |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 30 | |
| 31 | Clone Repositories |
| 32 | ------------------ |
| 33 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 34 | To initialize the AiaB environment, first clone the following repository |
| 35 | using your Gerrit ID:: |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 36 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 37 | cd ~ |
| 38 | git clone "ssh://<username>@gerrit.opencord.org:29418/aether-in-a-box" |
| 39 | |
| 40 | If you are going to install AiaB using published Helm charts, you can proceed to the |
| 41 | next section. |
| 42 | |
| 43 | If you wish to install SD-CORE from local Helm charts, clone these additional repositories:: |
| 44 | |
| 45 | mkdir -p ~/cord |
| 46 | cd ~/cord |
Andy Bavier | 7ff674f | 2021-11-05 12:47:17 -0700 | [diff] [blame] | 47 | git clone "ssh://<username>@gerrit.opencord.org:29418/sdcore-helm-charts" |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 48 | git clone "ssh://<username>@gerrit.opencord.org:29418/aether-helm-charts" |
| 49 | |
Andy Bavier | 112a33c | 2021-11-11 11:13:49 -0700 | [diff] [blame] | 50 | If you wish to install the ROC from local Helm charts, clone this:: |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 51 | |
| 52 | mkdir -p ~/cord |
| 53 | cd ~/cord |
Andy Bavier | e86261e | 2021-09-21 11:05:18 -0700 | [diff] [blame] | 54 | git clone "ssh://<username>@gerrit.opencord.org:29418/roc-helm-charts" |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 55 | |
| 56 | Now change to *~/aether-in-a-box* directory. |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 57 | |
| 58 | Set up Authentication Tokens |
| 59 | ---------------------------- |
| 60 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 61 | Edit the file *configs/authentication*. |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 62 | |
| 63 | Fill out REGISTRY_USERNAME and REGISTRY_CLI_SECRET as follows: |
| 64 | |
| 65 | * Log into the `Aether Harbor Registry <https://registry.aetherproject.org>`_ using your Crowd credentials |
| 66 | * Select *User Profile* from the drop-down menu in the upper right corner |
| 67 | * For REGISTRY_USERNAME, use the *Username* in your profile |
| 68 | * Copy the *CLI secret* to the clipboard and paste to REGISTRY_CLI_SECRET |
| 69 | |
Andy Bavier | 7ff674f | 2021-11-05 12:47:17 -0700 | [diff] [blame] | 70 | Also fill out REPO_USERNAME and REPO_PASSWORD with the information needed to authenticate |
| 71 | with Aether's Helm chart repositories. |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 72 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 73 | If you have already set up AiaB but you used incorrect credentials, first clean up AiaB as described |
Andy Bavier | 7254767 | 2022-02-03 14:45:16 -0700 | [diff] [blame^] | 74 | in the `Cleanup`_ section, and also run these commands:: |
| 75 | |
| 76 | kubectl -n omec delete secret aether.registry |
| 77 | rm /tmp/build/milestones/helm-ready |
| 78 | |
| 79 | Then edit *configs/authentication* and re-build AiaB. |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 80 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 81 | Installing the ROC |
| 82 | ------------------ |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 83 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 84 | Note that you must install the ROC *before* installing SD-CORE. |
| 85 | If you are not using the ROC to configure SD-CORE, you can skip this step. |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 86 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 87 | First choose whether you will install the 4G or 5G SD-CORE. To install the ROC to |
| 88 | configure the 4G SD-CORE:: |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 89 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 90 | make roc-4g-models |
| 91 | |
| 92 | To install the ROC to configure the 5G SD-CORE:: |
| 93 | |
| 94 | make roc-5g-models |
| 95 | |
| 96 | By default the above commands install the ROC from the local charts in the Git repos cloned |
| 97 | earlier. In order to install the ROC using the latest published charts, add *CHARTS=latest* |
| 98 | to the command, e.g.,:: |
| 99 | |
| 100 | CHARTS=latest make roc-4g-models |
| 101 | |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 102 | To install the Aether 1.6 release, add *CHARTS=release-1.6*:: |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 103 | |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 104 | CHARTS=release-1.6 make roc-4g-models |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 105 | |
| 106 | Start the 4G SD-CORE |
| 107 | -------------------- |
| 108 | |
| 109 | If you are installing the 5G SD-CORE, you can skip this step. |
| 110 | |
| 111 | To deploy the 4G SD-CORE and run a simple ping test:: |
| 112 | |
| 113 | make test |
| 114 | |
| 115 | By default the above commands install the 4G SD-CORE from the local charts in the Git repos cloned |
| 116 | earlier. In order to install the SD-CORE using the latest published charts, add *CHARTS=latest* |
| 117 | to the command, e.g.,:: |
| 118 | |
| 119 | CHARTS=latest make test |
| 120 | |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 121 | To install the Aether 1.6 release, add *CHARTS=release-1.6*:: |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 122 | |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 123 | CHARTS=release-1.6 make test |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 124 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 125 | Start the 5G SD-CORE |
| 126 | -------------------- |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 127 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 128 | If you have already installed the 4G SD-CORE, you must skip this step. Only one version of |
| 129 | the SD-CORE can be installed at a time. |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 130 | |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 131 | To deploy the 5G SD-CORE and run a test with gNBSim that performs Registration + UE-initiated |
| 132 | PDU Session Establishment + sends User Data packets:: |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 133 | |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 134 | make 5g-test |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 135 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 136 | By default the above commands install the 5G SD-CORE from the local charts in the Git repos cloned |
| 137 | earlier. In order to install the SD-CORE using the latest published charts, add *CHARTS=latest* |
| 138 | to the command, e.g.,:: |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 139 | |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 140 | CHARTS=latest make 5g-test |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 141 | |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 142 | To install the Aether 1.6 release, add *CHARTS=release-1.6*:: |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 143 | |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 144 | CHARTS=release-1.6 make 5g-test |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 145 | |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 146 | To change the behavior of the test run by gNBSim, change the contents of *gnb.conf* |
| 147 | in *ransim-values.yaml*. Consult the |
| 148 | `gNBSim documentation <https://docs.sd-core.opennetworking.org/master/developer/gnbsim.html>`_ for more information. |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 149 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 150 | Cleanup |
| 151 | ------- |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 152 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 153 | The first time you build AiaB, it takes a while because it sets up the Kubernetes cluster. |
| 154 | Subsequent builds will be much faster if you follow these steps to clean up the Helm charts without |
| 155 | destroying the Kubernetes cluster. |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 156 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 157 | * Clean up the 4G SD-CORE: *make reset-test* |
Andy Bavier | 7254767 | 2022-02-03 14:45:16 -0700 | [diff] [blame^] | 158 | * Reset the 4G UE / eNB in order to re-run the 4G test: *make reset-ue* |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 159 | * Clean up the 5G SD-CORE: *make reset-5g-test* |
| 160 | * Clean up the ROC: *make roc-clean* |
| 161 | |
| 162 | It's normal for the above commands to take a minute or two to complete. |
| 163 | |
| 164 | As an example, suppose that you want to test the 4G SD-CORE with the ROC, and then the 5G SD-CORE |
| 165 | with the ROC. You could run these commands:: |
| 166 | |
| 167 | CHARTS=latest make roc-4g-models # Install ROC with 4G configuration |
| 168 | CHARTS=latest make test # Install 4G SD-CORE and run ping test |
| 169 | make reset-test |
| 170 | make roc-clean |
| 171 | CHARTS=latest make roc-5g-models # Install ROC with 5G configuration |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 172 | CHARTS=latest make 5g-test # Install 5G SD-CORE and run gNB Sim test |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 173 | make reset-5g-test |
| 174 | make roc-clean |
| 175 | |
| 176 | Developer Loop |
| 177 | -------------- |
| 178 | |
| 179 | Suppose you wish to test a new build of a 5G SD-CORE services. You can deploy custom images |
| 180 | by editing `~/aether-in-a-box/5g-core-values.yaml`, for example:: |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 181 | |
| 182 | images: |
| 183 | tags: |
| 184 | webui: registry.aetherproject.org/omecproject/5gc-webui:onf-release3.0.5-roc-935305f |
| 185 | pullPolicy: IfNotPresent |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 186 | |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 187 | To upgrade a running 5G SD-CORE with the new image, or to deploy the 5G SD-CORE with the image:: |
Andy Bavier | f73c3d2 | 2021-08-30 10:29:06 -0700 | [diff] [blame] | 188 | |
Andy Bavier | 1d63176 | 2022-01-10 15:47:51 -0800 | [diff] [blame] | 189 | make 5g-test |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 190 | |
| 191 | Troubleshooting / Known Issues |
| 192 | ------------------------------ |
| 193 | |
| 194 | If you suspect a problem, first verify that all pods are in Running state:: |
| 195 | |
| 196 | kubectl -n omec get pods |
| 197 | kubectl -n aether-roc get pods |
| 198 | |
Andy Bavier | 7254767 | 2022-02-03 14:45:16 -0700 | [diff] [blame^] | 199 | Pods in ImagePullBackOff State |
| 200 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 201 | If the pods are stuck in ImagePullBackOff state, then it's likely an issue with credentials. To verify this, |
| 202 | run *kubectl describe* on a pod in that state, for example:: |
| 203 | |
| 204 | kubectl -n omec describe pod gnbsim-0 |
| 205 | |
| 206 | Look in the *Events* section for more information about why the image pull failed. If you see *unauthorized to |
| 207 | access repository* then it's probably a credentials issue; see `Set up Authentication Tokens`_ above. |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 208 | |
| 209 | 4G Test Fails |
| 210 | ^^^^^^^^^^^^^ |
| 211 | Occasionally *make test* (for 4G) fails for unknown reasons; this is true regardless of which Helm charts are used. |
Andy Bavier | 7254767 | 2022-02-03 14:45:16 -0700 | [diff] [blame^] | 212 | If this happens, first try recreating the simulated UE / eNB and re-running the test as follows:: |
| 213 | |
| 214 | make reset-ue |
| 215 | make test |
| 216 | |
| 217 | If that does not work, try cleaning up AiaB as described above and re-building it. |
| 218 | |
| 219 | If *make test* fails consistently, check whether the configuration has been pushed to the SD-CORE:: |
Andy Bavier | 840e0b4 | 2021-09-16 17:10:23 -0700 | [diff] [blame] | 220 | |
| 221 | kubectl -n omec logs config4g-0 | grep "Successfully" |
| 222 | |
| 223 | You should see that a device group and slice has been pushed:: |
| 224 | |
| 225 | [INFO][WebUI][CONFIG] Successfully posted message for device group 4g-oaisim-user to main config thread |
| 226 | [INFO][WebUI][CONFIG] Successfully posted message for slice default to main config thread |
| 227 | |
| 228 | Then tail the *config4g-0* log and make sure that the configuration has been successfully pushed to all |
| 229 | SD-CORE components. |