llp | b353464 | 2023-08-02 09:23:52 -0700 | [diff] [blame] | 1 | Closer Look |
| 2 | --------------- |
| 3 | |
Larry Peterson | 4453fb8 | 2023-10-19 13:26:59 -0700 | [diff] [blame] | 4 | Before tearing down your Quick Start deployment, there are three |
llp | b353464 | 2023-08-02 09:23:52 -0700 | [diff] [blame] | 5 | additional steps you can take to watch Aether in action. The first is |
| 6 | to bring up the Aether Management Plane (AMP), which includes |
Larry Peterson | 4453fb8 | 2023-10-19 13:26:59 -0700 | [diff] [blame] | 7 | Dashboards showing different aspects of Aether's runtime behavior. |
| 8 | The second is to inspect the logs written by the various microservices. |
| 9 | The third is to enable packet capture, and then run an analysis |
| 10 | tool to trace the flow of packets into and out of SD-Core. |
llp | b353464 | 2023-08-02 09:23:52 -0700 | [diff] [blame] | 11 | |
| 12 | |
| 13 | Install AMP |
| 14 | ~~~~~~~~~~~~~~~ |
| 15 | |
| 16 | The Aether Management Platform (AMP) is implemented by two Kubernetes |
| 17 | applications: *Runtime Operational Control (ROC)* and a *Monitoring |
| 18 | Service*.\ [#]_ AMP can be deployed on the same cluster as SD-Core by |
| 19 | executing the following Make target: |
| 20 | |
| 21 | .. code-block:: |
| 22 | |
| 23 | $ make aether-amp-install |
| 24 | |
| 25 | Once complete, ``kubectl`` will show the ``aether-roc`` and |
| 26 | ``cattle-monitoring-system`` namespaces running in support of these |
| 27 | two services, respectively, plus new ``atomix`` pods in the |
| 28 | ``kube-system`` namespace. Atomix is the scalable key-value store |
| 29 | that keeps the ROC data model persistent. |
| 30 | |
Larry Peterson | def1b67 | 2023-08-07 14:06:24 -0700 | [diff] [blame] | 31 | .. [#] Note that what the implementation calls ROC, `Chapter 6 |
| 32 | <https://5g.systemsapproach.org/cloud.html>`__ refers to |
| 33 | generically as *Service Orchestration*. |
llp | b353464 | 2023-08-02 09:23:52 -0700 | [diff] [blame] | 34 | |
| 35 | You can access the dashboards for the two subsystems, |
| 36 | respectively, at |
| 37 | |
| 38 | .. code-block:: |
| 39 | |
| 40 | http://<server_ip>:31194 |
| 41 | http://<server_ip>:30950 |
| 42 | |
| 43 | The programmatic API underlying the Control Dashboard, which was |
Larry Peterson | def1b67 | 2023-08-07 14:06:24 -0700 | [diff] [blame] | 44 | introduced in `Section 6.4 |
| 45 | <https://5g.systemsapproach.org/cloud.html#connectivity-api>`__, can |
| 46 | be accessed at ``http://10.76.28.113:31194/aether-roc-api/`` in our |
| 47 | example deployment (where Aether runs on host ``10.76.28.113``). |
llp | b353464 | 2023-08-02 09:23:52 -0700 | [diff] [blame] | 48 | |
| 49 | There is much more to say about the ROC and the Aether API, which we |
| 50 | return to in the section on Runtime Control. For now, we suggest you |
| 51 | simply peruse the Control Dashboard by starting with the dropdown menu |
| 52 | in the upper right corner. For example, selecting `Devices` will show |
Larry Peterson | c177f98 | 2023-08-14 16:54:51 -0700 | [diff] [blame] | 53 | the set of UEs registered with Aether, similar to the screenshot in |
| 54 | :numref:`Figure %s <fig-roc>`. In an operational setting, these values |
| 55 | would be entered into the ROC through either the GUI or the underlying |
| 56 | API. For the Quick Start scenario we're limiting ourselves to in this |
| 57 | section, these values are loaded from |
llp | b353464 | 2023-08-02 09:23:52 -0700 | [diff] [blame] | 58 | ``deps/amp/5g-roc/templates/roc-5g-models.json``. |
| 59 | |
Larry Peterson | c177f98 | 2023-08-14 16:54:51 -0700 | [diff] [blame] | 60 | .. _fig-roc: |
| 61 | .. figure:: figures/ROC-Dashboard.png |
| 62 | :width: 700px |
| 63 | :align: center |
| 64 | |
| 65 | Screenshot of the ROC dashboard, showing known *Devices*. The |
| 66 | dropdown menu on the right lists other available pages. |
| 67 | |
llp | b353464 | 2023-08-02 09:23:52 -0700 | [diff] [blame] | 68 | Turning to the Monitoring Dashboard, you will initially see |
| 69 | Kubernetes-related performance stats. Select the *5G Dashboard* option |
Larry Peterson | c177f98 | 2023-08-14 16:54:51 -0700 | [diff] [blame] | 70 | to display information reported by SD-Core. Similar to :numref:`Figure |
| 71 | %s <fig-monitor>`, the page shows an active (green) UPF, and once you |
| 72 | rerun the RAN simulator (gNBsim), some number of active base stations |
| 73 | and connected devices. The bottom panel shows the UPF throughput, |
| 74 | which due to gNBsim's focus on stressing the control plane, typically |
| 75 | shows only minimal activity. |
| 76 | |
| 77 | .. _fig-monitor: |
| 78 | .. figure:: figures/5G-Dashboard.png |
| 79 | :width: 700px |
| 80 | :align: center |
| 81 | |
| 82 | Screenshot of the monitoring subsystem's 5G dashboard. |
llp | b353464 | 2023-08-02 09:23:52 -0700 | [diff] [blame] | 83 | |
| 84 | When you are done experimenting with AMP, type the following |
| 85 | to tear it down: |
| 86 | |
| 87 | .. code-block:: |
| 88 | |
| 89 | $ make aether-amp-uninstall |
| 90 | |
Larry Peterson | cebca8d | 2023-10-26 07:24:27 -0700 | [diff] [blame^] | 91 | View Logs |
Larry Peterson | 4453fb8 | 2023-10-19 13:26:59 -0700 | [diff] [blame] | 92 | ~~~~~~~~~~~~~~~~ |
| 93 | |
| 94 | You've already seen the log file generated by gNBsim for each |
| 95 | emulation run, but you can also inspect the logs generated by |
| 96 | individual microservices that implement Aether. Doing so is certainly |
| 97 | helpful when debugging a failure, but it can also be an aid in |
| 98 | learning how each microservice works. For example, the following |
| 99 | command outputs the log for the ``bessd`` container, one of five |
| 100 | containers running as part of the ``upf-0`` pod: |
| 101 | |
| 102 | .. code-block:: |
| 103 | |
| 104 | $ kubectl logs -n omec -p upf-0 bessd |
| 105 | |
| 106 | While ``kubectl`` works just fine for tasks like this, you may also |
| 107 | want to install `k9s <https://k9scli.io/>`__\ , a terminal-based UI |
| 108 | that provides a convenient alternative for interacting with Kubernetes. |
| 109 | Once installed, the following command brings up the UI for the OMEC |
| 110 | namespace that implements SD-Core. |
| 111 | |
| 112 | .. code-block:: |
| 113 | |
| 114 | $ k9s -n omec |
| 115 | |
| 116 | :numref:`Figure %s <fig-k9s>` shows an example k9s display, where you |
| 117 | can scroll up and down, and then invoke one of the listed |
| 118 | commands—such as ``<l>`` (display log) or ``<s>`` (open a shell)—for |
| 119 | the selected pod. |
| 120 | |
| 121 | .. _fig-k9s: |
| 122 | .. figure:: figures/k9s.png |
| 123 | :width: 700px |
| 124 | :align: center |
| 125 | |
| 126 | Screenshot of k9s's UI for the OMEC namespace, with the AMF pod |
| 127 | currently selected. |
| 128 | |
| 129 | |
llp | b353464 | 2023-08-02 09:23:52 -0700 | [diff] [blame] | 130 | Run Ksniff and Wireshark |
| 131 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 132 | |
| 133 | In addition to the trace output generated by the simulator, a good way |
| 134 | to understand the inner working of Aether is to use `Ksniff |
| 135 | <https://github.com/eldadru/ksniff>`__ (a Kubernetes plugin) to |
| 136 | capture packets and display their headers as they flow into and out of |
| 137 | the microservices that implement Aether. Output from Ksniff can then |
| 138 | be fed into `Wireshark <https://www.wireshark.org/>`__. |
| 139 | |
| 140 | To install the Ksniff plugin on the server running Aether, you need to |
| 141 | first install ``krew``, the Kubernetes plugin manager. Instructions on |
| 142 | doing that can be found `online |
| 143 | <https://krew.sigs.k8s.io/docs/user-guide/setup/install/>`__. Once |
| 144 | that's done, you can install Ksniff by typing: |
| 145 | |
| 146 | .. code-block:: |
| 147 | |
| 148 | $ kubectl krew install sniff |
| 149 | |
| 150 | You can then run Ksniff in the context of a specific Kubernetes pod by |
| 151 | specifying their namespace and instance names, and then redirecting |
| 152 | the output to Wireshark. If you don't have a desktop environment on |
| 153 | your Aether server, you can either view the output using a simpler |
| 154 | packet analyzer, such as `tshark |
| 155 | <https://www.wireshark.org/docs/man-pages/tshark.html>`__, or by |
| 156 | redirecting the PCAP output in a file and transfer it a desktop |
| 157 | machine for viewing in Wireshark. |
| 158 | |
| 159 | For example, the following captures and displays traffic into and out |
| 160 | of the AMF, where you need to substitute the name of the AMP pod |
Larry Peterson | 5d6b3b3 | 2023-09-05 15:11:55 -0700 | [diff] [blame] | 161 | you learned from ``kubectl`` in place of ``amf-5887bbf6c5-pc9g2``. |
llp | b353464 | 2023-08-02 09:23:52 -0700 | [diff] [blame] | 162 | |
| 163 | .. code-block:: |
| 164 | |
| 165 | $ kubectl sniff -n omec amf-5887bbf6c5-pc9g2 -o - | tshark -r - |
| 166 | |
| 167 | Of course, you'll also need to restart the RAN emulator to generate |
| 168 | workload for this tool to capture. |