| .. vim: syntax=rst |
| |
| Aether-in-a-Box on Hardware Radios |
| ================================== |
| |
| This document describes how to set up an Aether-in-a-Box (AiaB) with |
| a Sercomm eNodeB and connect real devices (e.g., 4G phones). It assumes that |
| you are already familiar with running AiaB with emulated eNodeB/UE. AiaB on Hardware |
| Radios is suitable for laboratory experiments and proof-of-concept deployments. |
| Its goals are to demonstrate how to configure a real eNodeB to work with Aether, |
| and to provide an easy-to-install environment where Aether's features can be |
| explored with real devices. To create this setup you will need the following equipment: |
| |
| * Server for running AiaB (SD-CORE / UPF / ROC) |
| |
| * Haswell CPU family or newer |
| * At least 4 CPUs and 12GB RAM |
| * Internet connection |
| |
| * 4G or 5G small cell eNodeB |
| |
| * Example: :ref:`Sercomm CBRS LTE small cell eNodeB <edge_deployment/overview:eNB Radio>` |
| |
| * SIM card writer and blank SIM cards |
| |
| **IMPORTANT**: AiaB is for simple deployment scenarios and so makes some simplifying assumptions: |
| |
| * AiaB assumes either 4G or 5G SD-CORE is deployed. Running both 4G and 5G SD-CORE simultaneously in AiaB |
| is currently not supported. However, running both 4G and 5G SD-CORE simultaneously is supported by Aether. |
| * Performance and scalability are not goals. AiaB does not support I/O acceleration (e.g., with SR-IOV). However, |
| performance and scalability are goals of the Aether project. |
| * AiaB assumes that the server and the eNodeB are connected to the **same LAN** and |
| share the **same IP subnet**; in other words they can reach each other in a single hop and |
| there is no IP router between them. This simplifies communication between the eNodeB and the UPF, |
| which is running inside a container and has a private IP address that is not necessarily routable |
| on the local network. However, this is not a requirement for all Aether deployments. |
| * AiaB also assumes that the AiaB server's network is configured |
| using *systemd-networkd*, which is the default for Ubuntu, and copies some files into `/etc/systemd/network`; |
| the reason for this is to enable persistence of AiaB's networking configuration across server reboots. |
| This configuration method is specific to AiaB. |
| |
| Preparation |
| ----------- |
| |
| Create SIM cards by following the instructions for your SIM card writer. |
| Of course you are free to use any values for IMSI, etc. that you choose, |
| but these are the values that will work with the rest of the configuration |
| in this document: |
| |
| * IMSI: each one is unique, matching pattern ``315010*********`` (15 digits) |
| * OPc: ``69d5c2eb2e2e624750541d3bbc692ba5`` |
| * Transport Key: ``000102030405060708090a0b0c0d0e0f`` |
| |
| If you choose different values for your SIM cards, you will need to |
| modify subsequent configuration steps appropriately. |
| |
| Insert the SIM cards in devices that you wish to be able to connect to the Aether network. |
| |
| Server setup |
| ------------ |
| |
| The server will run Aether-in-a-Box. The eNodeB will connect to the server over the local network. |
| Perform these steps to prepare the server for the AiaB install: |
| |
| * Connect the server to the local network |
| * Perform a clean install of Ubuntu 18.04 or Ubuntu 20.04 on the server |
| * Verify that systemd-networkd is being used to configure networking |
| (e.g., run ``systemctl status systemd-networkd.service``) |
| * Set up password-less sudo for the user that will install Aether-in-a-Box |
| |
| After the steps above have been completed, install Aether-in-a-Box as follows:: |
| |
| sudo apt install git make |
| git clone "https://gerrit.opencord.org/aether-in-a-box" |
| cd aether-in-a-box |
| |
| Next, modify the file *sd-core-4g-values.yaml*. Under ``subscribers``, |
| add an IMSI range for the SIM cards you created, with the Transport Key |
| and OPc values you used earlier. For example, the following will add |
| IMSIs between 315010999912301 and 315010999912303:: |
| |
| subscribers: |
| - ueId-start: 315010999912301 |
| ueId-end: 315010999912303 |
| plmnId: 315010 |
| opc: 69d5c2eb2e2e624750541d3bbc692ba5 |
| key: 000102030405060708090a0b0c0d0e0f |
| sequenceNumber: 135 |
| |
| Determine which is the interface that has L3 connectivity to the |
| eNodeB -- this will be ``DATA_IFACE`` in the configuration later. If |
| the eNodeB will also be connected to the local network, then this is just the |
| server's primary interface. If the eNodeB will be connected via an |
| isolated L2/L3 network segment, then ``DATA_IFACE`` refers to the server |
| interface on that network. Remember this interface for later. |
| |
| Option 1: Configure Aether with ROC |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| The Aether ROC provides a GUI and API for dynamically configuring |
| Aether. If you don't wish to use the ROC to configure AiaB, you |
| can skip to the next section. |
| |
| Install AiaB as follows (specifying ``DATA_IFACE`` from above):: |
| |
| ENABLE_OAISIM=false DATA_IFACE=<iface> CHARTS=latest make roc-4g-models 4g-core |
| |
| Next, use the ROC to add information about your SIM cards. |
| The ROC GUI is available at `http://<server-ip>:31194`. |
| |
| Choose ``Configuration > Site`` from the drop-down at top right and edit |
| the ``AiaB site``. Change the following values and click ``Update``: |
| |
| * MCC: 315 |
| * MNC: 010 |
| |
| Choose ``Sim Cards`` from the drop-down at top right. Edit the |
| existing entries to reflect the SIM cards you are adding to devices |
| by replacing their IMSI values. Click ``Update`` after each edit. |
| If you want to connect more than two devices, consult the :ref:`ROC |
| documentation <operations/subscriber:Configure Connectivity Service for a new Device>`. |
| |
| Finally, click the Basket icon at top right and click the ``Commit`` button. |
| |
| Now jump to the `Verifying the AiaB installation`_ section. |
| |
| Option 2: Configure Aether without ROC |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| It is possible to configure Aether without the ROC, |
| using static YAML files and the SimApp service. If you have already |
| installed the ROC, you should skip this section. |
| |
| Edit *sd-core-4g-values.yaml*. Change ``mcc`` and ``mnc`` as follows:: |
| |
| plmn: |
| mcc: "315" |
| mnc: "010" |
| |
| Also add the IMSIs of your devices under ``imsis``, for example:: |
| |
| device-groups: |
| - name: "4g-oaisim-user" |
| imsis: |
| - "315010999912301" |
| - "315010999912302" |
| - "315010999912303" |
| |
| Install AiaB as follows (specifying ``DATA_IFACE`` from above):: |
| |
| ENABLE_OAISIM=false DATA_IFACE=<iface> CHARTS=latest make 4g-core |
| |
| Verifying the AiaB installation |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| Installing AiaB will take about 20 minutes with a fast Internet |
| connection. If you see any errors / timeouts, try running the ``make`` |
| command again. The build will finish with a message: |
| “Your MME IP address is… ” This is just the IP address assigned to |
| the ``DATA_IFACE``. Remember this for the eNodeB setup. |
| |
| When the install is complete, check that the 4G SD-CORE is running |
| as follows:: |
| |
| $ kubectl -n omec get pod |
| NAME READY STATUS RESTARTS AGE |
| cassandra-0 1/1 Running 0 7m27s |
| config4g-0 1/1 Running 0 7m27s |
| hss-0 1/1 Running 0 7m27s |
| mme-0 4/4 Running 0 7m27s |
| pcrf-0 1/1 Running 0 7m27s |
| simapp-65dc44b9d-stx6q 1/1 Running 0 7m27s |
| spgwc-0 2/2 Running 0 7m27s |
| upf-0 5/5 Running 0 7m27s |
| |
| You should see all pods in Running status. |
| |
| If you have installed the ROC, check that all its pods are running |
| as follows:: |
| |
| $ kubectl -n aether-roc get pod |
| NAME READY STATUS RESTARTS AGE |
| aether-roc-api-78cc548bb9-7vjs2 1/1 Running 0 4m16s |
| aether-roc-gui-v2-6d674fd446-tttb5 1/1 Running 0 4m16s |
| aether-roc-umbrella-grafana-74f8489c8f-s9p45 2/2 Running 0 4m16s |
| aether-roc-websocket-855d64549b-44fnc 1/1 Running 0 4m16s |
| onos-cli-5d448ff6c4-stq5t 1/1 Running 0 4m16s |
| onos-config-7f4df96b88-vtp5s 6/6 Running 0 4m16s |
| onos-consensus-store-0 1/1 Running 0 4m15s |
| onos-topo-585c7c8976-6jq7b 3/3 Running 0 4m16s |
| sdcore-adapter-v2-5646d455b9-2d6zl 1/1 Running 0 4m15s |
| |
| You should see all pods in Running status. |
| |
| The UPF pod connects to the ``DATA_IFACE`` specified above using macvlan networks called |
| ``core`` and ``access``. Next, check that these have been successfully created, e.g. using |
| ``ifconfig``:: |
| |
| $ ifconfig core |
| core: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 |
| inet 192.168.250.1 netmask 255.255.255.0 broadcast 192.168.250.255 |
| ether 16:9d:c1:0f:19:3a txqueuelen 1000 (Ethernet) |
| RX packets 513797 bytes 48400525 (48.4 MB) |
| RX errors 0 dropped 0 overruns 0 frame 0 |
| TX packets 102996 bytes 26530538 (26.5 MB) |
| TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 |
| |
| $ ifconfig access |
| access: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 |
| inet 192.168.252.1 netmask 255.255.255.0 broadcast 192.168.252.255 |
| ether 7a:9f:38:c0:18:15 txqueuelen 1000 (Ethernet) |
| RX packets 558162 bytes 64064410 (64.0 MB) |
| RX errors 0 dropped 0 overruns 0 frame 0 |
| TX packets 99553 bytes 16646682 (16.6 MB) |
| TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 |
| |
| Understanding AiaB networking |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| Why does AiaB create the ``core`` and ``access`` interfaces? These are necessary to enable |
| the UPF to exchange packets with the eNodeB (access) and Internet (core); they correspond to |
| the last two network interfaces below inside the UPF's `bessd` container:: |
| |
| $ kubectl -n omec exec -ti upf-0 bessd -- ip addr |
| 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 |
| link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 |
| inet 127.0.0.1/8 scope host lo |
| valid_lft forever preferred_lft forever |
| inet6 ::1/128 scope host |
| valid_lft forever preferred_lft forever |
| 3: eth0@if30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default |
| link/ether 8a:e2:64:10:4e:be brd ff:ff:ff:ff:ff:ff link-netnsid 0 |
| inet 192.168.84.19/32 scope global eth0 |
| valid_lft forever preferred_lft forever |
| inet6 fe80::88e2:64ff:fe10:4ebe/64 scope link |
| valid_lft forever preferred_lft forever |
| 4: access@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default |
| link/ether 82:b4:ea:00:50:3e brd ff:ff:ff:ff:ff:ff link-netnsid 0 |
| inet 192.168.252.3/24 brd 192.168.252.255 scope global access |
| valid_lft forever preferred_lft forever |
| inet6 fe80::80b4:eaff:fe00:503e/64 scope link |
| valid_lft forever preferred_lft forever |
| 5: core@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default |
| link/ether 4e:ac:69:31:a3:88 brd ff:ff:ff:ff:ff:ff link-netnsid 0 |
| inet 192.168.250.3/24 brd 192.168.250.255 scope global core |
| valid_lft forever preferred_lft forever |
| inet6 fe80::4cac:69ff:fe31:a388/64 scope link |
| valid_lft forever preferred_lft forever |
| |
| In other words, there are interfaces named ``access`` and ``core`` **both inside and outside** the UPF. All four |
| are MACVLAN interfaces |
| bridged with DATA_IFACE. There are two subnets on this bridge: the two ``access`` interfaces are on 192.168.252.0/24 |
| and the two ``core`` interfaces are on 192.168.250.0/24. It is helpful to think of two links, called |
| ``access`` and ``core``, connecting the AiaB host and UPF. AiaB sets up IP routes on the AiaB host and inside the UPF |
| to forward packets into and out of the UPF as explained below. |
| |
| The ``access`` interface **inside the UPF** has an IP address of 192.168.252.3; this is the destination IP address |
| of GTP-encapsulated data plane packets from the eNodeB. In order for these packets to actually find their way |
| to the UPF, they must arrive on the DATA_IFACE interface and then be forwarded on the ``access`` interface |
| **outside the UPF**. |
| The next section describes how to configure a static route on the eNodeB in order to send the GTP packets to |
| DATA_IFACE. Forwarding the packets to the ``access`` interface is done by the following kernel route on the |
| AiaB host (which should be present if your AiaB installation was successful):: |
| |
| $ route -n | grep "Iface\|access" |
| Destination Gateway Genmask Flags Metric Ref Use Iface |
| 192.168.252.0 0.0.0.0 255.255.255.0 U 0 0 0 access |
| |
| The high-level behavior of the UPF is to forward packets between its ``access`` to ``core`` interfaces, while |
| at the same time removing/adding GTP encapsulation on the ``access`` side. Upstream packets |
| arriving on the ``access`` side from a UE have their GTP headers removed and the raw IP packets are |
| forwarded to the ``core`` interface. The routes inside the UPF's `bessd` container will look something |
| like this:: |
| |
| $ kubectl -n omec exec -ti upf-0 -c bessd -- ip route |
| default via 169.254.1.1 dev eth0 |
| default via 192.168.250.1 dev core metric 110 |
| 128.105.144.0/22 via 192.168.252.1 dev access |
| 128.105.145.141 via 169.254.1.1 dev eth0 |
| 169.254.1.1 dev eth0 scope link |
| 192.168.250.0/24 dev core proto kernel scope link src 192.168.250.3 |
| 192.168.252.0/24 dev access proto kernel scope link src 192.168.252.3 |
| |
| The default route via 192.168.250.1 is directing upstream packets to the Internet via the ``core`` interface, |
| with a next hop of the ``core`` interface **outside the UPF**. |
| These packets undergo source NAT in the kernel (also configured by AiaB) and are sent to the IP destination |
| in the packet. The return (downstream) packets undergo reverse NAT and now have a destination IP address of the UE. |
| They are forwarded by the kernel to the ``core`` interface by these rules on the AiaB host:: |
| |
| $ route -n | grep "Iface\|core" |
| Destination Gateway Genmask Flags Metric Ref Use Iface |
| 172.250.0.0 192.168.250.3 255.255.0.0 UG 0 0 0 core |
| 192.168.250.0 0.0.0.0 255.255.255.0 U 0 0 0 core |
| |
| The first rule above matches packets to the UEs (on 172.250.0.0/16 subnet). The next hop for these |
| packets is the ``core`` IP address **inside the UPF**. The second rule says that next hop address is |
| reachable on the ``core`` interface **outside the UPF**. As a result the downstream packets arrive in the |
| UPF where they |
| are GTP-encapsulated with the IP address of the eNodeB. Inside the UPF these packets will match a route |
| like this one (see above; 128.105.144.0/22 in this case is the DATA_IFACE subnet):: |
| |
| 128.105.144.0/22 via 192.168.252.1 dev access |
| |
| These packets are forwarded to the ``access`` interface **outside the UPF** and out DATA_IFACE to the eNodeB. |
| Recall that AiaB assumes that the eNodeB is on the same subnet as DATA_IFACE, so in this case it also has an |
| IP address in the 128.105.144.0/22 range. |
| |
| Manual Sercomm eNodeB setup |
| --------------------------- |
| |
| The instructions in this section describe a basic configuration of |
| the eNodeB. For a more comprehensive guide to |
| eNodeB configuration see `eNB Installation <https://docs.aetherproject.org/master/edge_deployment/enb_installation.html>`_. |
| |
| The Sercomm eNodeB has two Ethernet ports: WAN and LAN. We will use |
| the LAN port for configuration of the eNodeB and the WAN port for |
| normal operation. Connect the eNodeB WAN port to the local network. |
| |
| Connect the eNodeB LAN port to a free Ethernet port on a Linux machine |
| (say, a laptop) that will be used for the initial configuration of |
| the eNodeB. On that machine run ``dhclient`` on the interface corresponding |
| to the Ethernet port, for example:: |
| |
| sudo dhclient eth1 |
| |
| The interface should receive an IP address from the Sercomm eNodeB on |
| the 11.11.11.0/24 subnet. Check this using ``ifconfig``:: |
| |
| $ ifconfig eth1 |
| eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 |
| inet 11.11.11.100 netmask 255.255.255.0 broadcast 11.11.11.255 |
| inet6 fe80::2e0:4cff:fe68:2f76 prefixlen 64 scopeid 0x20<link> |
| ether 00:e0:4c:68:2f:76 txqueuelen 1000 (Ethernet) |
| RX packets 264652 bytes 216094312 (216.0 MB) |
| RX errors 0 dropped 0 overruns 0 frame 0 |
| TX packets 183978 bytes 36528580 (36.5 MB) |
| TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 |
| |
| The eNodeB IP address should be 11.11.11.188 on that link. Verify |
| this using ping:: |
| |
| $ ping 11.11.11.188 |
| |
| To access the Sercomm eNodeB admin page, you can run a Web browser on |
| the laptop and direct it to `https://11.11.11.188`. Login to the admin |
| interface as user: ``sc_femto`` and password: ``scHt3pp``. |
| |
| Click the ``NetWork set`` menu at the top. In the first section |
| ``IP Address``, set *Connect type: DHCP* (assuming this is supported on |
| your local network, otherwise set up a static IP address). |
| Click the blue ``Save`` button at the bottom. |
| |
| Next, click the ``Manage`` menu at the top and then click the |
| ``LTE Basic Setting`` tab. Change these parameters and click ``Save``: |
| |
| * Carrier Number: 2 |
| * Carrier Aggregation: Unset |
| * BandWidth: 20 |
| * FrequencyBand: 48,48 |
| * EARFCN: 55440,55640 |
| * CellIdentity: 2,1 |
| * PCI: 100,101 |
| * TxPower: 20 |
| * Tunnel Type: IPv4 |
| * MME IP Address: <MME IP address from AiaB installation> |
| * PLMNID: 315010 |
| * TAC: 1 |
| * Sync Source: FREE_RUNNING |
| * Enable CWMP: Unset |
| |
| Click the ``SAS Configuration`` tab. In the ``Location Configuration`` |
| section, enter these values and click “Save”: |
| |
| * Location: Indoor |
| * Location Source: Manual |
| * Latitude: 0 |
| * Longitude: 0 |
| * Elevation: -18000 |
| |
| Next we need to add a static route to the UPF address, 192.168.252.3, |
| on the eNodeB. Click on ``TR098`` menu and then click on ``IP`` tab. |
| Scroll down to ``Device.Routing.Router.`` section. Click ``View List``. |
| Add this info on the blank line: |
| |
| * Enable: Set |
| * StaticRoute: Set |
| * DestIPAddress: 192.168.252.0 |
| * DestSubnetMask: 255.255.255.0 |
| * GatewayIPAddress: <Use MME IP address from AiaB installation> |
| * Interface: Device.IP.Interface.1. |
| |
| Then click the ``Add`` button at the far right. |
| |
| Finally click the ``FAPService`` menu and then go to the ``FAPControl`` |
| tab. Check the box next to ``AdminState`` in the first section and |
| click ``Save``. |
| |
| After these changes are made, reboot the eNodeB by clicking the red |
| ``power button`` square at top right and selecting ``Reboot``. When the |
| eNodeB comes back up, it should have an IP address on the network |
| (via the WAN port), and the admin page should now be available on |
| `https://<endoeb-ip>`. |
| |
| Test connectivity from the eNodeB to the MME and the UPF running on |
| the server as follows. Login to the eNodeB admin interface, click |
| the “Manage” menu at the top, and click the ``IP Diagnose`` tab. Under |
| ``Ping and Traceroute``, select ``ping``, and then type the following IP |
| addresses into the box to the right and click ``Run``: |
| |
| * <MME IP address from AiaB installation> |
| * 192.168.251.1 |
| * 192.168.252.3 |
| |
| If all of these are working, then you are ready to try to connect |
| devices to the network. |
| |
| Automated Sercomm eNodeB Setup (experimental) |
| --------------------------------------------- |
| |
| The eNodeB service is used to provide the automatic configuration of the eNodeB device. |
| It can be installed by the following command:: |
| |
| make enodebd |
| |
| The eNodeBD service will listen on port *31005* and you need to configure the management url of eNodeB device |
| to make eNodeB connect to the eNodeBD service. |
| By default, the eNodeBD service will configure the eNodeB with the pre-defined configuration, but you can also add |
| the customized configuration per serial number in the ``magma_config/serial_number/`` directory. For the example |
| configuration, please duplicate ``acs_common.yaml`` and rename it as the ``{serial_number}.yaml``. |
| |
| The automated setup via eNodeBD won't configure the static route to UPF service, please follow |
| the UPF route setup step describes in **Manual Sercomm eNodeB setup** to add the static route. |
| |
| Configuration and Validation |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| To configure the Sercomm eNodeB be managed by eNodeBD service, access the TR098 configuration page:: |
| |
| https://<enodeb_ip_address>/TR098_DevFeature.htm |
| |
| The configuring path is: TR098 > MgmtServer > Device.ManagementServer.URL |
| |
| 1. Check the EnableCWMP option to enable the management function. |
| 2. Fill the eNodeBD URL into Device.ManagementServer.URL field. |
| |
| After you fill in the AiaB IP with eNodeBD service port as URL into the field, you need to restart the device. |
| And you will be able to see the eNodeBD log by the following command (the pod name may be different):: |
| |
| kubectl -n aether-apps logs aether-enodebd-b46897745-fj5qj -f |
| |
| Also, you can install the monitoring dashboard to view the eNodeB status on Grafana. |
| |
| Connecting Devices |
| ------------------ |
| |
| Documenting how to configure different types of devices to work |
| with Aether is work-in-progress, but here are some basic guidelines. |
| |
| The values of IMSI, OPc, and Transport Key you have configured on your SIM cards |
| must be entered into the ``subscribers`` block under ``omec-sub-provision`` in the |
| ``sd-core-4g-values.yaml`` file. If you are not using the ROC, the IMSIs must also be |
| added under ``device-groups``, and the relevant device group added under ``network-slices``. |
| If you are using the ROC, then your devices must be configured there and the associated |
| device group added to a slice. In either case it is necessary to configure the basic info |
| under ``subscribers``. |
| |
| Be aware that not all phones support the CBRS frequency bands. AiaB is known to work |
| with recent iPhones (11 and greater) and Google Pixel phones (4 and up). CBRS may also be |
| supported by recent phones from Samsung, LG Electronics and Motorola Mobility, but these have |
| not been tested with AiaB. If you successfully test a phone on AiaB, please post details on |
| Slack so we can add it to the list. |
| |
| The APN to configure on your phone is ``internet``. |
| |
| Enable Monitoring |
| ----------------- |
| |
| To easily monitor your AiaB deployment, you can run the following command to |
| deploy the Prometheus/Grafana monitoring stack on your AiaB server:: |
| |
| make monitoring-4g |
| |
| After this step, Grafana is available at `http://<server-ip>:30950`. You will see a number of system dashboards |
| for monitoring Kubernetes, as well as a simple AiaB dashboard that enables inspection of the local Aether state. |
| |
| .. figure:: images/4g-aiab-grafana-dashboard.png |
| :align: center |
| :width: 100 % |
| |
| *4G Grafana AiaB Dashboard* |
| |
| The dashboard shows whether the eNodeB is connected to the core, how many active UEs there are, and the |
| uplink (Tx Bitrate) and downlink (Rx Bitrate) throughput at the UPF. |
| |
| To create a new dashboard for Aether, you can login to Grafana as user `admin` with password `prom-operator`. |
| You can also modify an existing dashboard by making a copy of the dashboard and editing that. Note that any |
| dashboard changes will be lost if Grafana is restarted; if you want to make your dashboard more permanent, you can |
| save its JSON to a file, add the file to directory `aether-in-a-box/resources/4g-monitoring`, and edit |
| `kustomization.yaml` in that directory to add the file to the ConfigMapGenerator block. After these |
| steps, re-running `make monitoring-4g` should install the dashboard in a ConfigMap and Grafana should |
| automatically load it. |
| |
| Troubleshooting |
| --------------- |
| |
| **NOTE: Running both 4G and 5G SD-CORE simultaneously in AiaB is currently not supported.** |
| |
| "make" fails immediately |
| ^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| AiaB connects macvlan networks to ``DATA_IFACE`` so that the UPF can communicate on the network. |
| To do this it assumes that the *systemd-networkd* service is installed and running, ``DATA_IFACE`` |
| is under its control, and the systemd-networkd configuration file for ``DATA_IFACE`` ends with |
| ``<DATA_IFACE>.network``, where ``<DATA_IFACE>`` stands for the actual interface name. It |
| tries to find this configuration file by looking in the standard paths. If it fails you'll see |
| a message like:: |
| |
| FATAL: Could not find systemd-networkd config for interface foobar, exiting now! |
| make: *** [Makefile:112: /users/acb/aether-in-a-box//build/milestones/interface-check] Error 1 |
| |
| In this case, you can specify a ``DATA_IFACE_PATH=<path to the config file>`` argument to ``make`` |
| so that AiaB can find the systemd-networkd configuration file for ``DATA_IFACE``. It's also possible |
| that your system does not use systemd-networkd to configure network interfaces (more likely if you |
| are running in a VM), in which case AiaB is currently not able to install in your setup. You |
| can check that systemd-networkd is installed and running as follows:: |
| |
| $ systemctl status systemd-networkd.service |
| ● systemd-networkd.service - Network Service |
| Loaded: loaded (/lib/systemd/system/systemd-networkd.service; disabled; vendor preset: enabled) |
| Active: active (running) since Tue 2022-07-12 13:42:18 CDT; 2h 26min ago |
| TriggeredBy: ● systemd-networkd.socket |
| Docs: man:systemd-networkd.service(8) |
| Main PID: 13777 (systemd-network) |
| Status: "Processing requests..." |
| Tasks: 1 (limit: 193212) |
| Memory: 6.4M |
| CGroup: /system.slice/systemd-networkd.service |
| └─13777 /lib/systemd/systemd-networkd |
| |
| Data plane is not working |
| ^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| The first step is to read `Understanding AiaB networking`_, which gives a high level picture |
| of the AiaB data plane and how the pieces fit together. In order to debug the problem you will |
| need to figure out where data plane packets from the eNodeB are dropped. One way to do this is to |
| run ``tcpdump`` on (1) DATA_IFACE to ensure that the data plane packets are arriving, (2) the |
| ``access`` interface to see that they make it to the UPF, and (3) the ``core`` to check that they |
| are forwarded upstream. |
| |
| If the upstream packets don't make it to DATA_IFACE, you probably need to add the static route |
| on the eNodeB so packets to the UPF have a next hop of DATA_IFACE. You can see these upstream |
| packets by running:: |
| |
| tcpdump -i <data-iface> -n udp port 2152 |
| |
| If they don't make it to ``access`` you should check that the kernel routing table is forwarding |
| a packet with destination 192.158.252.3 to the ``access`` interface. You can see them by running:: |
| |
| tcpdump -i access -n udp port 2152 |
| |
| If they don't make it to ``core`` then they are being dropped by the UPF for some reason. This |
| may be a configuration issue with the state loaded in the ROC / SD-CORE -- the UPF is being told |
| to discard these packets. You should check that the device's IMSI is part of a slice and that |
| the slice's policy settings allow traffic to that destination. You can view them via the following:: |
| |
| tcpdump -i core -n net 172.250.0.0/16 |
| |
| That command will capture all packets to/from the UE subnet. |
| |
| If you cannot figure out the issue, see `Getting Help`_. |
| |
| Restarting the AiaB Server |
| -------------------------- |
| |
| AiaB should come up in a mostly working state if the AiaB server is rebooted. If any pods are |
| stuck in an Error or CrashLoopBackoff state they can be restarted using ``kubectl delete pod``. |
| It might also be necessary to power cycle the Sercomm eNodeB in order to get it to reconnect to |
| the SD-CORE. |
| |
| Getting Help |
| ------------ |
| |
| Please introduce yourself and post your questions to the `#aether-dev` channel on the ONF Community Slack. |
| Details about how to join this channel can be found on the `ONF Wiki <https://wiki.opennetworking.org/display/COM/Aether>`_. |
| In your introduction please state your institution and position, and describe why you are interested in Aether |
| and what is your end goal. |
| |
| If you need help debugging your setup, please give as much detail as possible about |
| your environment: the OS version you have installed, are you running on bare metal or in a VM, |
| how much CPU and memory does your server have, are you installing behind a proxy, and so on. Also list the steps |
| you have performed so far, and post any error messages you have received. These details will aid the community |
| to understand where you are and how to help you make progress. |