Hung-Wei Chiu | 77c969e | 2020-10-23 18:13:07 +0000 | [diff] [blame] | 1 | .. |
| 2 | SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org> |
| 3 | SPDX-License-Identifier: Apache-2.0 |
| 4 | |
Hung-Wei Chiu | 77c969e | 2020-10-23 18:13:07 +0000 | [diff] [blame] | 5 | Overview |
| 6 | ======== |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 7 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 8 | There are many ways to deploy Aether, depending on the requirements of the edge |
| 9 | site. The Reliability, Availability, and Serviceability (RAS) of each set of |
| 10 | equipment will differ depending on the characteristics of each edge. |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 11 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 12 | This document provides several hardware deployment options and explains the |
| 13 | differences between them. |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 14 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 15 | Deployment Options |
| 16 | ------------------ |
| 17 | |
| 18 | Development Environments |
| 19 | """""""""""""""""""""""" |
| 20 | |
| 21 | For users looking for a development or fully software-simulated environment, |
| 22 | there is ``Aether-in-a-Box (AiaB)`` - instructions for running this can be |
| 23 | found in the :doc:`Aether SD-Core Developer Guide </developer/sdcore>`. AiaB |
| 24 | is only suitable for testing and developing software, and can't connect to |
| 25 | physical hardware, but is a good choice for learning about the different |
| 26 | software components of Aether. |
| 27 | |
| 28 | Production Environments |
| 29 | """"""""""""""""""""""" |
| 30 | |
| 31 | Deploying Aether on hardware is required for both production deployments and |
| 32 | testing. |
| 33 | |
| 34 | This document currently describes the P4-based UPF implementation of Aether. |
| 35 | There is also a :doc:`Software-only BESS UPF |
| 36 | </edge_deployment/bess_upf_deployment>`, which can be used for deployments that |
| 37 | do not have P4 switching hardware. |
| 38 | |
| 39 | Before deploying Aether, a detailed plan including the network topology, |
| 40 | hardware, and all cabling needs to be created. |
| 41 | |
| 42 | For redundancy of workloads running in Kubernetes, at least 3 compute nodes |
| 43 | must be available. A single or pair of compute nodes can be used, but software |
| 44 | would need to be configured without High Availability (HA) enabled. |
Zack Williams | e8c3b2c | 2021-02-01 12:47:28 -0700 | [diff] [blame] | 45 | |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 46 | Network Cable Plan |
| 47 | ------------------ |
| 48 | |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 49 | If only a single P4 switch is used, the :doc:`Simple |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 50 | <trellis:supported-topology>` topology can be used, but provides no network |
| 51 | redundancy: |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 52 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 53 | .. image:: images/edge_single.svg |
| 54 | :alt: Single Switch |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 55 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 56 | If another switch is added, and a "Paired Leaves" (aka :doc:`Paired Switches |
| 57 | <trellis:supported-topology>`) topology is used, which can tolerate the loss of |
| 58 | a leaf switch and retain connections for all dual-homed devices. Single homed |
| 59 | devices on the failed leaf would need another form of HA, for example, |
| 60 | deploying multiple eNBs where some are connected to each leaf, and can provide |
| 61 | radio coverage.: |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 62 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 63 | .. image:: images/edge_paired_leaves.svg |
| 64 | :alt: Paired Leaves |
Zack Williams | e8c3b2c | 2021-02-01 12:47:28 -0700 | [diff] [blame] | 65 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 66 | For larger deployments, a 2x2 fabric can be configured (aka :doc:`Single-Stage |
| 67 | Leaf-Spine <trellis:supported-topology>`), which provide Spine redundancy, but |
| 68 | does not support dual-homing of devices. |
| 69 | |
| 70 | .. image:: images/edge_2x2.svg |
| 71 | :alt: 2x2 Fabric |
| 72 | |
| 73 | .. note:: |
| 74 | |
| 75 | Connections to the Fabric switches in these diagrams can be made up of two |
| 76 | cables, and configured to tolerate the failure or replacement of one cable or |
| 77 | NIC port. This is recommended, especially for links between switches. |
Zack Williams | e8c3b2c | 2021-02-01 12:47:28 -0700 | [diff] [blame] | 78 | |
| 79 | |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 80 | Required Hardware |
| 81 | ----------------- |
| 82 | |
| 83 | Fabric Switches |
| 84 | """"""""""""""" |
| 85 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 86 | Aether recommends the use of fabric switches based on the Intel (was Barefoot) |
| 87 | Tofino chipset, which are capable of running the P4 UPF. There are multiple |
| 88 | variants of this switching chipset, with different resources and capabilities. |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 89 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 90 | Currently supported P4 switches include: |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 91 | |
| 92 | * `EdgeCore Wedge100BF-32X |
| 93 | <https://www.edge-core.com/productsInfo.php?cls=1&cls2=180&cls3=181&id=335>`_ |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 94 | - a "Dual Pipe" chipset |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 95 | |
| 96 | * `EdgeCore Wedge100BF-32QS |
| 97 | <https://www.edge-core.com/productsInfo.php?cls=1&cls2=180&cls3=181&id=770>`_ |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 98 | - a "Quad Pipe" chipset |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 99 | |
| 100 | Compute Servers |
| 101 | """"""""""""""" |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 102 | These servers run Kubernetes, Aether connectivity apps, and edge applications. |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 103 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 104 | Minimum hardware specifications: |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 105 | |
| 106 | * AMD64 (aka x86-64) architecture |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 107 | * 8 CPU Cores |
| 108 | * 32GB of RAM |
| 109 | * 250 GB of storage (SSD preferred) |
| 110 | * 2x 40GbE or 100GbE Ethernet connections to P4 switches |
| 111 | * 1x 1GbE management network port |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 112 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 113 | Optional: |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 114 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 115 | * Lights out management support, with either shared or separate NIC |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 116 | |
| 117 | Management Server |
| 118 | """"""""""""""""" |
| 119 | |
| 120 | One management server is required, which must have at least two 1GbE network |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 121 | ports, and runs a variety of network services to bootstrap and support the |
| 122 | edge. |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 123 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 124 | In current Aether deployments, the Management Server also functions as a router |
| 125 | and VPN gateway back to Aether Central. |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 126 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 127 | Minimum hardware specifications: |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 128 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 129 | * AMD64 (aka x86-64) architecture |
| 130 | * 4 CPU cores |
| 131 | * 8GB of RAM |
| 132 | * 120GB of storage (SSD preferred) |
| 133 | * 2x 1GbE Network interfaces (one for WAN, one to the management switch) |
| 134 | |
| 135 | Optional: |
| 136 | |
| 137 | * 10GbE or 40GbE network card to connect to fabric switch |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 138 | |
| 139 | Management Switch |
| 140 | """"""""""""""""" |
| 141 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 142 | A managed L2/L3 management switch is required to provide connectivity within |
| 143 | the cluster for bootstrapping equipment. It is configured with multiple VLANs |
| 144 | to separate the management plane, fabric, and the out-of-band and lights out |
| 145 | management connections on the equipment. |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 146 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 147 | Minimum requirements: |
| 148 | |
| 149 | * 16x 1GbE Copper ports |
| 150 | |
| 151 | * 2x 10GbE SFP+ or 40GbE QSFP interfaces (only required if management server |
| 152 | does not have a network card with these ports) |
| 153 | |
| 154 | * Managed via SSH or web interface |
| 155 | |
| 156 | * Capable supporting VLANs on each port, with both tagged and untagged traffic |
| 157 | sharing a port. |
| 158 | |
| 159 | Optional: |
| 160 | |
| 161 | * PoE+ support, which can power eNB and monitoring hardware. |
| 162 | |
| 163 | |
| 164 | eNB Radio |
| 165 | """"""""" |
| 166 | |
| 167 | The LTE eNB used in most deployments is the `Sercomm P27-SCE4255W Indoor CBRS |
| 168 | Small |
| 169 | Cell <https://www.sercomm.com/contpage.aspx?langid=1&type=prod3&L1id=2&L2id=1&L3id=107&Prodid=751>`_. |
| 170 | |
| 171 | This supports PoE+ power on the WAN port, which provides deployment |
| 172 | flexibility. |
| 173 | |
| 174 | Testing hardware |
| 175 | ---------------- |
| 176 | |
| 177 | The following hardware is used to test the network and determine uptime of |
| 178 | edges. It's currently required, to properly validate that an edge site is |
| 179 | functioning properly. |
Zack Williams | 9026f53 | 2020-11-30 11:34:32 -0700 | [diff] [blame] | 180 | |
Hyunsun Moon | 2b62676 | 2021-03-26 16:21:14 -0700 | [diff] [blame] | 181 | Monitoring Raspberry Pi and CBRS dongle |
| 182 | """"""""""""""""""""""""""""""""""""""" |
| 183 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 184 | One pair of Raspberry Pi and CBRS band supported LTE dongle is required to |
| 185 | monitor the connectivity service at the edge. |
Hyunsun Moon | 2b62676 | 2021-03-26 16:21:14 -0700 | [diff] [blame] | 186 | |
| 187 | The Raspberry Pi model used in Pronto is a `Raspberry Pi 4 Model B/2GB |
| 188 | <https://www.pishop.us/product/raspberry-pi-4-model-b-2gb/>`_ |
| 189 | |
| 190 | Which is configured with: |
| 191 | |
| 192 | * HighPi Raspberry Pi case for P4 |
| 193 | * USB-C Power Supply |
| 194 | * MicroSD Card with Raspbian - 16GB |
| 195 | |
Zack Williams | 1ae109e | 2021-07-27 11:17:04 -0700 | [diff] [blame^] | 196 | One LTE dongle model supported in Aether is the `Sercomm Adventure Wingle |
Hyunsun Moon | 2b62676 | 2021-03-26 16:21:14 -0700 | [diff] [blame] | 197 | <https://www.sercomm.com/contpage.aspx?langid=1&type=prod3&L1id=2&L2id=2&L3id=110&Prodid=767>`_. |