| .. SPDX-FileCopyrightText: 2021 Open Networking Foundation <info@opennetworking.org> |
| .. SPDX-License-Identifier: Apache-2.0 |
| |
| .. _deployment_guide: |
| |
| Deployment Guide |
| ================ |
| |
| Switch Hardware Selection |
| ------------------------- |
| We have verified and therefore recommend using the switch model listed in :ref:`verified_switch`. |
| Other Stratum-enabled switches listed in :ref:`all_switch` should also work in theory |
| but more integration work may be required. |
| |
| To use the P4 UPF, you must use fabric switches based on the `Intel (formerly Barefoot) Tofino chipset |
| <https://www.intel.com/content/www/us/en/products/network-io/programmable-ethernet-switch/tofino-series.html>`_. |
| There are two variants of this switching chipset, with different resources and capabilities. |
| The **Dual Pipe** Tofino ASIC is less expensive, |
| while the **Quad Pipe** Tofino ASIC has more chip resources and a faster embedded system with more memory and storage. |
| |
| The P4 UPF and SD-Fabric features run within the constraints of the Dual Pipe |
| system for production deployments, but for development of features in P4, the |
| larger capacity of the Quad Pipe is desirable. |
| |
| These switches feature 32 QSFP+ ports capable of running in 100GbE, 40GbE, or |
| 4x 10GbE mode (using a split DAC or fiber cable) and have a 1GbE management |
| network interface. |
| |
| See also the :ref:`Rackmount of Equipment |
| <aether:edge_deployment/site_planning:rackmount of equipment>` for how the Fabric |
| switches should be rack-mounted to ensure proper airflow within a rack. |
| |
| Deployment Overview |
| ------------------- |
| SD-Fabric is released with Helm chart and container images. |
| We recommend using **Kubernetes** and **Helm** to deploy SD-Fabric. |
| Here's a list of high level steps required to deploy SD-Fabric: |
| |
| 1. **Provision switch** |
| |
| We first need to install operating system with Docker and Kubernetes on the bare-metal switches. |
| |
| 2. **Prepare switches as special Kubernetes nodes** |
| |
| Kubernetes ``label`` and ``taint`` are used to configure switches as special Kubernetes worker nodes. |
| This is to make sure we deploy Stratum (and only Stratum) on switches. |
| |
| 3. **Prepare ONOS network configuration** |
| |
| Network configuration defines properties such as switch pipeconf, subnet and VLAN. |
| |
| 4. **Prepare Stratum chassis configuration for each switch** |
| |
| Chassis config defines switch properties such as port speed and breakout. |
| |
| 5. **Install SD-Fabric** using Helm |
| |
| Finally, we are going to install SD-Fabric with the information we prepared in Step 1 to 5. |
| |
| Step 1: Access to the switch console |
| |
| There are two ways to access the switch console: |
| |
| * Access via Baseboard Management Controller(BMC) |
| * Access via console interface |
| |
| Access via the BMC |
| ^^^^^^^^^^^^^^^^^^ |
| |
| For some platforms, you can connect to an embedded system loaded with BMC system. |
| For example, Wedge100BF series is loaded with OpenBMC system that allows you to connect it via SSH |
| When the switch started, OpenBMC uses DHCP to initiate it's IP address. You may setting up static DHCP record on |
| your DHCP server or checkout the DHCP lease file from the DHCP server to get the IP address. |
| |
| .. code-block:: |
| |
| $ ssh root@[Open BMC IP] |
| |
| The default user and password is `root`` and `0penBmc`. |
| |
| In the OpenBMC system you can use Serial-over-LAN(sol) to access the main board. |
| |
| Access via console interface |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| If the platform doesn't support BMC, you can attach your laptop/PC to the switch with a |
| console cable and use the following command to access it: |
| |
| .. code-block:: |
| |
| $ screen /dev/[console device] [baud rate] |
| |
| The `console device` may be vary, it will usually be something like `ttyUSB...`, `tty.usb.....`. |
| Please checkout the console cable vendor for more information. |
| |
| The `baud rate` can also be vary, depends on the switch vendor. |
| Please checkout the switch user manual or contact vendor to get the information. |
| |
| Step 2: Provision Switches |
| -------------------------- |
| |
| We follow Open Network Install Environment (ONIE) way to install SONiC image to switch. |
| To work with the SD-Fabric environment, we have customized the SONiC image to support related features. |
| |
| You can download pre-compiled images from `Github Release page <https://github.com/stratum/sonic-base-image/releases>`_ |
| |
| .. note:: |
| If you're not familiar with ONIE/SONiC environment, please check `Getting Started <https://github.com/sonic-net/SONiC/wiki/Quick-Start>`_ to |
| see how to install the SONiC to an ONIE supported switch. |
| |
| |
| Once SONiC is started on the switch, you need to disable SONiC services before deploy Stratum on it. |
| Otherwise Stratum containers won't be started. |
| |
| .. code-block:: |
| |
| admin@sonic$ sudo systemctl stop sonic.target sonic-delayed.taget |
| admin@sonic$ sudo systemctl disable sonic.target sonic-delayed.taget |
| |
| Step 3: Configure switches as special Kubernetes nodes |
| ------------------------------------------------------ |
| |
| Once the Kubernetes is ready, the `Stratum <https://opennetworking.org/stratum/>`_ application will be deployed to the switch to manage it. |
| |
| Unlike server, switch has less CPU and memory resources and we should avoid |
| deploying unnecessary workloads into switch. |
| Besides, the Stratum application should only be deployed to all switches. |
| |
| To achieve the above goals, please apply the resources to your Kubernetes cluster. |
| |
| 1. Set up Label to all switch node, e.g ``node-role.kubernetes.io=switch`` |
| 2. Set up Taint with ``NoSchedule`` to all switch node, e.g ``node-role.kubernetes.io=switch:NoSchedule`` |
| 3. Properly configure the ``NodeSelector`` and ``Toleration`` when deploying Stratum via DaemonSet |
| |
| Example of a five nodes Kubernetes cluster, two switches and three servers |
| |
| .. code-block:: |
| |
| ╰─$ kubectl get node -o custom-columns=NAME:.metadata.name,TAINT:.spec.taints |
| NAME TAINT |
| compute1 <none> |
| compute2 <none> |
| compute3 <none> |
| leaf1 [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]] |
| leaf2 [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]] |
| ╰─$ kubectl get nodes -lnode-role.kubernetes.io=switch |
| NAME STATUS ROLES AGE VERSION |
| leaf1 Ready worker 27d v1.18.8 |
| leaf2 Ready worker 27d v1.18.8 |
| |
| Step 4: Prepare ONOS network configuration |
| ------------------------------------------ |
| See :ref:`onos_network_config` for instructions |
| |
| Step 5: Prepare Stratum chassis configuration |
| --------------------------------------------- |
| See See :ref:`stratum_chassis_config` for instructions |
| |
| .. _install_sd_fabric: |
| |
| Step 6: Install SD-Fabric with Helm |
| ----------------------------------- |
| |
| To install SD-Fabric into your Kubernetes cluster, follow instructions |
| described on the `SD-Fabric Helm Chart README <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/HEAD/sdfabric/README.md>`_ |