deployment.rst

.. SPDX-FileCopyrightText: 2021 Open Networking Foundation <info@opennetworking.org>
.. SPDX-License-Identifier: Apache-2.0

.. _deployment_guide:

Deployment Guide
================

Switch Hardware Selection
-------------------------
We have verified and therefore recommend using the switch model listed in :ref:`verified_switch`.
Other Stratum-enabled switches listed in :ref:`all_switch` should also work in theory
but more integration work may be required.

To use the P4 UPF, you must use fabric switches based on the `Intel (formerly Barefoot) Tofino chipset
<https://www.intel.com/content/www/us/en/products/network-io/programmable-ethernet-switch/tofino-series.html>`_.
There are two variants of this switching chipset, with different resources and capabilities.
The **Dual Pipe** Tofino ASIC is less expensive,
while the **Quad Pipe** Tofino ASIC has more chip resources and a faster embedded system with more memory and storage.

The P4 UPF and SD-Fabric features run within the constraints of the Dual Pipe
system for production deployments, but for development of features in P4, the
larger capacity of the Quad Pipe is desirable.

These switches feature 32 QSFP+ ports capable of running in 100GbE, 40GbE, or
4x 10GbE mode (using a split DAC or fiber cable) and have a 1GbE management
network interface.

See also the :ref:`Rackmount of Equipment
<aether:edge_deployment/site_planning:rackmount of equipment>` for how the Fabric
switches should be rack-mounted to ensure proper airflow within a rack.

Deployment Overview
-------------------
SD-Fabric is released with Helm chart and container images.
We recommend using **Kubernetes** and **Helm** to deploy SD-Fabric.
Here's a list of high level steps required to deploy SD-Fabric:

1. **Provision switch**

   We first need to install operating system with Docker and Kubernetes on the bare-metal switches.

2. **Prepare switches as special Kubernetes nodes**

   Kubernetes ``label`` and ``taint`` are used to configure switches as special Kubernetes worker nodes.
   This is to make sure we deploy Stratum (and only Stratum) on switches.

3. **Prepare ONOS network configuration**

   Network configuration defines properties such as switch pipeconf, subnet and VLAN.

4. **Prepare Stratum chassis configuration for each switch**

   Chassis config defines switch properties such as port speed and breakout.

5. **Install SD-Fabric** using Helm

   Finally, we are going to install SD-Fabric with the information we prepared in Step 1 to 5.

Step 1: Access to the switch console

There are two ways to access the switch console:

 * Access via Baseboard Management Controller(BMC)
 * Access via console interface

Access via the BMC
^^^^^^^^^^^^^^^^^^

For some platforms, you can connect to an embedded system loaded with BMC system.
For example, Wedge100BF series is loaded with OpenBMC system that allows you to connect it via SSH
When the switch started, OpenBMC uses DHCP to initiate it's IP address. You may setting up static DHCP record on
your DHCP server or checkout the DHCP lease file from the DHCP server to get the IP address.

.. code-block::

   $ ssh root@[Open BMC IP]

The default user and password is `root`` and `0penBmc`.

In the OpenBMC system you can use Serial-over-LAN(sol) to access the main board.

Access via console interface
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If the platform doesn't support BMC, you can attach your laptop/PC to the switch with a
console cable and use the following command to access it:

.. code-block::

   $ screen /dev/[console device] [baud rate]

The `console device` may be vary, it will usually be something like `ttyUSB...`, `tty.usb.....`.
Please checkout the console cable vendor for more information.

The `baud rate` can also be vary, depends on the switch vendor.
Please checkout the switch user manual or contact vendor to get the information.

Step 2: Provision Switches
--------------------------

We follow Open Network Install Environment (ONIE) way to install SONiC image to switch.
To work with the SD-Fabric environment, we have customized the SONiC image to support related features.

You can download pre-compiled images from `Github Release page <https://github.com/stratum/sonic-base-image/releases>`_

.. note::
    If you're not familiar with ONIE/SONiC environment, please check `Getting Started <https://github.com/sonic-net/SONiC/wiki/Quick-Start>`_ to
    see how to install the SONiC to an ONIE supported switch.


Once SONiC is started on the switch, you need to disable SONiC services before deploy Stratum on it.
Otherwise Stratum containers won't be started.

.. code-block::

   admin@sonic$ sudo systemctl stop sonic.target sonic-delayed.taget
   admin@sonic$ sudo systemctl disable sonic.target sonic-delayed.taget

Step 3: Configure switches as special Kubernetes nodes
------------------------------------------------------

Once the Kubernetes is ready, the `Stratum <https://opennetworking.org/stratum/>`_ application will be deployed to the switch to manage it.

Unlike server, switch has less CPU and memory resources and we should avoid
deploying unnecessary workloads into switch.
Besides, the Stratum application should only be deployed to all switches.

To achieve the above goals, please apply the resources to your Kubernetes cluster.

1. Set up Label to all switch node, e.g ``node-role.kubernetes.io=switch``
2. Set up Taint with ``NoSchedule`` to all switch node, e.g ``node-role.kubernetes.io=switch:NoSchedule``
3. Properly configure the ``NodeSelector`` and ``Toleration`` when deploying Stratum via DaemonSet

Example of a five nodes Kubernetes cluster, two switches and three servers

.. code-block::

     ╰─$ kubectl get node -o custom-columns=NAME:.metadata.name,TAINT:.spec.taints
     NAME       TAINT
     compute1   <none>
     compute2   <none>
     compute3   <none>
     leaf1      [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]]
     leaf2      [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]]
     ╰─$ kubectl get nodes -lnode-role.kubernetes.io=switch
     NAME    STATUS   ROLES    AGE   VERSION
     leaf1   Ready    worker   27d   v1.18.8
     leaf2   Ready    worker   27d   v1.18.8

Step 4: Prepare ONOS network configuration
------------------------------------------
   See :ref:`onos_network_config` for instructions

Step 5: Prepare Stratum chassis configuration
---------------------------------------------
   See See :ref:`stratum_chassis_config` for instructions

.. _install_sd_fabric:

Step 6: Install SD-Fabric with Helm
-----------------------------------

To install SD-Fabric into your Kubernetes cluster, follow instructions
described on the `SD-Fabric Helm Chart README <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/HEAD/sdfabric/README.md>`_