deployment.rst - sdfabric-docs - Gitiles

 ..
    SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
    SPDX-License-Identifier: Apache-2.0

 Deployment Guide
 ================


 Provision Switches
 ------------------

 We follow Open Network Install Environment(ONIE) way to install Open Network Linux (ONL) image to switch.
 To work with the SD-Fabric environment, we have customized the ONL image to support related packages and dependencies.

 Image source file can be found on ONF repository `opennetworkinglab/OpenNetworkLinux <https://github.com/opennetworkinglab/OpenNetworkLinux>`_.
 You can also download pre-compiled artifacts from `Github Release page <https://github.com/opennetworkinglab/OpenNetworkLinux/releases>`_


 .. note::
     If you're not familiar with ONIE/ONL environment, please check `Getting Started <https://github.com/opencomputeproject/OpenNetworkLinux/blob/master/docs/GettingStarted.md>`_ to
     see how to install the ONL image to an ONIE supported switch.

 Below is an example about how to install the ONL image.

 1. Prepare a server which is accessible by the switch and then download the
 pre-compiled installer from the release page.

 .. code-block::

    wget https://github.com/opennetworkinglab/OpenNetworkLinux/releases/download/v1.4.3/ONL-onf-ONLPv2_ONL-OS_2021-07-16.2159-5195444_AMD64_INSTALLED_INSTALLER
    python -m http.server 8080

 2. Reboot the switch to enter ONIE installation mode

 .. note::
     Please access the switch via BMC or serial console to keep connection during the installation.


 .. code-block::

    onl-onie-boot-mode rescue; reboot

 3. Install ONL installer

 .. code-block::

    onie-nos-install http://$SERVER_IP:8080/ONL-onf-ONLPv2_ONL-OS_2021-07-16.2159-5195444_AMD64_INSTALLED_INSTALLER

 4. Setup switch IP and hostname after the installation.


 Kubernetes Environment
 ----------------------

 Our `ONL <https://github.com/opennetworkinglab/OpenNetworkLinux>`_ version
 includes all packages required by running the Kubernetes on top of it.
 Once the Kubernetes is ready, the `Stratum <https://opennetworking.org/stratum/>`_ application will be deployed to the switch to manage it.

 Unlike server, switch has less CPU and memory resources and we should avoid
 deploying unnecessary workloads into switch.
 Besides, the Stratum application should only be deployed to all switches.

 To achieve the above goals, please apply the resources to your Kubernetes cluster.

 1. Set up Label to all switch node, e.g ``node-role.kubernetes.io=switch``
 2. Set up Taint with ``NoSchedule`` to all switch node, e.g ``node-role.kubernetes.io=switch:NoSchedule``
 3. Properly configure the ``NodeSelector`` and ``Toleration`` when deploying Stratum via DaemonSet

 Example of a five nodes Kubernetes cluster, two switches and three servers

 .. code-block::

      ╰─$ kubectl get node -o custom-columns=NAME:.metadata.name,TAINT:.spec.taints
      NAME       TAINT
      compute1   <none>
      compute2   <none>
      compute3   <none>
      leaf1      [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]]
      leaf2      [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]]
      ╰─$ kubectl get nodes -lnode-role.kubernetes.io=switch
      NAME    STATUS   ROLES    AGE   VERSION
      leaf1   Ready    worker   27d   v1.18.8
      leaf2   Ready    worker   27d   v1.18.8


 Please follow the :ref:`Install SD-Fabric section <install_sd_fabric>` to
 check how to use Taint and NodeSelector during installation.


 # TODO
 Build Image
 -----------

 Fetch Images From Private Registry
 ----------------------------------

 Container images can be download from ONF self-hosted container registry but you have to gain the access token first.

 1. Login to `Aether Harbor Registry <https://registry.aetherproject.org/harbor/sign-in?redirect_url=%2Fharbor%2Fprojects>`_ using your ONF Crowd credential,
 2. Select ``User Profile`` drop-down menu in the upper-right corner
 3. Generate the CLI secret and it's the secret token you have to access the container registry via CLI tool.
 4. Login to the container registry with your username and access token
    by ``docker login command`` to ensure you can access it.

 .. code-block::

       ╰─$ docker login registry.aetherproject.org --username hwchiu
       Password:
       Login Succeeded


 Please follow the :ref:`Install SD-Fabric section <install_sd_fabric>` to
 check how to use Taint and NodeSelector during installation.

 .. _install_sd_fabric:

 Install SD-Fabric
 -----------------

 To install SD-Fabric into your Kubernetes cluster, follow instructions
 described on the `SD-Fabric Helm Chart Repository <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/HEAD/sdfabric/README.md>`_

 A workflow to install the SD-Fabric should be look like

 1. Clone the Helm Charts files from the `SD-Fabric Helm Chart Repository <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/HEAD/sdfabric/README.md>`_
 2. Customize the value file based on your environment
 3. Deploy it via Helm command

 Below is an example how to install SD-Fabric

 .. note::
     Please ensure you have installed switch into Kubernetes cluster and have configured them with proper taints and label,
     and have the permission to fetch the container image from the `Aether Harbor Registry <https://registry.aetherproject.org/harbor/sign-in?redirect_url=%2Fharbor%2Fprojects>`_.


 1. Use the git command to clone the `SD-Fabric Helm Chart Repository <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/HEAD/sdfabric/README.md>`_
 2. Update dependencies

 .. code-block::

       ╰─$ cd sdfabric
       ╰─$ helm dep update                                                                                                                                                                                                              146 ↵
       Downloading onos-classic from repo https://charts.onosproject.org
       Downloading stratum from repo https://charts.stratumproject.org/
       Deleting outdated charts

 3. Prepare your value file, you can modify an existing ``values.yaml`` or use a standalone
 value file, use the later approach in this example.

 .. code-block::

       ╰─$ cat myvaules.yaml
       image:
          credential:
             username: my_username
             password: my_access_token

       onos-classic:
          config:
             netcfg: >
                {
                "devices": {
                   "device:leaf1": {
                      "segmentrouting": {
                      "ipv4NodeSid": 101,
                      "ipv4Loopback": "10.11.22.33",
                      "routerMac": "aa:bb:cc:dd:ee:ff",
                      "pairDeviceId" : "device:leaf2",
                      "pairLocalPort" : 260,
                      "isEdgeRouter": true,
                      "adjacencySids": []
                      }
                   }
                }
             }


       stratum:
          nodeSelector:
             node-role.kubernetes.io: switch

          tolerations:
             - effect: NoSchedule
                value: switch
                key: node-role.kubernetes.io

 Please pay attention to the following fields.

    * Configure your ``registry username and password`` on image.credential
    * Configure the ``label`` on stratum.nodeSelector
    * Configure the ``taint`` on stratum.tolerations
    * Configure the ``network configuration`` on onos-classic.config.netcfg

 4. Verify your Configuration to ensure no YAML/Helm syntax errors

 .. code-block::

       ╰─$ helm template -f myvaules.yaml .

 5. Using the helm command to install it. (``helm version is above v3.2``)
 Following command will install the release `sdfabric` to namespace `sdfabric`.

 .. code-block::

       ╰─$ helm install -n sdfabric --create-namespace -f myvaules.yaml sdfabric .
       NAME: sdfabric
       LAST DEPLOYED: Mon Oct 11 11:12:59 2021
       NAMESPACE: sdfabric
       STATUS: deployed
       REVISION: 1
       TEST SUITE: None
       ╰─$ helm -n sdfabric ls
       NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
       sdfabric        sdfabric        1               2021-10-11 11:12:59.178789 -0700 PDT    deployed        sdfabric-1.0.1
	..
	SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
	SPDX-License-Identifier: Apache-2.0

	Deployment Guide
	================


	Provision Switches
	------------------

	We follow Open Network Install Environment(ONIE) way to install Open Network Linux (ONL) image to switch.
	To work with the SD-Fabric environment, we have customized the ONL image to support related packages and dependencies.

	Image source file can be found on ONF repository `opennetworkinglab/OpenNetworkLinux <https://github.com/opennetworkinglab/OpenNetworkLinux>`_.
	You can also download pre-compiled artifacts from `Github Release page <https://github.com/opennetworkinglab/OpenNetworkLinux/releases>`_


	.. note::
	If you're not familiar with ONIE/ONL environment, please check `Getting Started <https://github.com/opencomputeproject/OpenNetworkLinux/blob/master/docs/GettingStarted.md>`_ to
	see how to install the ONL image to an ONIE supported switch.

	Below is an example about how to install the ONL image.

	1. Prepare a server which is accessible by the switch and then download the
	pre-compiled installer from the release page.

	.. code-block::

	wget https://github.com/opennetworkinglab/OpenNetworkLinux/releases/download/v1.4.3/ONL-onf-ONLPv2_ONL-OS_2021-07-16.2159-5195444_AMD64_INSTALLED_INSTALLER
	python -m http.server 8080

	2. Reboot the switch to enter ONIE installation mode

	.. note::
	Please access the switch via BMC or serial console to keep connection during the installation.


	.. code-block::

	onl-onie-boot-mode rescue; reboot

	3. Install ONL installer

	.. code-block::

	onie-nos-install http://$SERVER_IP:8080/ONL-onf-ONLPv2_ONL-OS_2021-07-16.2159-5195444_AMD64_INSTALLED_INSTALLER

	4. Setup switch IP and hostname after the installation.


	Kubernetes Environment
	----------------------

	Our `ONL <https://github.com/opennetworkinglab/OpenNetworkLinux>`_ version
	includes all packages required by running the Kubernetes on top of it.
	Once the Kubernetes is ready, the `Stratum <https://opennetworking.org/stratum/>`_ application will be deployed to the switch to manage it.

	Unlike server, switch has less CPU and memory resources and we should avoid
	deploying unnecessary workloads into switch.
	Besides, the Stratum application should only be deployed to all switches.

	To achieve the above goals, please apply the resources to your Kubernetes cluster.

	1. Set up Label to all switch node, e.g ``node-role.kubernetes.io=switch``
	2. Set up Taint with ``NoSchedule`` to all switch node, e.g ``node-role.kubernetes.io=switch:NoSchedule``
	3. Properly configure the ``NodeSelector`` and ``Toleration`` when deploying Stratum via DaemonSet

	Example of a five nodes Kubernetes cluster, two switches and three servers

	.. code-block::

	╰─$ kubectl get node -o custom-columns=NAME:.metadata.name,TAINT:.spec.taints
	NAME TAINT
	compute1 <none>
	compute2 <none>
	compute3 <none>
	leaf1 [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]]
	leaf2 [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]]
	╰─$ kubectl get nodes -lnode-role.kubernetes.io=switch
	NAME STATUS ROLES AGE VERSION
	leaf1 Ready worker 27d v1.18.8
	leaf2 Ready worker 27d v1.18.8


	Please follow the :ref:`Install SD-Fabric section <install_sd_fabric>` to
	check how to use Taint and NodeSelector during installation.


	# TODO
	Build Image
	-----------

	Fetch Images From Private Registry
	----------------------------------

	Container images can be download from ONF self-hosted container registry but you have to gain the access token first.

	1. Login to `Aether Harbor Registry <https://registry.aetherproject.org/harbor/sign-in?redirect_url=%2Fharbor%2Fprojects>`_ using your ONF Crowd credential,
	2. Select ``User Profile`` drop-down menu in the upper-right corner
	3. Generate the CLI secret and it's the secret token you have to access the container registry via CLI tool.
	4. Login to the container registry with your username and access token
	by ``docker login command`` to ensure you can access it.

	.. code-block::

	╰─$ docker login registry.aetherproject.org --username hwchiu
	Password:
	Login Succeeded


	Please follow the :ref:`Install SD-Fabric section <install_sd_fabric>` to
	check how to use Taint and NodeSelector during installation.

	.. _install_sd_fabric:

	Install SD-Fabric
	-----------------

	To install SD-Fabric into your Kubernetes cluster, follow instructions
	described on the `SD-Fabric Helm Chart Repository <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/HEAD/sdfabric/README.md>`_

	A workflow to install the SD-Fabric should be look like

	1. Clone the Helm Charts files from the `SD-Fabric Helm Chart Repository <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/HEAD/sdfabric/README.md>`_
	2. Customize the value file based on your environment
	3. Deploy it via Helm command

	Below is an example how to install SD-Fabric

	.. note::
	Please ensure you have installed switch into Kubernetes cluster and have configured them with proper taints and label,
	and have the permission to fetch the container image from the `Aether Harbor Registry <https://registry.aetherproject.org/harbor/sign-in?redirect_url=%2Fharbor%2Fprojects>`_.


	1. Use the git command to clone the `SD-Fabric Helm Chart Repository <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/HEAD/sdfabric/README.md>`_
	2. Update dependencies

	.. code-block::

	╰─$ cd sdfabric
	╰─$ helm dep update 146 ↵
	Downloading onos-classic from repo https://charts.onosproject.org
	Downloading stratum from repo https://charts.stratumproject.org/
	Deleting outdated charts

	3. Prepare your value file, you can modify an existing ``values.yaml`` or use a standalone
	value file, use the later approach in this example.

	.. code-block::

	╰─$ cat myvaules.yaml
	image:
	credential:
	username: my_username
	password: my_access_token

	onos-classic:
	config:
	netcfg: >
	{
	"devices": {
	"device:leaf1": {
	"segmentrouting": {
	"ipv4NodeSid": 101,
	"ipv4Loopback": "10.11.22.33",
	"routerMac": "aa:bb:cc:dd:ee:ff",
	"pairDeviceId" : "device:leaf2",
	"pairLocalPort" : 260,
	"isEdgeRouter": true,
	"adjacencySids": []
	}
	}
	}
	}


	stratum:
	nodeSelector:
	node-role.kubernetes.io: switch

	tolerations:
	- effect: NoSchedule
	value: switch
	key: node-role.kubernetes.io

	Please pay attention to the following fields.

	* Configure your ``registry username and password`` on image.credential
	* Configure the ``label`` on stratum.nodeSelector
	* Configure the ``taint`` on stratum.tolerations
	* Configure the ``network configuration`` on onos-classic.config.netcfg

	4. Verify your Configuration to ensure no YAML/Helm syntax errors

	.. code-block::

	╰─$ helm template -f myvaules.yaml .

	5. Using the helm command to install it. (``helm version is above v3.2``)
	Following command will install the release `sdfabric` to namespace `sdfabric`.

	.. code-block::

	╰─$ helm install -n sdfabric --create-namespace -f myvaules.yaml sdfabric .
	NAME: sdfabric
	LAST DEPLOYED: Mon Oct 11 11:12:59 2021
	NAMESPACE: sdfabric
	STATUS: deployed
	REVISION: 1
	TEST SUITE: None
	╰─$ helm -n sdfabric ls
	NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
	sdfabric sdfabric 1 2021-10-11 11:12:59.178789 -0700 PDT deployed sdfabric-1.0.1