Merge "SEBA-762, add testing/releasing software docs"
diff --git a/SUMMARY.md b/SUMMARY.md
index 7af614c..ae2390b 100644
--- a/SUMMARY.md
+++ b/SUMMARY.md
@@ -87,6 +87,7 @@
* [Delete an OLT](profiles/seba/operate/olt-ops.md)
* [Runtime configuration](profiles/seba/operate/config-ops.md)
* [Add a cross-connect in the AGG switch](profiles/seba/operate/agg-ops.md)
+ * [Command-line Management Tools](profiles/seba/operate/cordctl.md)
* [Try a new version of *](profiles/seba/operate/k8s-helm-ops.md)
* [Known Issues](profiles/seba/known-issues.md)
* Troubleshoot (FAQ)
@@ -109,18 +110,13 @@
* [vRouter](vrouter/README.md)
* [AT&T Workflow Driver](att-workflow-driver/README.md)
* [Kubernetes](kubernetes-service/kubernetes-service.md)
- * [OpenStack](openstack/openstack-service.md)
* [SimpleExampleService](simpleexampleservice/simple-example-service.md)
- * [VTN](vtn-service/README.md)
* [Helm Reference](charts/helm.md)
* [Base Kubernetes](charts/base-kubernetes.md)
- * [Base OpenStack](charts/base-openstack.md)
- * [OpenStack](prereqs/openstack-helm.md)
- * [VTN Setup](prereqs/vtn-setup.md)
* [BBSim](charts/bbsim.md)
* [Kafka](charts/kafka.md)
* [Logging and Monitoring](charts/logging-monitoring.md)
- * [M-CORD](charts/mcord.md)
+ * [Mininet](charts/mininet.md)
* [ONOS](charts/onos.md)
* [Persistent Storage](charts/storage.md)
* [PONNET](charts/ponnet.md)
@@ -130,4 +126,3 @@
* [VOLTHA](charts/voltha.md)
* [XOS-CORE](charts/xos-core.md)
* [XOSSH](charts/xossh.md)
-
diff --git a/charts/mininet.md b/charts/mininet.md
new file mode 100644
index 0000000..4a81d93
--- /dev/null
+++ b/charts/mininet.md
@@ -0,0 +1,25 @@
+# Mininet Helm Chart
+
+The `mininet` chart deploys Mininet for use by [SiaB](../profiles/seba/siab.md).
+It creates a virtual agg switch (Open vSwitch) that connects the downstream
+Ponsim OLTs to an upstream virtual BNG.
+
+You must install the [PONNET](ponnet.md) chart before installing this one.
+
+{% include "../partials/helm/add-cord-repo.md" %}
+
+To install:
+
+```shell
+helm install -n mininet cord/mininet \
+ --set numOlts=1 \
+ --set numOnus=1
+```
+
+Arguments _numOlts_ and _numOnus_ can be set between 1 and 4. The chart connects bridge _nniX_
+to the agg switch for _X_ between 0 and _numOlts_ - 1.
+
+For each _X_ above, a double-tagged interface is created on the virtual BNG for _Y_ between 0 and
+_numOnus_ - 1. This interface has an IP address of _172.(18 + X).Y.10_, an outer VLAN of 222 + _X_,
+and an inner VLAN of 111 + _Y_. The virtual BNG also runs a DHCP server on each interface serving
+addresses in the _172.(18 + X).Y.0/24_ subnet.
diff --git a/charts/ponnet.md b/charts/ponnet.md
index b40f402..04eda55 100644
--- a/charts/ponnet.md
+++ b/charts/ponnet.md
@@ -1,8 +1,8 @@
# PONNET Helm Chart
The `ponnet` Helm chart installs and configures Kubernetes CNI plugins
-for [PONSIM](ponsimv2.md). Currently it creates two Linux bridges, `pon0`
-and `pon1`, that allow a L2 dataplane to be created between the PONSIM
+for [PONSIM](ponsimv2.md). It creates Linux bridges
+that allow a L2 dataplane to be created between the PONSIM
RG and components upstream of the PONSIM OLT. Note that the bridges
are not actually created until [PONSIM](ponsimv2.md) is installed.
@@ -11,7 +11,13 @@
You can then install the chart using:
```bash
-helm install -n ponnet cord/ponnet
+helm install -n ponnet cord/ponnet \
+ --set numOlts=1 \
+ --set numOnus=1
```
-The chart modifies the underlying Kubernetes setup by installing the *bridge* CNI and adding configuration files to create the two bridges. Kubernetes must be configured with CNI enabled. Note that this chart does not seem to work on *minikube*.
+The chart modifies the underlying Kubernetes setup by installing the *bridge* CNI and adding configuration files in `/etc/cni/net.d`
+to create Linux bridges for Ponsim. Arguments _numOlts_ and _numOnus_ can be set between 1 and 4. The chart writes `nniX.conf` for
+_X_ between 0 and _numOlts_ - 1; for each _X_, it writes `ponX.Y.conf` for _Y_ between 0 and _numOnus_ - 1.
+
+In order to use this chart, Kubernetes must be configured with CNI enabled. Note that this chart does not seem to work on *minikube*.
diff --git a/charts/ponsim-pod.md b/charts/ponsim-pod.md
index a8329f8..5ead734 100644
--- a/charts/ponsim-pod.md
+++ b/charts/ponsim-pod.md
@@ -7,5 +7,9 @@
You can then install it using:
```shell
-helm install -n ponsim-pod cord/ponsim-pod
+helm install -n ponsim-pod cord/ponsim-pod \
+ --set numOlts=1 \
+ --set numOnus=1
```
+
+Arguments _numOlts_ and _numOnus_ can be set between 1 and 4.
\ No newline at end of file
diff --git a/charts/ponsimv2.md b/charts/ponsimv2.md
index 599740e..32c382d 100644
--- a/charts/ponsimv2.md
+++ b/charts/ponsimv2.md
@@ -9,25 +9,27 @@
{% include "../partials/helm/add-cord-repo.md" %}
-Then, install ponsim doing:
+To install:
```shell
-helm install -n ponsimv2 cord/ponsimv2
+helm install -n ponsimv2 cord/ponsimv2 \
+ --set numOlts=1 \
+ --set numOnus=1
```
+Arguments _numOlts_ and _numOnus_ can be set between 1 and 4.
+
After a successful install you will see containers like these running in the
VOLTHA namespace:
```bash
-$ kubectl -n voltha get pod
-NAME READY STATUS RESTARTS AGE
-...
-olt-77468cfccd-7ltzr 1/1 Running 0 10m
-onu-6d7d5db8f-pk59s 1/1 Running 0 10m
-rg-5fbddf9bdf-b292r 1/1 Running 0 10m
-...
+$ kubectl -n voltha get pod -l app=ponsim
+NAME READY STATUS RESTARTS AGE
+olt0-fb58fb79f-26g46 1/1 Running 0 22h
+onu0-0-5db946744d-bh5ms 1/1 Running 0 22h
+rg0-0-69cdbf6b58-dx6lm 1/1 Running 0 21h
```
If any of the containers do not come up successfully, the issue is likely
that the [PONNET](ponnet.md) chart is not loaded or was not able to create
-the two Linux bridges.
+the necessary Linux bridges.
diff --git a/charts/voltha.md b/charts/voltha.md
index cf35276..0cacc64 100644
--- a/charts/voltha.md
+++ b/charts/voltha.md
@@ -1,6 +1,6 @@
# Deploy VOLTHA
-## First Time Installation
+## Prerequisites
Install the etcd-operator helm chart first. This chart provides a convenient way of creating and managing etcd clusters. When VOLTHA installs it will attempt to use etcd-operator to create its etcd cluster. Once installed etcd-operator can be left running.
@@ -8,34 +8,32 @@
helm install -n etcd-operator stable/etcd-operator --version 0.8.3
```
-Allow etcd-operator enough time to create the EtdCluster CustomResourceDefinitions. This should only be a couple of seconds after the etcd-operator pods are running. Check the CRD are ready by running the following:
+Wait for etcd-operator to create the EtdCluster CustomResourceDefinitions. This should only be a couple of seconds after the etcd-operator pods are running. Check the CRD are ready by running the following:
```shell
kubectl get crd | grep etcd
```
+## Install
+
{% include "../partials/helm/add-cord-repo.md" %}
-Then, install the VOLTHA helm chart. This will create the VOLTHA pods and will create the etcd-cluster pods.
+To install the VOLTHA helm chart:
```shell
-helm install -n voltha cord/voltha
+helm install -n voltha cord/voltha --version=1.0.6 \
+ --set etcd-cluster.clusterSize=3
```
-Allow enough time for the 3 etcd-cluster pods to start before using the VOLTHA pods.
+Allow all etcd-cluster pods to start before using VOLTHA. If not all etcd-cluster pods are starting successfully,
+you may want to try `--set etcd-cluster.clusterSize=1` above.
-## Standard Uninstall
+## Uninstall
```shell
helm delete --purge voltha
```
-## Standard Install
-
-```shell
-helm install -n voltha cord/voltha
-```
-
## Nodeports Exposed
* Voltha CLI
@@ -105,5 +103,5 @@
Then, install VOLTHA using:
```shell
-helm install -n voltha -f voltha-values.yaml cord/voltha
+helm install -n voltha -f voltha-values.yaml cord/voltha --version=1.0.6
```
diff --git a/installation/platform.md b/installation/platform.md
index d8666fc..67deb21 100644
--- a/installation/platform.md
+++ b/installation/platform.md
@@ -10,7 +10,7 @@
Then, to install the CORD Platform you can use the corresponding chart:
```shell
-helm install -n cord-platform cord/cord-platform --version=6.1.0
+helm install -n cord-platform cord/cord-platform --version=7.0.0
```
### Disabling monitoring and logging
@@ -28,7 +28,7 @@
To disable both of them you can use:
```shell
-helm install -n cord-platform cord/cord-platform --version=6.1.0 --set logging.enabled=false --set nem-monitoring.enabled=false
+helm install -n cord-platform cord/cord-platform --version=7.0.0 --set logging.enabled=false --set nem-monitoring.enabled=false
```
## Alternatively, install the CORD Platform as set of Separate Components
diff --git a/prereqs/hardware.md b/prereqs/hardware.md
index dabfe11..6f1173e 100644
--- a/prereqs/hardware.md
+++ b/prereqs/hardware.md
@@ -84,7 +84,7 @@
* Celestica Tellion GP-1204
* Movistar ONU (with CPE included) (manifactured by Telefonica: <http://www.movistar.es/particulares/movil/moviles/hgu>)
* **XGS-PON**
- * **OLT**: EdgeCore ASFVOLT16 (for more info <bartek_raszczyk@edge-core.com>)
+ * **OLT**: Edgecore ASXvOLT16 (for more info <bartek_raszczyk@edge-core.com>)
* Compatible **OLT optics**
* Hisense/Ligent: LTH7226-PC, LTH7226-PC+
* Source Photonics: XPP-XG2-N1-CDFA
diff --git a/profiles/seba/install.md b/profiles/seba/install.md
index 8b18206..c9761dc 100644
--- a/profiles/seba/install.md
+++ b/profiles/seba/install.md
@@ -24,7 +24,7 @@
Then, proceed with the SEBA chart installation:
```shell
-helm install -n seba cord/seba --version=1.0.0
+helm install -n seba cord/seba --version=2.0.0-alpha1
```
### Alternatively, install SEBA as separate components
diff --git a/profiles/seba/known-issues.md b/profiles/seba/known-issues.md
index f9fb8f8..78fbfad 100644
--- a/profiles/seba/known-issues.md
+++ b/profiles/seba/known-issues.md
@@ -1,35 +1,20 @@
# Known Issues
-## SEBA 1.0 Release
+## SEBA 2.0-alpha Release
-There are known major issues having to do with rebooting of the physical hardware, both
-the OLT and the AGG switch, as well as continuously deleting/re-creating of an OLT Device via NEM. These issues are described in more detail in below:
+This release of SEBA is qualified as 'alpha' due to known major issues related to the use of OLT software from Broadcom referred to as BAL - Broadband Adaptation Layer. The SEBA 2.0-alpha release uses [BAL 2.6](https://github.com/balapi/bal-api-2.6), which is no longer supported by Broadcom.
-* [SEBA-388](https://jira.opencord.org/browse/SEBA-388)
- With this issue, DHCP may not work after AGG switch reboot. A possible workaround is to use a different config for the DHCPl2Relay app that uses the OLT's uplink to reach the DHCP Server, instead of the Switch's uplink. More details are described [here](troubleshoot/no-dhcp.md)
-* [SEBA-385](https://jira.opencord.org/browse/SEBA-385)
- With this issue, the OLT may not pass traffic anymore after reboot. Currently, the only workaround is to, enter the ONOS CLI and force a 'device-remove <<device-id>' for the device from VOLTHA (in other words, force VOLTHA to reconnect to ONOS), and then perform the authentication/dhcp steps again for the subscribers
-* [SEBA-393](https://jira.opencord.org/browse/SEBA-393)
- With this issue, if an OLT device gets deleted and re-created many times within a short period of time (10+ times within an hour or so), VOLTHA may run into an issue where it's unable to save any information pertaining the OLT device such as its ports or any ONU devices connected to those ports. The workaround for this issue is to re-install VOLTHA and re-create the OLT device through NEM.
+These issues are described in more detail in below:
-Fixes to these issues will be addressed soon after the release.
+* [SEBA-670](https://jira.opencord.org/browse/SEBA-670)
+ This issue can be triggered by disable and subsequent re-enable of the ONU via NEM (or VOLTHA). In the SEBA pod, using the AT&T workflow, these actions are accompanied by the removal of subscriber flows upon disable of the ONU, and the reprogramming of eapol flows upon re-enable of the ONU. An irrecoverable error is seen in BAL (specifically bal_core_dist) which does not allow flows to be added to the OLT. As a result authentication of RG's fail.
+* [SEBA-775](https://jira.opencord.org/browse/SEBA-775)
+ This issue is closely related to SEBA-670. The fundamental issue is the removal of flows in the OLT which possibly leaves behind state in BAL 2.6 that does not allow the subsequent reprogramming of flows. It can be triggered by the disable/re-enable of ONUs (as in SEBA-670) or by the remove-subscriber followed by the add-subscriber calls (without change of ONU state). It can be triggered in systems with a single-ONU, as well as in multi-ONU cases. Empirically the issue is more easily reproduced in multi-ONU scenarios. It is observed with single gem ports as well as when multiple gem ports are used for a subscriber in the technology profile.
+* [SEBA-777](https://jira.opencord.org/browse/SEBA-777)
+ Due to the issues mentioned above, subscriber speed profiles cannot be updated reliably, as the update requires the removal of subscriber flows that point to a bandwidth meter, and the reprogramming of flows that point to a new meter.
+* [SEBA-776](https://jira.opencord.org/browse/SEBA-776)
+ This issue is seen sometimes with the use of multiple gem ports in a technology profile. Packet duplication appears to mirror the number of gem ports - using 4 gem ports results in 4 packets for every packet transmitted.
-Another minor issue that does not affect functionality is related to the state of an ONU on the VOLTHA CLI, after disable/re-enable of the ONU.
+Fixes to these issues will be addressed in a future release as we upgrade the OLT software to BAL 3.0.
-```shell
-(voltha) devices
-Devices:
-+------------------+-------------------+------+------------------+------------------+-------------+-------------+----------------+----------------+------------------+------------------------+-------------------------+--------------------------+----------------------+------------------------------+
-| id | type | root | parent_id | serial_number | admin_state | oper_status | connect_status | parent_port_no | host_and_port | reason | proxy_address.device_id | proxy_address.channel_id | proxy_address.onu_id | proxy_address.onu_session_id |
-+------------------+-------------------+------+------------------+------------------+-------------+-------------+----------------+----------------+------------------+------------------------+-------------------------+--------------------------+----------------------+------------------------------+
-| 0001a82d21249c28 | openolt | True | 000100000a5a007a | 10.90.0.122:9191 | ENABLED | ACTIVE | REACHABLE | | 10.90.0.122:9191 | | | | | |
-| 00014f81e9f21dbb | brcm_openomci_onu | True | 0001a82d21249c28 | ALPHe3d1cf9d | ENABLED | ACTIVE | REACHABLE | 536870912 | | initial-mib-downloaded | 0001a82d21249c28 | | 1 | 1 |
-| 0001666321e17127 | brcm_openomci_onu | True | 0001a82d21249c28 | ALPHe3d1ced5 | ENABLED | ACTIVE | REACHABLE | 536870912 | | initial-mib-downloaded | 0001a82d21249c28 | | 2 | 2 |
-| 000175a511654437 | brcm_openomci_onu | True | 0001a82d21249c28 | ISKT71e80080 | ENABLED | ACTIVE | REACHABLE | 536870927 | | omci-flows-pushed | 0001a82d21249c28 | 15 | 1 | 1 |
-| 0001cb66a703449d | brcm_openomci_onu | True | 0001a82d21249c28 | ALPHe3d1cfe3 | ENABLED | ACTIVE | REACHABLE | 536870912 | | initial-mib-downloaded | 0001a82d21249c28 | | 3 | 3 |
-| 0001246c72a99ddd | brcm_openomci_onu | True | 0001a82d21249c28 | ALPHe3d1cf8e | ENABLED | ACTIVE | REACHABLE | 536870912 | | initial-mib-downloaded | 0001a82d21249c28 | | 4 | 4 |
-| 000192f54601ecb4 | brcm_openomci_onu | True | 0001a82d21249c28 | ALPHe3d1cf70 | ENABLED | ACTIVE | REACHABLE | 536870912 | | initial-mib-downloaded | 0001a82d21249c28 | | 5 | 5 |
-+------------------+-------------------+------+------------------+------------------+-------------+-------------+----------------+----------------+------------------+------------------------+-------------------------+--------------------------+----------------------+------------------------------+
-```
-
-In the `reason` column of the devices table, the state of the ONU devices are displayed. Normally after an ONU is disabled, and then re-enabled, it should display the `initial-mib-downloaded` state, indicating that the RG behind the ONU should be ready to re-authenticate (as per the AT&T workflow). However, in this release, the state is incorrectly displayed as `omci-flows-pushed`. This does not affect authentication, dhcp or the subscribers.
+In addition, OLT-reboot [SEBA-385](https://jira.opencord.org/browse/SEBA-385) can result in some ONUs not returning to ACTIVE state in VOLTHA.
diff --git a/profiles/seba/operate/cordctl.md b/profiles/seba/operate/cordctl.md
new file mode 100644
index 0000000..0460a2c
--- /dev/null
+++ b/profiles/seba/operate/cordctl.md
@@ -0,0 +1,177 @@
+# Command-line management of a SEBA Pod
+
+SEBA includes a tool named `cordctl` that may be used for command-line management of the pod.
+
+## Installation and Configuration
+
+This tool is installed on the operator's computer as a single binary. Binaries are available for multiple platforms, including Linux (AMD and ARM), Darwin, and Windows. Start by finding the binary appropriate to your computer by visiting the [cordctl release page](https://github.com/opencord/cordctl/releases).
+
+
+![cordctl release page](./screenshots/cordctl-releases.png)
+
+One way to download the binary is by clicking the appropriate link in your web browser and saving the file. Another method is to copy the link and use `curl` in a shell session to download the binary. We will assume for the sake of this tutorial that you're using a Linux AMD64 workstation.
+
+```bash
+sudo curl -sSL https://github.com/opencord/cordctl/releases/download/1.1.2/cordctl-linux-amd64 -o /usr/local/bin/cordctl
+sudo chmod a+x /usr/local/bin/cordctl
+```
+
+You can verify the binary is installed correctly by using the `version` command:
+
+```bash
+$ cordctl version --client-only
+Client:
+ Version 1.1.2
+ Go version: go1.12
+ Git commit: 1a86568
+ Git dirty: false
+ Built: 2019-07-09T15:12:28Z
+ OS/Arch: linux/amd64
+```
+
+Note that we used the `--client-only` flag because we have not yet configured `cordctl` with the address of our SEBA pod. We'll do that next. You'll need to know the IP address or hostname of your SEBA pod and the username and password used to access XOS on that pod. For the sake of this tutorial, we'll assume a hostname of `mysebapod`, a username of `admin@opencord.org` and a password of `letmein`.
+
+```bash
+$ cordctl -u admin@opencord.org -p letmein -s mysebapod:30011 version
+Client:
+ Version 1.1.2
+ Go version: go1.12
+ Git commit: 1a86568
+ Git dirty: false
+ Built: 2019-07-09T15:12:28Z
+ OS/Arch: linux/amd64
+
+Server:
+ Version 3.3.1
+ Python version: 2.7.16
+ Django version: 1.11.22.final.0
+ Git commit: bb1725739c7949a4ed0a86b18cdb6d707779244f
+ Built: 2019-07-09T15:
+```
+
+As we can see, `cordctl` has retrieved information about the server. This confirms that it is successfully talking to a SEBA pod. Rather than having to specify the username, password, and server for every `cordctl` command, we can create a config file that has this information in it:
+
+```bash
+mkdir -p ~/.cord
+cordctl -u admin@opencord.org -p letmein -s mysebapod:30011 config > ~/.cord/config
+```
+
+Once this is done, we no longer need to specify those arguments:
+
+```bash
+$ cordctl version
+Client:
+ Version 1.1.2
+ Go version: go1.12
+ Git commit: 1a86568
+ Git dirty: false
+ Built: 2019-07-09T15:12:28Z
+ OS/Arch: linux/amd64
+
+Server:
+ Version 3.3.1
+ Python version: 2.7.16
+ Django version: 1.11.22.final.0
+ Git commit: bb1725739c7949a4ed0a86b18cdb6d707779244f
+ Built: 2019-07-09T15:
+```
+
+## Listing the service directory
+
+A useful cordctl command is to see the set of services that are installed. For example,
+
+```bash
+$ cordctl service list
+NAME VERSION STATE
+fabric 2.2.2 present
+onos 2.1.2 present
+kubernetes 1.3.1 present
+volt 2.2.4 present
+att-workflow-driver 1.2.3 present
+fabric-crossconnect 1.2.2 present
+rcord 1.3.2 present
+core 3.3.1 present
+```
+
+We can see that a typical stack of XOS SEBA services are present, including the att-workflow-driver service, the volt service, and the rcord service. The services are all present, and the versions are listed.
+
+## Interacting with models
+
+`cordctl` has several commands for interacting with models. Let's see how we might accomplish a common SEBA workflow such as disabling an ONU using `cordctl`. First lets see what ONUs are known to the pod:
+
+```bash
+$ cordctl model list ONUDevice
+ID ADMIN_STATE DEVICE_TYPE PON_PORT_ID SERIAL_NUMBER VENDOR
+1 ENABLED ponsim_onu 1 PSMO00000000 ponsim
+```
+
+This particular pod is a Seba-in-a-Box pod and it has one ONU available, and that ONU has a serial number of `PSMO00000000`. Now let's disable that ONU:
+
+```bash
+$ cordctl model update ONUDevice --filter serial_number=PSMO00000000 --set-field admin_state=ADMIN_DISABLED
+ID MESSAGE
+1 Updated
+```
+
+If we look at the ONU again, we can see it has been administratively disabled:
+
+```bash
+$ cordctl model list ONUDevice
+ID ADMIN_STATE DEVICE_TYPE PON_PORT_ID SERIAL_NUMBER VENDOR
+1 ADMIN_DISABLED ponsim_onu 1 PSMO00000000 ponsim
+```
+
+You can also create new objects using `cordctl`. For example, let's add a serial number to the white list. First we have to take a short side trip to get the service identifier of the AttWorkflowDriverService:
+
+```bash
+$ cordctl model list Service --filter name=att-workflow-driver
+ID DESCRIPTION KIND NAME SERVICE_SPECIFIC_ID VERSIONNUMBER
+1 oss att-workflow-driver
+```
+
+The ID is `1`, we'll need to specify that ID as the `owner_id` when creating our white list entry. We're now ready to create the white list entry:
+
+```bash
+$ cordctl model create AttWorkflowDriverWhiteListEntry --set-field pon_port_id=3,serial_number=1234,device_id=5678,owner_id=1
+ID MESSAGE
+2 Created
+```
+
+Note that we invented a pon_port_id, serial_number, and device_id. In a real SEBA pod, use the actual values relevant to the ONU that you're intending to add to the white list.
+
+Deleting the white list entry is straightforward:
+
+```bash
+$ cordctl model delete AttWorkflowDriverWhiteListEntry --filter serial_number=1234
+ID MESSAGE
+2 Deleted
+```
+
+## Backup and Restore
+
+It's possible to backup and restore the XOS data model using `cordctl`. To backup, do the following:
+
+```bash
+$ cordctl backup create mybackup
+Waiting for sync oooooooooooooooooooooooooooooooooooo........................................................................................................................................................................................................................................................................................................................
+Status: created
+URI file:///var/run/xos/backup/local/autogenerated-file-2019-07-09-21-03-41
+Downloading mybackup
+STATUS CHECKSUM CHUNKS BYTES
+SUCCESS sha256:e33122596821a138080f522c75ee72dbb5179310a53352fc10cdf9443dc38d91 6 375656
+```
+
+This will backup the data model and place that backup in a local file called `mybackup`. If you want to restore the backup at a later time, then use the following example:
+
+```bash
+$ cordctl backup restore mybackup
+Created backup file 4
+Created backup-restore operation id=4 uuid=b3d4db88-04ae-444e-be19-2d1f6d4ede56
+Waiting for completion ooooooooooooooooooo........................................................................................................................................................................................................................................................................................................................................................x
+STATUS CHECKSUM CHUNKS BYTES
+SUCCESS sha256:e33122596821a138080f522c75ee72dbb5179310a53352fc10cdf9443dc38d91 6 375656
+```
+
+## Getting more information
+
+This short tutorial only demonstrates part of `cordctl`'s functionality. For more information, see the [cordctl documentation](../../../cordctl).
diff --git a/profiles/seba/operate/onu-ops.md b/profiles/seba/operate/onu-ops.md
index a54b73e..0f0a53b 100644
--- a/profiles/seba/operate/onu-ops.md
+++ b/profiles/seba/operate/onu-ops.md
@@ -11,7 +11,9 @@
## Enable/Disable an ONU
Once you are in the ONU details view, you can just change it's `Admin State`
-to enable or disable the device.
+to `ENABLED` if you wish the device to be enabled, or `ADMIN_DISABLED` if
+you wish the device to be administratively disabled. Avoid manunally setting
+the state to `DISABLED` as the workflow may toggle it back to `ENABLED`.
![ONU Detail view in XOS](./screenshots/onu_detail_view.png "ONU Detail view in XOS")
diff --git a/profiles/seba/operate/screenshots/cordctl-releases.png b/profiles/seba/operate/screenshots/cordctl-releases.png
new file mode 100644
index 0000000..226bc6f
--- /dev/null
+++ b/profiles/seba/operate/screenshots/cordctl-releases.png
Binary files differ
diff --git a/profiles/seba/siab-with-fabric-switch.md b/profiles/seba/siab-with-fabric-switch.md
index 9305456..f4ac493 100644
--- a/profiles/seba/siab-with-fabric-switch.md
+++ b/profiles/seba/siab-with-fabric-switch.md
@@ -12,19 +12,19 @@
1. Fabric switch bringup. To bring up the fabric switch, follow the procedure [here](../../installation/fabric-setup.md) and verify the fabric switch is discovered by onos (`ssh -p 30115 onos@K8S_NODE_IP devices`).
-1. Enable EAPOL on linux-bridge `pon0` and add the physical interface on the k8node, running PONSIM-OLT to `pon1` bridge.
+1. Enable EAPOL on Linux bridge `pon0.0` and add the physical interface on the Kubernetes node running PONSIM-OLT to `nni0` bridge.
```bash
- $ sudo brctl addif pon1 ens1d1
+ $ sudo brctl addif nni0 ens1d1
$ brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.02426df5adb1 no
- pon0 8000.52f6b00415e4 no veth212f887a
- veth84cbf365
- pon1 8000.0a580a170001 no veth6abcdca3
+ nni0 8000.0a580a170001 no veth6abcdca3
ens1d1
- $ echo 8 > /tmp/pon0_group_fwd_mask
- $ sudo cp /tmp/pon0_group_fwd_mask /sys/class/net/pon0/bridge/group_fwd_mask
+ pon0.0 8000.52f6b00415e4 no veth212f887a
+ veth84cbf365
+ $ echo 8 > /tmp/group_fwd_mask
+ $ sudo cp /tmp/group_fwd_mask /sys/class/net/pon0.0/bridge/group_fwd_mask
```
1. Modify `~/cord/helm-charts/xos-profiles/ponsim-pod/tosca/020-pod-olt.yaml`. Update `switch_datapath_id` to your fabric-switch dpid and `switch_port` to the port number of Fabric-switch to which the Kubernetes node running OLT is connected.
diff --git a/profiles/seba/siab.md b/profiles/seba/siab.md
index eed5f44..7e91f42 100644
--- a/profiles/seba/siab.md
+++ b/profiles/seba/siab.md
@@ -31,11 +31,14 @@
To build a SiaB that uses the released service versions specified in the Helm charts:
```bash
-make # or 'make stable'
+make [stable] [NUM_OLTS=n] [NUM_ONUS_PER_OLT=m] # `make` and `make stable` are the same
```
> NOTE that `make` or `make stable` will install SEBA with the container versions that are
-> defined in the helm charts. If you want to install SEBA 1.0 please use: `make siab-1.0`
+> defined in the helm charts. If you want to install SEBA 2.0 please use: `make siab-2.0-alpha1`
+
+You can specify the number of OLTs (up to 4) and number of ONUs per OLT (up to 4) that you want to
+create.
After a successful install, you will see the message:
@@ -57,6 +60,9 @@
make run-tests
```
+Note that the tests currently assume a single OLT/ONU, so some tests will
+likely fail if you have configured multiple OLTs and ONUs.
+
### Quick start: Build SiaB using latest development code
To build a SiaB that uses the latest development code:
@@ -64,69 +70,8 @@
```bash
make latest [NUM_OLTS=n] [NUM_ONUS_PER_OLT=m]
```
-
-With the `latest` target, you can specify the number of OLTs (up to 4) and number of ONUs per OLT that you want to
-create. Each OLT associates with "m" number of ONUs. If you specify more than one OLT you will see several OLT/ONU/RG containers when you run `kubectl -n voltha get pod`:
-
-Naming convention:
-```
-1st OLT - olt0-xxx
-2nd OLT - olt1-xxx
-1st ONU attached to 1st OLT - onu0-0-xx (onu<olt>-<onu>)
-2nd ONU attached to 1st OLT - onu0-1-xx
-1st ONU attached to 2nd OLT - onu1-0-xx
-2nd ONU attached to 2nd OLT - onu1-1-xx
-RG also follows the same naming logic as ONU (rg0-0-xx, rg0-1-xx, rg1-0-xx, rg1-1-xx)
-linux bridges interconnecting ONU and RG also follows the same naming logic as ONU (pon0.0, pon0.1 ..)
-```
-
-```bash
-$ kubectl -n voltha get pod
-NAME READY STATUS RESTARTS AGE
-voltha olt0-774f9cb5f7-9mwwg 1/1 Running 0 33m
-voltha olt1-5f7c44f554-n47mv 1/1 Running 0 33m
-voltha onu0-0-5768c4567c-tc2rt 1/1 Running 0 33m
-voltha onu0-1-859c87ccd9-sr9fq 1/1 Running 0 33m
-voltha onu1-0-6c58d9957f-6bbk4 1/1 Running 0 33m
-voltha onu1-1-8555c74487-6fzwb 1/1 Running 0 33m
-voltha rg0-0-77fcd5d6bc-55cxt 1/1 Running 0 33m
-voltha rg0-1-57cdc6956f-xm2gp 1/1 Running 0 33m
-voltha rg1-0-7d6689bd85-tgjcp 1/1 Running 0 33m
-voltha rg1-1-54994485c5-swnd2 1/1 Running 0 33m
-```
-
-Likewise `brctl show` will output:
-
-```bash
-$ brctl show
-bridge name bridge id STP enabled interfaces
-docker0 8000.02427dd2bfc4 no veth0fbf0dd
-nni0 8000.76030be9e97b no veth3c7ade40
- vethc01838f1
-nni1 8000.ae08243d745e no vethe0df415e
- vetheef40c90
-pon0.0 8000.2aa5060d44b7 no vethaa880e65
- vethae9c7b9d
-pon0.1 8000.3602b50c2521 no veth32a2f3d2
- veth971b571b
-pon1.0 8000.7efc437e91e4 no veth1ea11fe3
- veth51cbc451
-pon1.1 8000.e2423416a798 no veth3323ad21
- veth3718d925
-```
-
-Above there are four separate datapath chains:
-```
-rg0-0 -> pon0.0 -> onu0-0 -> olt0 -> nni0
-rg0-1 -> pon0.1 -> onu0-1 -> olt0 -> nni0
-rg1-0 -> pon1.0 -> onu1-0 -> olt1 -> nni1
-rg1-1 -> pon1.1 -> onu1-1 -> olt1 -> nni1
-```
-All of the `nniX` bridges connect to the agg switch in Mininet on different ports.
-
-A subscriber is created for each RG `rg<olt>-<onu>` with S-tag of `222+<olt>` and C-tag of `111+<onu>`.
-After `rg<olt>-<onu>` is authenticated, it will get an IP address on subnet `172.18+<olt>.<onu>.0/24` and ping
-`172.18+<olt>.<onu>.10` as its BNG.
+You can specify the number of OLTs (up to 4) and number of ONUs per OLT (up to 4) that you want to
+create.
After a successful install, you will see the message:
@@ -142,11 +87,12 @@
make run-tests-latest
```
-Note that the tests currently assume a single OLT, so some tests will likely fail if you have configured multiple OLTs.
+Note that the tests currently assume a single OLT/ONU, so some tests will
+likely fail if you have configured multiple OLTs and ONUs.
## Installation procedure
-The rest of this page describes a manual method for installing SEBA-in-a-Box.
+The rest of this page describes a manual method for installing SEBA-in-a-Box. It also provides an overview of what is installed by each chart.
### Prerequisites
@@ -235,7 +181,7 @@
Install the `cordctl` command line tool:
```bash
-export CORDCTL_VERSION=1.0.0
+export CORDCTL_VERSION=1.1.2
export CORDCTL_PLATFORM=linux-amd64
curl -L -o /tmp/cordctl "https://github.com/opencord/cordctl/releases/download/$CORDCTL_VERSION/cordctl-$CORDCTL_PLATFORM"
sudo mv /tmp/cordctl /usr/local/bin/cordctl
@@ -271,6 +217,16 @@
helm install -n onos onos
```
+You should see the following pods running:
+
+```bash
+$ kubectl get pod
+NAME READY STATUS RESTARTS AGE
+cord-kafka-0 1/1 Running 1 14h
+cord-kafka-zookeeper-0 1/1 Running 0 14h
+onos-558445d9bc-c2cd5 2/2 Running 0 14h
+```
+
## Install VOLTHA charts
Run these commands to install VOLTHA:
@@ -285,57 +241,83 @@
kubectl get crd | grep etcd
# After EtcdCluster CRD is in place
helm dep up voltha
-helm install -n voltha -f configs/seba-ponsim.yaml voltha
+helm install -n voltha voltha --set etcd-cluster.clusterSize=1
```
**Before proceeding**
-Run: `kubectl get pod|grep etcd-cluster`
+Run: `kubectl get pod -l app=etcd`
You should see the etcd-cluster pod up and running.
```bash
-$ kubectl get pod|grep etcd-cluster
-etcd-cluster-q9zhrwvllh 1/1 Running 0 20m
+$ kubectl get pod -l app=etcd
+NAME READY STATUS RESTARTS AGE
+etcd-cluster-jcjk2x97w6 1/1 Running 0 14h
```
+You should see the VOLTHA pods created:
+
+```bash
+$ kubectl get pod -n voltha
+NAME READY STATUS RESTARTS AGE
+default-http-backend-798fb4f44c-fb696 1/1 Running 0 14h
+freeradius-754bc76b5-22lcm 1/1 Running 0 14h
+netconf-66b767bddc-hbsgr 1/1 Running 0 14h
+nginx-ingress-controller-5fc7b87c86-bd55x 1/1 Running 0 14h
+ofagent-556cd6c978-lknd4 1/1 Running 0 14h
+vcli-67c996f87d-vw4pk 1/1 Running 0 14h
+vcore-0 1/1 Running 0 14h
+voltha-6f8d7bf7b-4gkkj 1/1 Running 1 14h
+```
+
+
## Install Ponsim charts
Run these commands to install Ponsim (after installing VOLTHA):
```bash
cd ~/cord/helm-charts
-helm install -n ponnet ponnet
+NUM_OLTS=1 # can be between 1 and 4
+NUM_ONUS_PER_OLT=1 # can be between 1 and 4
+helm install -n ponnet ponnet --set numOlts=$(NUM_OLTS) --set numOnus=$(NUM_ONUS_PER_OLT)
# Wait for CNI changes
~/cord/helm-charts/scripts/wait_for_pods.sh kube-system
-helm install -n ponsimv2 ponsimv2
+helm install -n ponsimv2 ponsimv2 --set numOlts=$(NUM_OLTS) --set numOnus=$(NUM_ONUS_PER_OLT)
# Iptables setup
sudo iptables -P FORWARD ACCEPT
```
+Setting `numOlts` and `numOnus` is optional; the default is 1.
+
**Before proceeding**
-Run: `kubectl -n voltha get pod`
+Run: `kubectl -n voltha get pod -l app=ponsim`
-Make sure that all of the pods in the voltha namespace are in Running state.
```bash
-$ kubectl -n voltha get pod
-NAME READY STATUS RESTARTS AGE
-default-http-backend-846b65fb5f-rklfb 1/1 Running 0 6h
-freeradius-765c9b486c-6qs7t 1/1 Running 0 6h
-netconf-7d7c96c88b-29cv2 1/1 Running 0 6h
-nginx-ingress-controller-6db99757f7-d9cpk 1/1 Running 0 6h
-ofagent-7d7b854cd4-fx6gq 1/1 Running 0 6h
-olt0-5455744678-hqbwh 1/1 Running 0 6h
-onu0-5df655b9c9-prfjz 1/1 Running 0 6h
-rg0-75845c54bc-fjgrf 1/1 Running 0 6h
-vcli-6875544cf-rfdrh 1/1 Running 0 6h
-vcore-0 1/1 Running 0 6h
-voltha-546cb8fd7f-5n9x4 1/1 Running 3 6h
+$ kubectl -n voltha get pod -l app=ponsim
+NAME READY STATUS RESTARTS AGE
+olt0-f4744dc5-xdrjb 1/1 Running 0 15h
+onu0-0-6bf67bf6c6-76gn7 1/1 Running 0 15h
+rg0-0-7b9d5cdb5c-jc8p5 1/1 Running 0 14h
```
-If you see the olt pod in CrashLoopBackOff state, try deleting (`helm delete --purge`) and reinstalling the ponsimv2 chart.
+Make sure that all of the pods in the voltha namespace are in Running state.
+If you see the `olt0` pod in CrashLoopBackOff state, try deleting (`helm delete --purge`) and reinstalling the ponsimv2 chart.
+
+If you install more than one OLT/ONU then you will see more containers above. The naming convention:
+```
+1st OLT - olt0-xxx
+2nd OLT - olt1-xxx
+1st ONU attached to 1st OLT - onu0-0-xx (onu<olt>-<onu>)
+2nd ONU attached to 1st OLT - onu0-1-xx
+1st ONU attached to 2nd OLT - onu1-0-xx
+2nd ONU attached to 2nd OLT - onu1-1-xx
+RG follows the same naming logic as ONU (rg0-0-xx, rg0-1-xx, rg1-0-xx, rg1-1-xx)
+Linux bridges interconnecting ONU and RG follow the same naming logic as ONU (pon0.0, pon0.1 ...)
+Linux bridges interconnecting OLT and Mininet follow same naming logic as OLT (nni0, nni1, ...)
+```
Run `http GET http://127.0.0.1:30125/health|jq '.state'`. It should return `"HEALTHY"`:
@@ -344,6 +326,7 @@
"HEALTHY"
```
+
## Install NEM charts
Run these commands:
@@ -376,26 +359,26 @@
Run these commands:
```bash
-helm install -n ponsim-pod xos-profiles/ponsim-pod
+helm install -n ponsim-pod xos-profiles/ponsim-pod --set numOlts=$(NUM_OLTS) --set numOnus=$(NUM_ONUS_PER_OLT)
~/cord/helm-charts/scripts/wait_for_pods.sh
```
+The TOSCA creates a subscriber for each RG `rg<olt>-<onu>` with S-tag of `222+<olt>` and C-tag of `111+<onu>`.
+
**Before proceeding**
-Log into the XOS GUI at `http://<hostname>:30001` (credentials: admin@opencord.org / letmein). You should see an AttWorkflowDriver Service Instance with authentication state AWAITING.
-
-To run the check from the command line:
+Log into the XOS GUI at `http://<hostname>:30001` (credentials: admin@opencord.org / letmein). You should see an AttWorkflowDriver Service Instance with authentication state AWAITING. To check this from the command line:
```bash
cordctl model list AttWorkflowDriverServiceInstance -f "authentication_state=AWAITING"
```
-This will show only the AttWorkflowDriver Service Instances in AWAITING state. Wait until you see something like:
+This will show only the AttWorkflowDriver Service Instances in AWAITING state. Wait until you see a line for each ONU:
```bash
$ cordctl model list AttWorkflowDriverServiceInstance -f "authentication_state=AWAITING"
-OWNER_ID SERIAL_NUMBER OF_DPID UNI_PORT_ID STATUS_MESSAGE ID NAME
-2 PSMO12345678 of:0000aabbccddeeff 128 ONU has been validated - Awaiting Authentication 56
+ID NAME OF_DPID OWNER_ID SERIAL_NUMBER STATUS_MESSAGE UNI_PORT_ID
+56 of:0000d0d3e158fede 2 PSMO00000000 ONU has been validated - Awaiting Authentication 128
```
## Install Mininet
@@ -416,7 +399,7 @@
```bash
cd ~/cord/helm-charts
-helm install -n mininet mininet
+helm install -n mininet mininet --set numOlts=$(NUM_OLTS) --set numOnus=$(NUM_ONUS_PER_OLT)
~/cord/helm-charts/scripts/wait_for_pods.sh
```
@@ -432,30 +415,32 @@
Run: `brctl show`
-You should see two interfaces on each of the pon0 and nni0 Linux bridges.
+You should see two interfaces on the `ponX.Y` and `nniX` Linux bridges.
```bash
$ brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.02429d07b4e2 no
-pon0 8000.bec4912b1f6a no veth25c1f40b
+pon0.0 8000.bec4912b1f6a no veth25c1f40b
veth2a4c914f
nni0 8000.0a580a170001 no veth3cc603fe
vethb6820963
```
-## Enable pon0 to forward EAPOL packets
+You will see more bridges if you've configured multiple OLTs and ONUs. All of the `nniX` Linux bridges connect to the agg switch in Mininet on different ports.
-This is necessary to enable the RG to authenticate. Run these commands:
+## Enable pon bridges to forward EAPOL packets
+
+This is necessary to enable the RG to authenticate:
```bash
-echo 8 > /tmp/pon0_group_fwd_mask
-sudo cp /tmp/pon0_group_fwd_mask /sys/class/net/pon0/bridge/group_fwd_mask
+echo 8 > /tmp/group_fwd_mask
+for BRIDGE in /sys/class/net/pon*; do sudo cp /tmp/group_fwd_mask $BRIDGE/bridge/group_fwd_mask; done
```
## ONOS customizations
-Right now it’s necessary to install some custom configuration to ONOS directly. Run this command:
+It’s necessary to install some custom configuration to ONOS directly. Run this command:
```bash
http -a karaf:karaf POST \
@@ -464,7 +449,7 @@
The above command instructs the ONU to exchange untagged packets with the RG, rather than packets tagged with VLAN 0.
-At this point the system should be fully installed and functional.
+At this point the system should be fully installed and functional.
## Validating the install
@@ -473,10 +458,11 @@
Enter the RG pod in the voltha namespace:
```bash
-RG_POD=$( kubectl -n voltha get pod -l "app=rg0-0" -o jsonpath='{.items[0].metadata.name}' )
+RG_POD=$( kubectl -n voltha get pod | grep rg0-0 | awk '{print $1}' )
kubectl -n voltha exec -ti $RG_POD bash
```
+If you built SiaB with multiple OLTs and ONUs, you can choose any RG to authenticate.
Inside the pod, run this command:
```bash
@@ -500,8 +486,7 @@
**Before proceeding**
-In the XOS GUI, the AttDriverWorkflow Service Instance should now be in APPROVED state.
-You can check for this on the command line by running:
+In the XOS GUI, the AttDriverWorkflow Service Instance should now be in APPROVED state. You can check for this on the command line by running:
```bash
cordctl model list AttWorkflowDriverServiceInstance -f "authentication_state=APPROVED"
@@ -511,12 +496,11 @@
```bash
$ cordctl model list AttWorkflowDriverServiceInstance -f "authentication_state=APPROVED"
-OF_DPID UNI_PORT_ID STATUS_MESSAGE ID NAME OWNER_ID SERIAL_NUMBER
-of:0000aabbccddeeff 128 ONU has been validated - Authentication succeeded 56 2 PSMO12345678
+ID NAME OF_DPID OWNER_ID SERIAL_NUMBER STATUS_MESSAGE UNI_PORT_ID
+56 of:0000d0d3e158fede 2 PSMO00000000 ONU has been validated - Authentication succeeded 128
```
-The FabricCrossconnect Service Instance should have a check in the Backend status column in the GUI.
-You can check for this on the command line by running:
+The FabricCrossconnect Service Instance should have a check in the Backend status column in the GUI. You can check for this on the command line by running:
```bash
cordctl model list FabricCrossconnectServiceInstance -f 'backend_status=OK'
@@ -526,8 +510,8 @@
```bash
$ cordctl model list FabricCrossconnectServiceInstance -f 'backend_status=OK'
-SWITCH_DATAPATH_ID SOURCE_PORT ID NAME OWNER_ID S_TAG
-of:0000000000000001 2 59 5 222
+ID NAME OWNER_ID S_TAG SOURCE_PORT SWITCH_DATAPATH_ID
+59 4 222 2 of:0000000000000001
```
### Obtain an IP address for the RG
@@ -556,7 +540,7 @@
**Before proceeding**
-Make sure that eth0 inside the RG container has an IP address on the 172.18.0.0/24 subnet:
+`rg<olt>-<onu>` will get an IP address on subnet `172.18+<olt>.<onu>.0/24`. Make sure that eth0 inside the RG container has an IP address on the proper subnet:
```bash
$ ifconfig eth0
@@ -571,7 +555,7 @@
### Ping the emulated BNG
-The emulated BNG has an IP address of 172.18.0.10. After successfully running dhclient you should be able to ping it from the RG.
+`rg<olt>-<onu>` pings `172.18+<olt>.<onu>.10` as its BNG.
```bash
$ ping -c 3 172.18.0.10
diff --git a/profiles/seba/workflows/att-install.md b/profiles/seba/workflows/att-install.md
index e1182de..3a80da0 100644
--- a/profiles/seba/workflows/att-install.md
+++ b/profiles/seba/workflows/att-install.md
@@ -5,7 +5,7 @@
## Install the `att-workflow` chart
```shell
-helm install -n att-workflow cord/att-workflow --version=1.0.2
+helm install -n att-workflow cord/att-workflow --version=1.2.4
```
> NOTE: if you have installed the `cord-platform` chart as a sum of its components,
diff --git a/quickstart/seba_quickstart.md b/quickstart/seba_quickstart.md
index 10a9cc7..1aa43c0 100644
--- a/quickstart/seba_quickstart.md
+++ b/quickstart/seba_quickstart.md
@@ -6,26 +6,30 @@
## Install components as a whole
+The commands below will bring up the SEBA 2.0-alpha release
+
```shell
# Add the CORD repository and update indexes
helm repo add cord https://charts.opencord.org
helm repo update
# Install the CORD platform
-helm install -n cord-platform --version 6.1.0 cord/cord-platform
+helm install -n cord-platform --version 7.0.0 cord/cord-platform
# Wait until 3 etcd CRDs are present in Kubernetes
kubectl get crd | grep -i etcd | wc -l
# Install the SEBA profile
-helm install -n seba --version 1.0.0 cord/seba
+helm install -n seba --version 2.0.0-alpha1 cord/seba
# Install the AT&T workflow
-helm install -n att-workflow --version 1.0.2 cord/att-workflow
+helm install -n att-workflow --version 1.2.4 cord/att-workflow
```
## Alternatively, install as separate components
+The commands below will bring up SEBA using the latest released versions of the Helm charts. The resulting system should work but it may not correspond to an official SEBA release.
+
```shell
# Add the official Kubernetes incubator repostiory (for Kafka) and update the indexes
helm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubator
@@ -63,12 +67,12 @@
kubectl get crd | grep -i etcd | wc -l
# Install the rest of the SEBA profile components
-helm install -n voltha cord/voltha
+helm install -n voltha --version 1.0.6 cord/voltha # Install VOLTHA 1.7.0
helm install -n seba-service cord/seba-services
helm install -n base-kubernetes cord/base-kubernetes
# Install the AT&T workflow
-helm install -n att-workflow --version 1.0.2 cord/att-workflow
+helm install -n att-workflow cord/att-workflow
```
## Verify your installation and next steps