Reorganization pass on Aether Docs
Change-Id: I0653109d6fe8d340278580ff5c7758ca264b512e
diff --git a/pronto_deployment_guide/connectivity_service_update.rst b/edge_deployment/connectivity_service_update.rst
similarity index 100%
rename from pronto_deployment_guide/connectivity_service_update.rst
rename to edge_deployment/connectivity_service_update.rst
diff --git a/pronto_deployment_guide/enb_installation.rst b/edge_deployment/enb_installation.rst
similarity index 100%
rename from pronto_deployment_guide/enb_installation.rst
rename to edge_deployment/enb_installation.rst
diff --git a/edge_deployment/fabric_switch_bootstrap.rst b/edge_deployment/fabric_switch_bootstrap.rst
new file mode 100644
index 0000000..bfac6b6
--- /dev/null
+++ b/edge_deployment/fabric_switch_bootstrap.rst
@@ -0,0 +1,168 @@
+..
+ SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
+ SPDX-License-Identifier: Apache-2.0
+
+Fabric Switch Bootstrap
+=======================
+
+The installation of the ONL OS image on the fabric switches uses the DHCP and
+HTTP server set up on the management server.
+
+The default image is downloaded during that installation process by the
+``onieboot`` role. Make changes to that roll and rerun the management playbook
+to download a newer switch image.
+
+Preparation
+-----------
+
+The switches have a single ethernet port that is shared between OpenBMC and
+ONL. Find out the MAC addresses for both of these ports and enter it into
+NetBox.
+
+Change boot mode to ONIE Rescue mode
+------------------------------------
+
+In order to reinstall an ONL image, you must change the ONIE bootloader to
+"Rescue Mode".
+
+Once the switch is powered on, it should retrieve an IP address on the OpenBMC
+interface with DHCP. OpenBMC uses these default credentials::
+
+ username: root
+ password: 0penBmc
+
+Login to OpenBMC with SSH::
+
+ $ ssh root@10.0.0.131
+ The authenticity of host '10.0.0.131 (10.0.0.131)' can't be established.
+ ECDSA key fingerprint is SHA256:...
+ Are you sure you want to continue connecting (yes/no)? yes
+ Warning: Permanently added '10.0.0.131' (ECDSA) to the list of known hosts.
+ root@10.0.0.131's password:
+ root@bmc:~#
+
+Using the Serial-over-LAN Console, enter ONL::
+
+ root@bmc:~# /usr/local/bin/sol.sh
+ You are in SOL session.
+ Use ctrl-x to quit.
+ -----------------------
+
+ root@onl:~#
+
+.. note::
+ If `sol.sh` is unresponsive, please try to restart the mainboard with::
+
+ root@onl:~# wedge_power.sh restart
+
+
+Change the boot mode to rescue mode with the command ``onl-onie-boot-mode
+rescue``, and reboot::
+
+ root@onl:~# onl-onie-boot-mode rescue
+ [1053033.768512] EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
+ [1053033.936893] EXT4-fs (sda3): re-mounted. Opts: (null)
+ [1053033.996727] EXT4-fs (sda3): re-mounted. Opts: (null)
+ The system will boot into ONIE rescue mode at the next restart.
+ root@onl:~# reboot
+
+At this point, ONL will go through it's shutdown sequence and ONIE will start.
+If it does not start right away, press the Enter/Return key a few times - it
+may show you a boot selection screen. Pick ``ONIE`` and ``Rescue`` if given a
+choice.
+
+Installing an ONL image over HTTP
+---------------------------------
+
+Now that the switch is in Rescue mode
+
+First, activate the Console by pressing Enter::
+
+ discover: Rescue mode detected. Installer disabled.
+
+ Please press Enter to activate this console.
+ To check the install status inspect /var/log/onie.log.
+ Try this: tail -f /var/log/onie.log
+
+ ** Rescue Mode Enabled **
+ ONIE:/ #
+
+Then run the ``onie-nos-install`` command, with the URL of the management
+server on the management network segment::
+
+ ONIE:/ # onie-nos-install http://10.0.0.129/onie-installer
+ discover: Rescue mode detected. No discover stopped.
+ ONIE: Unable to find 'Serial Number' TLV in EEPROM data.
+ Info: Fetching http://10.0.0.129/onie-installer ...
+ Connecting to 10.0.0.129 (10.0.0.129:80)
+ installer 100% |*******************************| 322M 0:00:00 ETA
+ ONIE: Executing installer: http://10.0.0.129/onie-installer
+ installer: computing checksum of original archive
+ installer: checksum is OK
+ ...
+
+The installation will now start, and then ONL will boot culminating in::
+
+ Open Network Linux OS ONL-wedge100bf-32qs, 2020-11-04.19:44-64100e9
+
+ localhost login:
+
+The default ONL login is::
+
+ username: root
+ password: onl
+
+If you login, you can verify that the switch is getting it's IP address via
+DHCP::
+
+ root@localhost:~# ip addr
+ ...
+ 3: ma1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
+ link/ether 00:90:fb:5c:e1:97 brd ff:ff:ff:ff:ff:ff
+ inet 10.0.0.130/25 brd 10.0.0.255 scope global ma1
+ ...
+
+
+Post-ONL Configuration
+----------------------
+
+A ``terraform`` user must be created on the switches to allow them to be
+configured.
+
+This is done using Ansible. Verify that your inventory (Created earlier from the
+``inventory/example-aether.ini`` file) includes an ``[aetherfabric]`` section
+that has all the names and IP addresses of the compute nodes in it.
+
+Then run a ping test::
+
+ ansible -i inventory/sitename.ini -m ping aetherfabric
+
+This may fail with the error::
+
+ "msg": "Using a SSH password instead of a key is not possible because Host Key checking is enabled and sshpass does not support this. Please add this host's fingerprint to your known_hosts file to manage this host."
+
+Comment out the ``ansible_ssh_pass="onl"`` line, then rerun the ping test. It
+may ask you about authorized keys - answer ``yes`` for each host to trust the
+keys::
+
+ The authenticity of host '10.0.0.138 (<no hostip for proxy command>)' can't be established.
+ ECDSA key fingerprint is SHA256:...
+ Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
+
+Once you've trusted the host keys, the ping test should succeed::
+
+ spine1.role1.site | SUCCESS => {
+ "changed": false,
+ "ping": "pong"
+ }
+ leaf1.role1.site | SUCCESS => {
+ "changed": false,
+ "ping": "pong"
+ }
+ ...
+
+Then run the playbook to create the ``terraform`` user::
+
+ ansible-playbook -i inventory/sitename.ini playbooks/aetherfabric-playbook.yml
+
+Once completed, the switch should now be ready for TOST runtime install.
diff --git a/pronto_deployment_guide/images/enb-ser-1.jpg b/edge_deployment/images/enb-ser-1.jpg
similarity index 100%
rename from pronto_deployment_guide/images/enb-ser-1.jpg
rename to edge_deployment/images/enb-ser-1.jpg
Binary files differ
diff --git a/pronto_deployment_guide/images/enb-ser-2.png b/edge_deployment/images/enb-ser-2.png
similarity index 100%
rename from pronto_deployment_guide/images/enb-ser-2.png
rename to edge_deployment/images/enb-ser-2.png
Binary files differ
diff --git a/pronto_deployment_guide/images/enb-ser-3.png b/edge_deployment/images/enb-ser-3.png
similarity index 100%
rename from pronto_deployment_guide/images/enb-ser-3.png
rename to edge_deployment/images/enb-ser-3.png
Binary files differ
diff --git a/pronto_deployment_guide/images/enb-ser-4.png b/edge_deployment/images/enb-ser-4.png
similarity index 100%
rename from pronto_deployment_guide/images/enb-ser-4.png
rename to edge_deployment/images/enb-ser-4.png
Binary files differ
diff --git a/pronto_deployment_guide/images/enb-ser-5.png b/edge_deployment/images/enb-ser-5.png
similarity index 100%
rename from pronto_deployment_guide/images/enb-ser-5.png
rename to edge_deployment/images/enb-ser-5.png
Binary files differ
diff --git a/pronto_deployment_guide/images/enb-ser-6.png b/edge_deployment/images/enb-ser-6.png
similarity index 100%
rename from pronto_deployment_guide/images/enb-ser-6.png
rename to edge_deployment/images/enb-ser-6.png
Binary files differ
diff --git a/pronto_deployment_guide/images/enb-ser-7.png b/edge_deployment/images/enb-ser-7.png
similarity index 100%
rename from pronto_deployment_guide/images/enb-ser-7.png
rename to edge_deployment/images/enb-ser-7.png
Binary files differ
diff --git a/pronto_deployment_guide/images/enb-ser-8.png b/edge_deployment/images/enb-ser-8.png
similarity index 100%
rename from pronto_deployment_guide/images/enb-ser-8.png
rename to edge_deployment/images/enb-ser-8.png
Binary files differ
diff --git a/pronto_deployment_guide/images/jenkins-onos-params.png b/edge_deployment/images/jenkins-onos-params.png
similarity index 100%
rename from pronto_deployment_guide/images/jenkins-onos-params.png
rename to edge_deployment/images/jenkins-onos-params.png
Binary files differ
diff --git a/pronto_deployment_guide/images/mgmtsrv-000.png b/edge_deployment/images/mgmtsrv-000.png
similarity index 100%
rename from pronto_deployment_guide/images/mgmtsrv-000.png
rename to edge_deployment/images/mgmtsrv-000.png
Binary files differ
diff --git a/pronto_deployment_guide/images/mgmtsrv-001.png b/edge_deployment/images/mgmtsrv-001.png
similarity index 100%
rename from pronto_deployment_guide/images/mgmtsrv-001.png
rename to edge_deployment/images/mgmtsrv-001.png
Binary files differ
diff --git a/pronto_deployment_guide/images/mgmtsrv-002.png b/edge_deployment/images/mgmtsrv-002.png
similarity index 100%
rename from pronto_deployment_guide/images/mgmtsrv-002.png
rename to edge_deployment/images/mgmtsrv-002.png
Binary files differ
diff --git a/pronto_deployment_guide/images/mgmtsrv-003.png b/edge_deployment/images/mgmtsrv-003.png
similarity index 100%
rename from pronto_deployment_guide/images/mgmtsrv-003.png
rename to edge_deployment/images/mgmtsrv-003.png
Binary files differ
diff --git a/pronto_deployment_guide/images/mgmtsrv-004.png b/edge_deployment/images/mgmtsrv-004.png
similarity index 100%
rename from pronto_deployment_guide/images/mgmtsrv-004.png
rename to edge_deployment/images/mgmtsrv-004.png
Binary files differ
diff --git a/pronto_deployment_guide/images/mgmtsrv-005.png b/edge_deployment/images/mgmtsrv-005.png
similarity index 100%
rename from pronto_deployment_guide/images/mgmtsrv-005.png
rename to edge_deployment/images/mgmtsrv-005.png
Binary files differ
diff --git a/pronto_deployment_guide/images/mgmtsrv-006.png b/edge_deployment/images/mgmtsrv-006.png
similarity index 100%
rename from pronto_deployment_guide/images/mgmtsrv-006.png
rename to edge_deployment/images/mgmtsrv-006.png
Binary files differ
diff --git a/pronto_deployment_guide/images/mgmtsrv-007.png b/edge_deployment/images/mgmtsrv-007.png
similarity index 100%
rename from pronto_deployment_guide/images/mgmtsrv-007.png
rename to edge_deployment/images/mgmtsrv-007.png
Binary files differ
diff --git a/pronto_deployment_guide/images/pronto_logical_diagram.svg b/edge_deployment/images/pronto_logical_diagram.svg
similarity index 100%
rename from pronto_deployment_guide/images/pronto_logical_diagram.svg
rename to edge_deployment/images/pronto_logical_diagram.svg
diff --git a/pronto_deployment_guide/images/pswi-000.png b/edge_deployment/images/pswi-000.png
similarity index 100%
rename from pronto_deployment_guide/images/pswi-000.png
rename to edge_deployment/images/pswi-000.png
Binary files differ
diff --git a/pronto_deployment_guide/images/pswi-001.png b/edge_deployment/images/pswi-001.png
similarity index 100%
rename from pronto_deployment_guide/images/pswi-001.png
rename to edge_deployment/images/pswi-001.png
Binary files differ
diff --git a/pronto_deployment_guide/images/pswi-003.png b/edge_deployment/images/pswi-003.png
similarity index 100%
rename from pronto_deployment_guide/images/pswi-003.png
rename to edge_deployment/images/pswi-003.png
Binary files differ
diff --git a/pronto_deployment_guide/images/pswi-004.png b/edge_deployment/images/pswi-004.png
similarity index 100%
rename from pronto_deployment_guide/images/pswi-004.png
rename to edge_deployment/images/pswi-004.png
Binary files differ
diff --git a/pronto_deployment_guide/images/pswi-005.png b/edge_deployment/images/pswi-005.png
similarity index 100%
rename from pronto_deployment_guide/images/pswi-005.png
rename to edge_deployment/images/pswi-005.png
Binary files differ
diff --git a/edge_deployment/management_net_bootstrap.rst b/edge_deployment/management_net_bootstrap.rst
new file mode 100644
index 0000000..499e5b1
--- /dev/null
+++ b/edge_deployment/management_net_bootstrap.rst
@@ -0,0 +1,63 @@
+..
+ SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
+ SPDX-License-Identifier: Apache-2.0
+
+Management Network Bootstrap
+============================
+
+The current Pronto deployment uses an HP/Aruba 2540 24G PoE+ 4SFP+ JL356A
+switch to run the management network and other VLAN's that are used internally.
+
+By default the switch will pull an IP address via DHCP and ``http://<switch IP>``
+will display a management webpage for the switch. You need to be able to access
+this webpage before you can update the configuration.
+
+Loading the Management Switch Configuration
+-------------------------------------------
+
+1. Obtain a copy of the Management switch configuration file (this ends in ``.pcc``).
+
+2. Open the switch web interface at ``http://<switch IP>``. You may be
+ prompted to login - the default credentials are both ``admin``:
+
+ .. image:: images/pswi-000.png
+ :alt: User Login for switch
+ :scale: 50%
+
+3. Go to the "Management" section at bottom left:
+
+ .. image:: images/pswi-001.png
+ :alt: Update upload
+ :scale: 50%
+
+ In the "Update" section at left, drag the configuration file into the upload
+ area, or click Browse and select it.
+
+4. In the "Select switch configuration file to update" section, select
+ "config1", so it overwrites the default configuration.
+
+5. In the "Select switch configuration file to update" section, select
+ "config1", so it overwrites the default configuration. Click "Update".
+ You'll be prompted to reboot the switch, which you can do with the power
+ symbol at top right. You may be prompted to select an image used to reboot -
+ the "Previously Selected" is the correct one to use:
+
+ .. image:: images/pswi-003.png
+ :alt: Switch Image Select
+ :scale: 30%
+
+6. Wait for the switch to reboot:
+
+ .. image:: images/pswi-004.png
+ :alt: Switch Reboot
+ :scale: 50%
+
+ The switch is now configured with the correct VLANs for Pronto Use. If you
+ go to Interfaces > VLANs should see a list of VLANs configured on the
+ switch:
+
+ .. image:: images/pswi-005.png
+ :alt: Mgmt VLANs
+ :scale: 50%
+
+
diff --git a/pronto_deployment_guide/overview.rst b/edge_deployment/overview.rst
similarity index 100%
rename from pronto_deployment_guide/overview.rst
rename to edge_deployment/overview.rst
diff --git a/pronto_deployment_guide/run_time_deployment.rst b/edge_deployment/runtime_deployment.rst
similarity index 84%
rename from pronto_deployment_guide/run_time_deployment.rst
rename to edge_deployment/runtime_deployment.rst
index 513f1fb..c97f55d 100644
--- a/pronto_deployment_guide/run_time_deployment.rst
+++ b/edge_deployment/runtime_deployment.rst
@@ -2,16 +2,20 @@
SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
SPDX-License-Identifier: Apache-2.0
-==========================
-Aether Run-Time Deployment
-==========================
-This section describes how to install Aether edge runtime and Aether managed applications.
-We will be using GitOps based Aether CD pipeline for this,
-so we just need to create a patch to **aether-pod-configs** repository.
+Aether Runtime Deployment
+=========================
+
+This section describes how to install Aether edge runtime and Aether managed
+applications.
+
+We will be using GitOps based Aether CD pipeline for this, so we just need to
+create a patch to **aether-pod-configs** repository.
Download aether-pod-configs repository
-======================================
-Download aether-pod-configs repository if you don't have it already in your develop machine.
+--------------------------------------
+
+Download the ``aether-pod-configs`` repository if you don't have it already in
+your development machine.
.. code-block:: shell
@@ -19,7 +23,8 @@
$ git clone "ssh://[username]@gerrit.opencord.org:29418/aether-pod-configs"
Update global resource maps
-===========================
+---------------------------
+
.. attention::
Skip this section if you have already done the same step in the
@@ -30,9 +35,10 @@
* user_map.tfvars
* cluster_map.tfvars
-As a note, you can find several other global resource maps under the `production` directory.
-Resource definitions that need to be shared among clusters or are better managed in a
-single file to avoid configuration conflicts are maintained in this way.
+As a note, you can find several other global resource maps under the
+`production` directory. Resource definitions that need to be shared among
+clusters or are better managed in a single file to avoid configuration
+conflicts are maintained in this way.
.. code-block:: diff
@@ -98,9 +104,13 @@
$ git review
Create runtime configurations
-=============================
-In this step, we will add several Terraform configurations and overriding values for the managed applications.
-Run the following commands to auto-generate necessary files under the target ACE directory.
+-----------------------------
+
+In this step, we will add several Terraform configurations and overriding
+values for the managed applications.
+
+Run the following commands to auto-generate necessary files under the target
+ACE directory.
.. code-block:: shell
@@ -124,7 +134,8 @@
Created ../production/ace-test/app_values/omec-upf-pfcp-agent.yml
Create a review request
-=======================
+-----------------------
+
.. code-block:: shell
$ cd $WORKDIR/aether-pod-configs
diff --git a/edge_deployment/server_bootstrap.rst b/edge_deployment/server_bootstrap.rst
new file mode 100644
index 0000000..c7fe5e1
--- /dev/null
+++ b/edge_deployment/server_bootstrap.rst
@@ -0,0 +1,293 @@
+..
+ SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
+ SPDX-License-Identifier: Apache-2.0
+
+Server Bootstrap
+================
+
+Management Server Bootstrap
+"""""""""""""""""""""""""""
+
+The management server is bootstrapped into a customized version of the standard
+Ubuntu 18.04 OS installer.
+
+The `iPXE boot firmware <https://ipxe.org/>`_. is used to start this process
+and is built using the steps detailed in the `ipxe-build
+<https://gerrit.opencord.org/plugins/gitiles/ipxe-build>`_. repo, which
+generates both USB and PXE chainloadable boot images.
+
+Once a system has been started using these images started, these images will
+download a customized script from an external webserver to continue the boot
+process. This iPXE to webserver connection is secured with mutual TLS
+authentication, enforced by the nginx webserver.
+
+The iPXE scripts are created by the `pxeboot
+<https://gerrit.opencord.org/plugins/gitiles/ansible/role/pxeboot>`_ role,
+which creates both a boot menu, downloads the appropriate binaries for
+bootstrapping an OS installation, and creates per-node installation preseed files.
+
+The preseed files contain configuration steps to install the OS from the
+upstream Ubuntu repos, as well as customization of packages and creating the
+``onfadmin`` user.
+
+Creating a bootable USB drive
+'''''''''''''''''''''''''''''
+
+1. Get a USB key. Can be tiny as the uncompressed image is floppy sized
+ (1.4MB). Download the USB image file (``<date>_onf_ipxe.usb.zip``) on the
+ system you're using to write the USB key, and unzip it.
+
+2. Put a USB key in the system you're using to create the USB key, then
+ determine which USB device file it's at in ``/dev``. You might look at the
+ end of the ``dmesg`` output on Linux/Unix or the output of ``diskutil
+ list`` on macOS.
+
+ Be very careful about this, as if you accidentally overwrite some other disk in
+ your system that would be highly problematic.
+
+3. Write the image to the device::
+
+ $ dd if=/path/to/20201116_onf_ipxe.usb of=/dev/sdg
+ 2752+0 records in
+ 2752+0 records out
+ 1409024 bytes (1.4 MB, 1.3 MiB) copied, 2.0272 s, 695 kB/s
+
+ You may need to use `sudo` for this.
+
+Boot and Image Management Server
+''''''''''''''''''''''''''''''''
+
+1. Connect a USB keyboard and VGA monitor to the management node. Put the USB
+ Key in one of the management node's USB ports (port 2 or 3):
+
+ .. image:: images/mgmtsrv-000.png
+ :alt: Management Server Ports
+ :scale: 50%
+
+2. Turn on the management node, and press the F11 key as it starts to get into
+ the Boot Menu:
+
+ .. image:: images/mgmtsrv-001.png
+ :alt: Management Server Boot Menu
+ :scale: 50%
+
+3. Select the USB key (in this case "PNY USB 2.0", your options may vary) and press return. You should see iPXE load:
+
+ .. image:: images/mgmtsrv-002.png
+ :alt: iPXE load
+ :scale: 50%
+
+4. A menu will appear which displays the system information and DHCP discovered
+ network settings (your network must provide the IP address to the management
+ server via DHCP):
+
+ Use the arrow keys to select "Ubuntu 18.04 Installer (fully automatic)":
+
+ .. image:: images/mgmtsrv-003.png
+ :alt: iPXE Menu
+ :scale: 50%
+
+ There is a 10 second default timeout if left untouched (it will continue the
+ system boot process) so restart the system if you miss the 10 second window.
+
+5. The Ubuntu 18.04 installer will be downloaded and booted:
+
+ .. image:: images/mgmtsrv-004.png
+ :alt: Ubuntu Boot
+ :scale: 50%
+
+6. Then the installer starts and takes around 10 minutes to install (depends on
+ your connection speed):
+
+ .. image:: images/mgmtsrv-005.png
+ :alt: Ubuntu Install
+ :scale: 50%
+
+
+7. At the end of the install, the system will restart and present you with a
+ login prompt:
+
+ .. image:: images/mgmtsrv-006.png
+ :alt: Ubuntu Install Complete
+ :scale: 50%
+
+
+Management Server Configuration
+'''''''''''''''''''''''''''''''
+
+Once the OS is installed on the management server, Ansible is used to remotely
+install software on the management server.
+
+To checkout the ONF ansible repo and enter the virtualenv with the tooling::
+
+ mkdir infra
+ cd infra
+ repo init -u ssh://<your gerrit username>@gerrit.opencord.org:29418/infra-manifest
+ repo sync
+ cd ansible
+ make galaxy
+ source venv_onfansible/bin/activate
+
+Obtain the ``undionly.kpxe`` iPXE artifact for bootstrapping the compute
+servers, and put it in the ``playbook/files`` directory.
+
+Next, create an inventory file to access the NetBox API. An example is given
+in ``inventory/example-netbox.yml`` - duplicate this file and modify it. Fill
+in the ``api_endpoint`` address and ``token`` with an API key you get out of
+the NetBox instance. List the IP Prefixes used by the site in the
+``ip_prefixes`` list.
+
+Next, run the ``scripts/netbox_edgeconfig.py`` to generate a host_vars file for
+the management server. Assuming that the management server in the edge is
+named ``mgmtserver1.stage1.menlo``, you'd run::
+
+ python scripts/netbox_edgeconfig.py inventory/my-netbox.yml > inventory/host_vars/mgmtserver1.stage1.menlo.yml
+
+One manual change needs to be made to this output - edit the
+``inventory/host_vars/mgmtserver1.stage1.menlo.yml`` file and add the following
+to the bottom of the file, replacing the IP addresses with the management
+server IP address for each segment.
+
+In the case of the Fabric that has two leaves and IP ranges, add the Management
+server IP address used for the leaf that it is connected to, and then add a
+route for the other IP address range for the non-Management-connected leaf that
+is via the Fabric router address in the connected leaf range.
+
+This configures the `netplan <https://netplan.io>`_ on the management server,
+and creates a SNAT rule for the UE range route, and will be automated away
+soon::
+
+ # added manually
+ netprep_netplan:
+ ethernets:
+ eno2:
+ addresses:
+ - 10.0.0.1/25
+ vlans:
+ mgmt800:
+ id: 800
+ link: eno2
+ addresses:
+ - 10.0.0.129/25
+ fabr801:
+ id: 801
+ link: eno2
+ addresses:
+ - 10.0.1.129/25
+ routes:
+ - to: 10.0.1.0/25
+ via: 10.0.1.254
+ metric: 100
+
+ netprep_nftables_nat_postrouting: >
+ ip saddr 10.0.1.0/25 ip daddr 10.168.0.0/20 counter snat to 10.0.1.129;
+
+
+Using the ``inventory/example-aether.ini`` as a template, create an
+:doc:`ansible inventory <ansible:user_guide/intro_inventory>` file for the
+site. Change the device names, IP addresses, and ``onfadmin`` password to match
+the ones for this site. The management server's configuration is in the
+``[aethermgmt]`` and corresponding ``[aethermgmt:vars]`` section.
+
+Then, to configure a management server, run::
+
+ ansible-playbook -i inventory/sitename.ini playbooks/aethermgmt-playbook.yml
+
+This installs software with the following functionality:
+
+- VLANs on second Ethernet port to provide connectivity to the rest of the pod.
+- Firewall with NAT for routing traffic
+- DHCP and TFTP for bootstrapping servers and switches
+- DNS for host naming and identification
+- HTTP server for serving files used for bootstrapping switches
+- Downloads the Tofino switch image
+- Creates user accounts for administrative access
+
+Compute Server Bootstrap
+""""""""""""""""""""""""
+
+Once the management server has finished installation, it will be set to offer
+the same iPXE bootstrap file to the computer.
+
+Each node will be booted, and when iPXE loads select the ``Ubuntu 18.04
+Installer (fully automatic)`` option.
+
+The nodes can be controlled remotely via their BMC management interfaces - if
+the BMC is at ``10.0.0.3`` a remote user can SSH into them with::
+
+ ssh -L 2443:10.0.0.3:443 onfadmin@<mgmt server ip>
+
+And then use their web browser to access the BMC at::
+
+ https://localhost:2443
+
+The default BMC credentials for the Pronto nodes are::
+
+ login: ADMIN
+ password: Admin123
+
+The BMC will also list all of the MAC addresses for the network interfaces
+(including BMC) that are built into the logic board of the system. Add-in
+network cards like the 40GbE ones used in compute servers aren't listed.
+
+To prepare the compute nodes, software must be installed on them. As they
+can't be accessed directly from your local system, a :ref:`jump host
+<ansible:use_ssh_jump_hosts>` configuration is added, so the SSH connection
+goes through the management server to the compute systems behind it. Doing this
+requires a few steps:
+
+First, configure SSH to use Agent forwarding - create or edit your
+``~/.ssh/config`` file and add the following lines::
+
+ Host <management server IP>
+ ForwardAgent yes
+
+Then try to login to the management server, then the compute node::
+
+ $ ssh onfadmin@<management server IP>
+ Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-54-generic x86_64)
+ ...
+ onfadmin@mgmtserver1:~$ ssh onfadmin@10.0.0.138
+ Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-54-generic x86_64)
+ ...
+ onfadmin@node2:~$
+
+Being able to login to the compute nodes from the management node means that
+SSH Agent forwarding is working correctly.
+
+Verify that your inventory (Created earlier from the
+``inventory/example-aether.ini`` file) includes an ``[aethercompute]`` section
+that has all the names and IP addresses of the compute nodes in it.
+
+Then run a ping test::
+
+ ansible -i inventory/sitename.ini -m ping aethercompute
+
+It may ask you about authorized keys - answer ``yes`` for each host to trust the keys::
+
+ The authenticity of host '10.0.0.138 (<no hostip for proxy command>)' can't be established.
+ ECDSA key fingerprint is SHA256:...
+ Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
+
+You should then see a success message for each host::
+
+ node1.stage1.menlo | SUCCESS => {
+ "changed": false,
+ "ping": "pong"
+ }
+ node2.stage1.menlo | SUCCESS => {
+ "changed": false,
+ "ping": "pong"
+ }
+ ...
+
+Once you've seen this, run the playbook to install the prerequisites (Terraform
+user, Docker)::
+
+ ansible-playbook -i inventory/sitename.ini playbooks/aethercompute-playbook.yml
+
+Note that Docker is quite large and may take a few minutes for installation
+depending on internet connectivity.
+
+Now that these compute nodes have been brought up, the rest of the installation
+can continue.
diff --git a/edge_deployment/site_planning.rst b/edge_deployment/site_planning.rst
new file mode 100644
index 0000000..4af68f7
--- /dev/null
+++ b/edge_deployment/site_planning.rst
@@ -0,0 +1,298 @@
+..
+ SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
+ SPDX-License-Identifier: Apache-2.0
+
+Site Planning
+=============
+
+Site Design in Netbox
+---------------------
+
+Once the hardware has been ordered, the installation can be planned. The
+following information needs to be added to `NetBox
+<https://netbox.readthedocs.io/en/stable>`_ to describe each edge site:
+
+1. Add a Site for the edge (if one doesn't already exist), which has the
+ physical location and contact information for the edge.
+
+2. Add equipment Racks to the Site (if they don't already exist).
+
+3. Add a Tenant for the edge (who owns/manages it), assigned to the ``Pronto``
+ or ``Aether`` Tenant Group.
+
+4. Add a VRF (Routing Table) for the edge site. This is usually just the name
+ of the site. Make sure that ``Enforce unique space`` is checked, so that IP
+ addresses within the VRF are forced to be unique, and that the Tenant Group
+ and Tenant are set.
+
+5. Add a VLAN Group to the edge site, which groups the site's VLANs and
+ requires that they have a unique VLAN number.
+
+6. Add VLANs for the edge site. These should be assigned a VLAN Group, the
+ Site, and Tenant.
+
+ There can be multiple of the same VLAN in NetBox (VLANs are layer 2, and
+ local to the site), but not within the VLAN group.
+
+ The minimal list of VLANs:
+
+ * ADMIN 1
+ * UPLINK 10
+ * MGMT 800
+ * FAB 801
+
+ If you have multiple deployments at a site using the same management server,
+ add additional VLANs incremented by 10 for the MGMT/FAB - for example:
+
+ * DEVMGMT 810
+ * DEVFAB 801
+
+7. Add IP Prefixes for the site. This should have the Tenant and VRF assigned.
+
+ All edge IP prefixes fit into a ``/22`` sized block.
+
+ The description of the Prefix contains the DNS suffix for all Devices that
+ have IP addresses within this Prefix. The full DNS names are generated by
+ combining the first ``<devname>`` component of the Device names with this
+ suffix.
+
+ An examples using the ``10.0.0.0/22`` block. There are 4 edge
+ prefixes, with the following purposes:
+
+ * ``10.0.0.0/25``
+
+ * Has the Server BMC/LOM and Management Switch
+ * Assign the ADMIN 1 VLAN
+ * Set the description to ``admin.<deployment>.<site>.aetherproject.net`` (or
+ ``prontoproject.net``).
+
+ * ``10.0.0.128/25``
+
+ * Has the Server Management plane, Fabric Switch Management/BMC
+ * Assign MGMT 800 VLAN
+ * Set the description to ``<deployment>.<site>.aetherproject.net`` (or
+ ``prontoproject.net``).
+
+ * ``10.0.1.0/25``
+
+ * IP addresses of the qsfp0 port of the Compute Nodes to Fabric switches, devices
+ connected to the Fabric like the eNB
+ * Assign FAB 801 VLAN
+ * Set the description to ``fab1.<deployment>.<site>.aetherproject.net`` (or
+ ``prontoproject.net``).
+
+ * ``10.0.1.128/25``
+
+ * IP addresses of the qsfp1 port of the Compute Nodes to fabric switches
+ * Assign FAB 801 VLAN
+ * Set the description to ``fab2.<deployment>.<site>.aetherproject.net`` (or
+ ``prontoproject.net``).
+
+ There also needs to be a parent range of the two fabric ranges added:
+
+ * ``10.0.1.0/24``
+
+ * This is used to configure the correct routes, DNS, and TFTP servers
+ provided by DHCP to the equipment that is connected to the fabric
+ leaf switch that the management server (which provides those
+ services) is not connected to.
+
+ Additionally, these edge prefixes are used for Kubernetes but don't need to
+ be created in NetBox:
+
+ * ``10.0.2.0/24``
+
+ * Kubernetes Pod IP's
+
+ * ``10.0.3.0/24``
+
+ * Kubernetes Cluster IP's
+
+8. Add Devices to the site, for each piece of equipment. These are named with a
+ scheme similar to the DNS names used for the pod, given in this format::
+
+ <devname>.<deployment>.<site>
+
+ Examples::
+
+ mgmtserver1.ops1.tucson
+ node1.stage1.menlo
+
+ Note that these names are transformed into DNS names using the Prefixes, and
+ may have additional components - ``admin`` or ``fabric`` may be added after
+ the ``<devname>`` for devices on those networks.
+
+ Set the following fields when creating a device:
+
+ * Site
+ * Tenant
+ * Rack & Rack Position
+ * Serial number
+
+ If a specific Device Type doesn't exist for the device, it must be created,
+ which is detailed in the NetBox documentation, or ask the OPs team for help.
+
+ See `Rackmount of Equipment`_ below for guidance on how equipment should be
+ mounted in the Rack.
+
+9. Add Services to the management server:
+
+ * name: ``dns``
+ protocol: UDP
+ port: 53
+
+ * name: ``tftp``
+ protocol: UDP
+ port: 69
+
+ These are used by the DHCP and DNS config to know which servers offer
+ DNS or TFTP service.
+
+10. Set the MAC address for the physical interfaces on the device.
+
+ You may also need to add physical network interfaces if aren't already
+ created by the Device Type. An example would be if additional add-in
+ network cards were installed.
+
+11. Add any virtual interfaces to the Devices. When creating a virtual
+ interface, it should have it's ``label`` field set to the physical network
+ interface that it is assigned
+
+ These are needed are two cases for the Pronto deployment:
+
+ 1. On the Management Server, there should bet (at least) two VLAN
+ interfaces created attached to the ``eno2`` network port, which
+ are used to provide connectivity to the management plane and fabric.
+ These should be named ``<name of vlan><vlan ID>``, so the MGMT 800 VLAN
+ would become a virtual interface named ``mgmt800``, with the label
+ ``eno2``.
+
+ 2. On the Fabric switches, the ``eth0`` port is shared between the OpenBMC
+ interface and the ONIE/ONL installation. Add a ``bmc`` virtual
+ interface with a label of ``eth0`` on each fabric switch, and check the
+ ``OOB Management`` checkbox.
+
+12. Create IP addresses for the physical and virtual interfaces. These should
+ have the Tenant and VRF set.
+
+ The Management Server should always have the first IP address in each
+ range, and they should be incremental, in this order. Examples are given as
+ if there was a single instance of each device - adding additional devices
+ would increment the later IP addresses.
+
+ * Management Server
+
+ * ``eno1`` - site provided public IP address, or blank if DHCP
+ provided
+
+ * ``eno2`` - 10.0.0.1/25 (first of ADMIN) - set as primary IP
+ * ``bmc`` - 10.0.0.2/25 (next of ADMIN)
+ * ``mgmt800`` - 10.0.0.129/25 (first of MGMT)
+ * ``fab801`` - 10.0.1.1/25 (first of FAB)
+
+ * Management Switch
+
+ * ``gbe1`` - 10.0.0.3/25 (next of ADMIN) - set as primary IP
+
+ * Fabric Switch
+
+ * ``eth0`` - 10.0.0.130/25 (next of MGMT), set as primary IP
+ * ``bmc`` - 10.0.0.131/25
+
+ * Compute Server
+
+ * ``eth0`` - 10.0.0.132/25 (next of MGMT), set as primary IP
+ * ``bmc`` - 10.0.0.4/25 (next of ADMIN)
+ * ``qsfp0`` - 10.0.1.2/25 (next of FAB)
+ * ``qsfp1`` - 10.0.1.3/25
+
+ * Other Fabric devices (eNB, etc.)
+
+ * ``eth0`` or other primary interface - 10.0.1.4/25 (next of FAB)
+
+13. Add DHCP ranges to the IP Prefixes for IP's that aren't reserved. These are
+ done like any other IP Address, but with the ``Status`` field is set to
+ ``DHCP``, and they'll consume the entire range of IP addresses given in the
+ CIDR mask.
+
+ For example ``10.0.0.32/27`` as a DHCP block would take up 1/4 of the ADMIN
+ prefix.
+
+14. Add router IP reservations to the IP Prefix for both Fabric prefixes. These
+ are IP addresses used by ONOS to route traffic to the other leaf, and have
+ the following attributes:
+
+ - Have the last usable address in range (in the ``/25`` fabric examples
+ above, these would be ``10.0.1.126/25`` and ``10.0.1.254/25``)
+
+ - Have a ``Status`` of ``Reserved``, and the VRF, Tenant Group, and Tenant
+ set.
+
+ - The Description must start with the word ``router``, such as: ``router
+ for leaf1 Fabric``
+
+ - A custom field named ``RFC3442 Routes`` is set to the CIDR IP address of
+ the opposite leaf - if the leaf's prefix is ``10.0.1.0/25`` and the
+ router IP is ``10.0.1.126/25`` then ``RFC3442 Routes`` should be set to
+ ``10.0.1.128\25`` (and the reverse - on ``10.0.1.254/25`` the ``RFC3442
+ Routes`` would be set to be ``10.0.1.0/25``). This creates an `RFC3442
+ Classless Static Route Option <https://tools.ietf.org/html/rfc3442>`_
+ for the subnet in DHCP.
+
+15. Add Cables between physical interfaces on the devices
+
+ The topology needs to match the logical diagram presented in the
+ :ref:`network_cable_plan`. Note that many of the management interfaces
+ need to be located either on the MGMT or ADMIN VLANs, and the management
+ switch is
+ used to provide that separation.
+
+Rackmount of Equipment
+----------------------
+
+Most of the Pronto equipment has a 19" rackmount form factor.
+
+Guidelines for mounting this equipment:
+
+- The EdgeCore Wedge Switches have a front-to-back (aka "port-to-power") fan
+ configuration, so hot air exhaust is out the back of the switch near the
+ power inlets, away from the 32 QSFP network ports on the front of the switch.
+
+- The full-depth 1U and 2U Supermicro servers also have front-to-back airflow
+ but have most of their ports on the rear of the device.
+
+- Airflow through the rack should be in one direction to avoid heat being
+ pulled from one device into another. This means that to connect the QSFP
+ network ports from the servers to the switches, cabling should be routed
+ through the rack from front (switch) to back (server). Empty rack spaces
+ should be reserved for this purpose.
+
+- The short-depth management HP Switch and 1U Supermicro servers should be
+ mounted on the rear of the rack. They both don't generate an appreciable
+ amount of heat, so the airflow direction isn't a significant factor in
+ racking them.
+
+Inventory
+---------
+
+Once equipment arrives, any device needs to be recorded in inventory if it:
+
+1. Connects to the network (has a MAC address)
+2. Has a serial number
+3. Isn't a subcomponent (disk, add-in card, linecard, etc.) of a larger device.
+
+The following information should be recorded for every device:
+
+- Manufacturer
+- Model
+- Serial Number
+- MAC address (for the primary and any management/BMC/IPMI interfaces)
+
+This information should be be added to the corresponding Devices within the ONF
+NetBox instance. The accuracy of this information is very important as it is
+used in bootstrapping the compute systems, which is currently done by Serial
+Number, as reported to iPXE by SMBIOS.
+
+Once inventory has been completed, let the Infra team know, and the pxeboot
+configuration will be generated to have the OS preseed files corresponding to the
+new servers based on their serial numbers.
diff --git a/pronto_deployment_guide/tost_deployment.rst b/edge_deployment/tost_deployment.rst
similarity index 92%
rename from pronto_deployment_guide/tost_deployment.rst
rename to edge_deployment/tost_deployment.rst
index 97ae868..2ac0225 100644
--- a/pronto_deployment_guide/tost_deployment.rst
+++ b/edge_deployment/tost_deployment.rst
@@ -311,24 +311,32 @@
- **onos-netcfg.json** in **onos** folder
- Chassis config in **stratum** folder
- There should be one chassis config for each switch. The file name needs to be **${hostname}-chassis-config.pb.txt**
-- **telegraf.yaml** in **telegraf** folder need to be updated with all switch IP addresses
+ There should be one chassis config for each switch. The file name needs to be
+ **${hostname}-chassis-config.pb.txt**
+- **telegraf.yaml** in **telegraf** folder need to be updated with all switch
+ IP addresses
Double check these files and make sure they have been updated accordingly.
Create a review request
^^^^^^^^^^^^^^^^^^^^^^^
-We also need to create a gerrit review request, similar to what we have done in the **Aether Run-Time Deployment**.
-Please refer to :doc:`Aether Run-Time Deployment <run_time_deployment>` to create a review request.
+We also need to create a gerrit review request, similar to what we have done in
+the **Aether Runtime Deployment**.
+
+Please refer to :doc:`Aether Runtime Deployment <runtime_deployment>` to
+create a review request.
Create TOST deployment job in Jenkins
=====================================
-There are three major components in the Jenkins system, the Jenkins pipeline and Jenkins Job Builder and Jenkins Job.
+There are three major components in the Jenkins system, the Jenkins pipeline
+and Jenkins Job Builder and Jenkins Job.
-We follow the Infrastructure as Code principle to place three major components in a Git repo, **aether-ci-management**
-Download **aether-ci-management** repository .
+We follow the Infrastructure as Code principle to place three major components
+in a Git repo, ``aether-ci-management``
+
+Download the ``aether-ci-management`` repository.
.. code-block:: shell
@@ -336,7 +344,8 @@
$ git clone "ssh://[username]@gerrit.opencord.org:29418/aether-ci-management"
-Here is the example of folder structure, we put everything related to three major components under the jjb folder.
+Here is the example of folder structure, we put everything related to three
+major components under the jjb folder.
.. code-block:: console
@@ -356,21 +365,27 @@
Jenkins pipeline
^^^^^^^^^^^^^^^^
-Jenkins pipeline runs the Terraform scripts to install desired applications into the specified Kubernetes cluster.
+Jenkins pipeline runs the Terraform scripts to install desired applications
+into the specified Kubernetes cluster.
-Both ONOS and Stratum will read configuration files (network config, chassis config) from aether-pod-config.
-The default git branch is master.
-For testing purpose, we also provide two parameters to specify the number of reviews and patchset.
+Both ONOS and Stratum will read configuration files (network config, chassis
+config) from aether-pod-config.
+
+The default git branch is master. For testing purpose, we also provide two
+parameters to specify the number of reviews and patchset.
+
We will explain more in the next section.
.. note::
Currently, we don’t perform the incremental upgrade for TOST application.
Instead, we perform the clean installation.
- In the pipeline script, Terraform will destroy all existing resources and then create them again.
+ In the pipeline script, Terraform will destroy all existing resources and
+ then create them again.
-We put all pipeline scripts under the pipeline directory, the language of the pipeline script is groovy.
+We put all pipeline scripts under the pipeline directory, the language of the
+pipeline script is groovy.
.. code-block:: console
@@ -399,9 +414,10 @@
3. tost-telegraf.groovy
4. tost.groovy
-tost-[onos/stratum/telegraf].groovy are used to deploy the individual application respectively,
-and tost.groovy is a high level script, used to deploy the TOST application, it will execute
-the above three scripts in its pipeline script.
+tost-[onos/stratum/telegraf].groovy are used to deploy the individual
+application respectively, and tost.groovy is a high level script, used to
+deploy the TOST application, it will execute the above three scripts in its
+pipeline script.
Jenkins jobs
@@ -415,9 +431,13 @@
- Source code management
We created one Jenkins job for each TOST component, per Aether edge.
-We have four Jenkins jobs (HostPath provisioner, ONOS, Stratum and Telegraf) for each edge as of today.
-There are 10+ parameters in Jenkins jobs and they can be divided into two parts, cluster-level and application-level.
+We have four Jenkins jobs (HostPath provisioner, ONOS, Stratum and Telegraf)
+for each edge as of today.
+
+There are 10+ parameters in Jenkins jobs and they can be divided into two
+parts, cluster-level and application-level.
+
Here is an example of supported parameters.
.. image:: images/jenkins-onos-params.png
@@ -430,12 +450,13 @@
the config for aether-pod-configs repo from a specified gerrit review, instead of the
HEAD branch. It’s good for developer to test its change before merge.
- **onos_user**: used to login ONOS controller
-- **git_repo/git_server/git_user/git_password_env**: information of git repository, **git_password_env** is a key for
- Jenkins Credential system.
+- **git_repo/git_server/git_user/git_password_env**: information of git
+ repository, **git_password_env** is a key for Jenkins Credential system.
Cluster level
"""""""""""""
-- **gcp_credential**: Google Cloud Platform credential for remote storage, used by Terraform.
+- **gcp_credential**: Google Cloud Platform credential for remote storage, used
+ by Terraform.
- **terraform_dir**: The root directory of the TOST directory.
- **rancher_cluster**: target Rancher cluster name.
- **rancher_api_env**: Rancher credential to access Rancher, used by Terraform.
@@ -447,12 +468,13 @@
Jenkins Job Builder (JJB)
^^^^^^^^^^^^^^^^^^^^^^^^^
-We prefer to apply the IaC (Infrastructure as Code) for everything.
-We use the JJB (Jenkins Job Builder) to create new Jenkins Job, including the Jenkins pipeline.
-We need to clone a set of Jenkins jobs when a new edge is deployed.
+We prefer to apply the IaC (Infrastructure as Code) for everything. We use the
+JJB (Jenkins Job Builder) to create new Jenkins Job, including the Jenkins
+pipeline. We need to clone a set of Jenkins jobs when a new edge is deployed.
-In order to provide the flexibility and avoid re-inventing the wheel, we used the job template to declare your job.
-Thanks to the JJB, we can use the parameters in the job template to render different kinds of jobs easily.
+In order to provide the flexibility and avoid re-inventing the wheel, we used
+the job template to declare your job. Thanks to the JJB, we can use the
+parameters in the job template to render different kinds of jobs easily.
All the template files are placed under templates directory.
@@ -480,8 +502,8 @@
├── verify-licensed.yaml
└── versioning.yaml
-
-we defined all TOST required job templates in tost.yaml and here is its partial content.
+We defined all TOST required job templates in tost.yaml and here is its partial
+content.
.. code-block:: yaml
diff --git a/pronto_deployment_guide/troubleshooting.rst b/edge_deployment/troubleshooting.rst
similarity index 100%
rename from pronto_deployment_guide/troubleshooting.rst
rename to edge_deployment/troubleshooting.rst
diff --git a/edge_deployment/vpn_bootstrap.rst b/edge_deployment/vpn_bootstrap.rst
new file mode 100644
index 0000000..223e3d8
--- /dev/null
+++ b/edge_deployment/vpn_bootstrap.rst
@@ -0,0 +1,309 @@
+..
+ SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
+ SPDX-License-Identifier: Apache-2.0
+
+VPN Bootstrap
+=============
+
+This section walks you through how to set up a VPN between ACE and Aether
+Central in GCP. We will be using GitOps based Aether CD pipeline for this, so
+we just need to create a patch to **aether-pod-configs** repository. Note that
+some of the steps described here are not directly related to setting up a VPN,
+but rather are a prerequisite for adding a new ACE.
+
+.. attention::
+
+ If you are adding another ACE to an existing VPN connection, go to
+ :ref:`Add ACE to an existing VPN connection <add_ace_to_vpn>`
+
+Before you begin
+----------------
+
+* Make sure firewall in front of ACE allows UDP port 500, UDP port 4500, and
+ ESP packets from **gcpvpn1.infra.aetherproject.net(35.242.47.15)** and
+ **gcpvpn2.infra.aetherproject.net(34.104.68.78)**
+
+* Make sure that the external IP on ACE side is owned by or routed to the
+ management node
+
+To help your understanding, the following sample ACE environment will be used
+in the rest of this section. Make sure to replace the sample values when you
+actually create a review request.
+
++-----------------------------+----------------------------------+
+| Management node external IP | 128.105.144.189 |
++-----------------------------+----------------------------------+
+| ASN | 65003 |
++-----------------------------+----------------------------------+
+| GCP BGP IP address | Tunnel 1: 169.254.0.9/30 |
+| +----------------------------------+
+| | Tunnel 2: 169.254.1.9/30 |
++-----------------------------+----------------------------------+
+| ACE BGP IP address | Tunnel 1: 169.254.0.10/30 |
+| +----------------------------------+
+| | Tunnel 2: 169.254.1.10/30 |
++-----------------------------+----------------------------------+
+| PSK | UMAoZA7blv6gd3IaArDqgK2s0sDB8mlI |
++-----------------------------+----------------------------------+
+| Management Subnet | 10.91.0.0/24 |
++-----------------------------+----------------------------------+
+| K8S Subnet | Pod IP: 10.66.0.0/17 |
+| +----------------------------------+
+| | Cluster IP: 10.66.128.0/17 |
++-----------------------------+----------------------------------+
+
+Download aether-pod-configs repository
+--------------------------------------
+
+.. code-block:: shell
+
+ $ cd $WORKDIR
+ $ git clone "ssh://[username]@gerrit.opencord.org:29418/aether-pod-configs"
+
+.. _update_global_resource:
+
+Update global resource maps
+---------------------------
+
+Add a new ACE information at the end of the following global resource maps.
+
+* ``user_map.tfvars``
+* ``cluster_map.tfvars``
+* ``vpn_map.tfvars``
+
+As a note, you can find several other global resource maps under the
+``production`` directory. Resource definitions that need to be shared among
+clusters or are better managed in a single file to avoid configuration
+conflicts are maintained in this way.
+
+.. code-block:: diff
+
+ $ cd $WORKDIR/aether-pod-configs/production
+ $ vi user_map.tfvars
+
+ # Add the new cluster admin user at the end of the map
+ $ git diff user_map.tfvars
+ --- a/production/user_map.tfvars
+ +++ b/production/user_map.tfvars
+ @@ user_map = {
+ username = "menlo"
+ password = "changeme"
+ global_roles = ["user-base", "catalogs-use"]
+ + },
+ + test_admin = {
+ + username = "test"
+ + password = "changeme"
+ + global_roles = ["user-base", "catalogs-use"]
+ }
+ }
+
+.. code-block:: diff
+
+ $ cd $WORKDIR/aether-pod-configs/production
+ $ vi cluster_map.tfvars
+
+ # Add the new K8S cluster information at the end of the map
+ $ git diff cluster_map.tfvars
+ --- a/production/cluster_map.tfvars
+ +++ b/production/cluster_map.tfvars
+ @@ cluster_map = {
+ kube_dns_cluster_ip = "10.53.128.10"
+ cluster_domain = "prd.menlo.aetherproject.net"
+ calico_ip_detect_method = "can-reach=www.google.com"
+ + },
+ + ace-test = {
+ + cluster_name = "ace-test"
+ + management_subnets = ["10.91.0.0/24"]
+ + k8s_version = "v1.18.8-rancher1-1"
+ + k8s_pod_range = "10.66.0.0/17"
+ + k8s_cluster_ip_range = "10.66.128.0/17"
+ + kube_dns_cluster_ip = "10.66.128.10"
+ + cluster_domain = "prd.test.aetherproject.net"
+ + calico_ip_detect_method = "can-reach=www.google.com"
+ }
+ }
+ }
+
+.. code-block:: diff
+
+ $ cd $WORKDIR/aether-pod-configs/production
+ $ vi vpn_map.tfvars
+
+ # Add VPN and tunnel information at the end of the map
+ $ git diff vpn_map.tfvars
+ --- a/production/vpn_map.tfvars
+ +++ b/production/vpn_map.tfvars
+ @@ vpn_map = {
+ bgp_peer_ip_address_1 = "169.254.0.6"
+ bgp_peer_ip_range_2 = "169.254.1.5/30"
+ bgp_peer_ip_address_2 = "169.254.1.6"
+ + },
+ + ace-test = {
+ + peer_name = "production-ace-test"
+ + peer_vpn_gateway_address = "128.105.144.189"
+ + tunnel_shared_secret = "UMAoZA7blv6gd3IaArDqgK2s0sDB8mlI"
+ + bgp_peer_asn = "65003"
+ + bgp_peer_ip_range_1 = "169.254.0.9/30"
+ + bgp_peer_ip_address_1 = "169.254.0.10"
+ + bgp_peer_ip_range_2 = "169.254.1.9/30"
+ + bgp_peer_ip_address_2 = "169.254.1.10"
+ }
+ }
+
+.. note::
+ Unless you have a specific requirement, set ASN and BGP addresses to the next available values in the map.
+
+
+Create ACE specific configurations
+----------------------------------
+
+In this step, we will create a directory under `production` with the same name
+as ACE, and add several Terraform configurations and Ansible inventory needed
+to configure a VPN connection.
+
+Throughout the deployment procedure, this directory will contain all ACE
+specific configurations.
+
+Run the following commands to auto-generate necessary files under the target
+ACE directory.
+
+.. code-block:: shell
+
+ $ cd $WORKDIR/aether-pod-configs/tools
+ $ cp ace_env /tmp/ace_env
+ $ vi /tmp/ace_env
+ # Set environment variables
+
+ $ source /tmp/ace_env
+ $ make vpn
+ Created ../production/ace-test
+ Created ../production/ace-test/main.tf
+ Created ../production/ace-test/variables.tf
+ Created ../production/ace-test/gcp_fw.tf
+ Created ../production/ace-test/gcp_ha_vpn.tf
+ Created ../production/ace-test/ansible
+ Created ../production/ace-test/backend.tf
+ Created ../production/ace-test/cluster_val.tfvars
+ Created ../production/ace-test/ansible/hosts.ini
+ Created ../production/ace-test/ansible/extra_vars.yml
+
+.. attention::
+ The predefined templates are tailored to Pronto BOM. You'll need to fix `cluster_val.tfvars` and `ansible/extra_vars.yml`
+ when using a different BOM.
+
+Create a review request
+-----------------------
+
+.. code-block:: shell
+
+ $ cd $WORKDIR/aether-pod-configs/production
+ $ git status
+ On branch tools
+ Changes not staged for commit:
+
+ modified: cluster_map.tfvars
+ modified: user_map.tfvars
+ modified: vpn_map.tfvars
+
+ Untracked files:
+ (use "git add <file>..." to include in what will be committed)
+
+ ace-test/
+
+ $ git add .
+ $ git commit -m "Add test ACE"
+ $ git review
+
+Once the review request is accepted and merged,
+CD pipeline will create VPN tunnels on both GCP and the management node.
+
+Verify VPN connection
+---------------------
+
+You can verify the VPN connections after successful post-merge job by checking
+the routing table on the management node and trying to ping to one of the
+central cluster VMs.
+
+Make sure two tunnel interfaces, `gcp_tunnel1` and `gcp_tunnel2`, exist
+and three additional routing entries via one of the tunnel interfaces.
+
+.. code-block:: shell
+
+ # Verify routings
+ $ netstat -rn
+ Kernel IP routing table
+ Destination Gateway Genmask Flags MSS Window irtt Iface
+ 0.0.0.0 128.105.144.1 0.0.0.0 UG 0 0 0 eno1
+ 10.45.128.0 169.254.0.9 255.255.128.0 UG 0 0 0 gcp_tunnel1
+ 10.52.128.0 169.254.0.9 255.255.128.0 UG 0 0 0 gcp_tunnel1
+ 10.66.128.0 10.91.0.8 255.255.128.0 UG 0 0 0 eno1
+ 10.91.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eno1
+ 10.168.0.0 169.254.0.9 255.255.240.0 UG 0 0 0 gcp_tunnel1
+ 128.105.144.0 0.0.0.0 255.255.252.0 U 0 0 0 eno1
+ 169.254.0.8 0.0.0.0 255.255.255.252 U 0 0 0 gcp_tunnel1
+ 169.254.1.8 0.0.0.0 255.255.255.252 U 0 0 0 gcp_tunnel2
+
+ # Verify ACC VM access
+ $ ping 10.168.0.6
+
+ # Verify ACC K8S cluster access
+ $ nslookup kube-dns.kube-system.svc.prd.acc.gcp.aetherproject.net 10.52.128.10
+
+You can further verify whether the ACE routes are propagated well to GCP
+by checking GCP dashboard **VPC Network > Routes > Dynamic**.
+
+
+Post VPN setup
+--------------
+
+Once you verify the VPN connections, please update `ansible` directory name to
+`_ansible` to prevent the ansible playbook from running again. Note that it is
+no harm to re-run the ansible playbook but not recommended.
+
+.. code-block:: shell
+
+ $ cd $WORKDIR/aether-pod-configs/production/$ACE_NAME
+ $ mv ansible _ansible
+ $ git add .
+ $ git commit -m "Mark ansible done for test ACE"
+ $ git review
+
+.. _add_ace_to_vpn:
+
+Add another ACE to an existing VPN connection
+"""""""""""""""""""""""""""""""""""""""""""""
+
+VPN connections can be shared when there are multiple ACE clusters in a site.
+In order to add ACE to an existing VPN connection, you'll have to SSH into the
+management node and manually update BIRD configuration.
+
+.. note::
+
+ This step needs improvements in the future.
+
+.. code-block:: shell
+
+ $ sudo vi /etc/bird/bird.conf
+ protocol static {
+ ...
+ route 10.66.128.0/17 via 10.91.0.10;
+
+ # Add routings for the new ACE's K8S cluster IP range via cluster nodes
+ # TODO: Configure iBGP peering with Calico nodes and dynamically learn these routings
+ route <NEW-ACE-CLUSTER-IP> via <SERVER1>
+ route <NEW-ACE-CLUSTER-IP> via <SERVER2>
+ route <NEW-ACE-CLUSTER-IP> via <SERVER3>
+ }
+
+ filter gcp_tunnel_out {
+ # Add the new ACE's K8S cluster IP range and the management subnet if required to the list
+ if (net ~ [ 10.91.0.0/24, 10.66.128.0/17, <NEW-ACE-CLUSTER-IP-RANGE> ]) then accept;
+ else reject;
+ }
+ # Save and exit
+
+ $ sudo birdc configure
+
+ # Confirm the static routes are added
+ $ sudo birdc show route
+
diff --git a/index.rst b/index.rst
index bfff747..866ff58 100644
--- a/index.rst
+++ b/index.rst
@@ -8,20 +8,13 @@
can be found on the `ONF Wiki <https://wiki.opennetworking.org/>`_.
.. toctree::
- :maxdepth: 1
- :caption: Overview
- :hidden:
- :glob:
-
- readme
-
-.. toctree::
:maxdepth: 3
:caption: Operations
:hidden:
:glob:
- operations
+ operations/procedures
+ operations/sop
.. toctree::
:maxdepth: 3
@@ -29,12 +22,30 @@
:hidden:
:glob:
- pronto_deployment_guide/overview.rst
- pronto_deployment_guide/hw_installation.rst
- pronto_deployment_guide/bootstrapping.rst
- pronto_deployment_guide/run_time_deployment.rst
- pronto_deployment_guide/tost_deployment.rst
- pronto_deployment_guide/connectivity_service_update.rst
- pronto_deployment_guide/enb_installation.rst
- pronto_deployment_guide/acceptance_test_specification.rst
- pronto_deployment_guide/troubleshooting.rst
+ edge_deployment/overview
+ edge_deployment/site_planning
+ edge_deployment/management_net_bootstrap
+ edge_deployment/server_bootstrap
+ edge_deployment/fabric_switch_bootstrap
+ edge_deployment/vpn_bootstrap
+ edge_deployment/runtime_deployment
+ edge_deployment/tost_deployment
+ edge_deployment/connectivity_service_update
+ edge_deployment/enb_installation
+ edge_deployment/troubleshooting
+
+.. toctree::
+ :maxdepth: 3
+ :caption: Test Automation
+ :hidden:
+ :glob:
+
+ testing/acceptance_specification
+
+.. toctree::
+ :maxdepth: 1
+ :caption: Meta
+ :hidden:
+ :glob:
+
+ readme
diff --git a/operations.rst b/operations/procedures.rst
similarity index 96%
rename from operations.rst
rename to operations/procedures.rst
index a3b1e14..4c16b1e 100644
--- a/operations.rst
+++ b/operations/procedures.rst
@@ -2,8 +2,8 @@
SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
SPDX-License-Identifier: Apache-2.0
-Operations
-==========
+General Procedures
+==================
Edge shutdown procedure
-----------------------
diff --git a/operations/sop.rst b/operations/sop.rst
new file mode 100644
index 0000000..94fd81a
--- /dev/null
+++ b/operations/sop.rst
@@ -0,0 +1,8 @@
+..
+ SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
+ SPDX-License-Identifier: Apache-2.0
+
+Standard Operating Procedures
+=============================
+
+
diff --git a/pronto_deployment_guide/bootstrapping.rst b/pronto_deployment_guide/bootstrapping.rst
deleted file mode 100644
index 5b0d895..0000000
--- a/pronto_deployment_guide/bootstrapping.rst
+++ /dev/null
@@ -1,468 +0,0 @@
-..
- SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
- SPDX-License-Identifier: Apache-2.0
-
-Bootstrapping
-=============
-
-.. _switch-install:
-
-OS Installation - Switches
---------------------------
-
-The installation of the ONL OS image on the fabric switches uses the DHCP and
-HTTP server set up on the management server.
-
-The default image is downloaded during that installation process by the
-``onieboot`` role. Make changes to that roll and rerun the management playbook
-to download a newer switch image.
-
-Preparation
-"""""""""""
-
-The switches have a single ethernet port that is shared between OpenBMC and
-ONL. Find out the MAC addresses for both of these ports and enter it into
-NetBox.
-
-Change boot mode to ONIE Rescue mode
-""""""""""""""""""""""""""""""""""""
-
-In order to reinstall an ONL image, you must change the ONIE bootloader to
-"Rescue Mode".
-
-Once the switch is powered on, it should retrieve an IP address on the OpenBMC
-interface with DHCP. OpenBMC uses these default credentials::
-
- username: root
- password: 0penBmc
-
-Login to OpenBMC with SSH::
-
- $ ssh root@10.0.0.131
- The authenticity of host '10.0.0.131 (10.0.0.131)' can't be established.
- ECDSA key fingerprint is SHA256:...
- Are you sure you want to continue connecting (yes/no)? yes
- Warning: Permanently added '10.0.0.131' (ECDSA) to the list of known hosts.
- root@10.0.0.131's password:
- root@bmc:~#
-
-Using the Serial-over-LAN Console, enter ONL::
-
- root@bmc:~# /usr/local/bin/sol.sh
- You are in SOL session.
- Use ctrl-x to quit.
- -----------------------
-
- root@onl:~#
-
-.. note::
- If `sol.sh` is unresponsive, please try to restart the mainboard with::
-
- root@onl:~# wedge_power.sh restart
-
-
-Change the boot mode to rescue mode with the command ``onl-onie-boot-mode
-rescue``, and reboot::
-
- root@onl:~# onl-onie-boot-mode rescue
- [1053033.768512] EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
- [1053033.936893] EXT4-fs (sda3): re-mounted. Opts: (null)
- [1053033.996727] EXT4-fs (sda3): re-mounted. Opts: (null)
- The system will boot into ONIE rescue mode at the next restart.
- root@onl:~# reboot
-
-At this point, ONL will go through it's shutdown sequence and ONIE will start.
-If it does not start right away, press the Enter/Return key a few times - it
-may show you a boot selection screen. Pick ``ONIE`` and ``Rescue`` if given a
-choice.
-
-Installing an ONL image over HTTP
-"""""""""""""""""""""""""""""""""
-
-Now that the switch is in Rescue mode
-
-First, activate the Console by pressing Enter::
-
- discover: Rescue mode detected. Installer disabled.
-
- Please press Enter to activate this console.
- To check the install status inspect /var/log/onie.log.
- Try this: tail -f /var/log/onie.log
-
- ** Rescue Mode Enabled **
- ONIE:/ #
-
-Then run the ``onie-nos-install`` command, with the URL of the management
-server on the management network segment::
-
- ONIE:/ # onie-nos-install http://10.0.0.129/onie-installer
- discover: Rescue mode detected. No discover stopped.
- ONIE: Unable to find 'Serial Number' TLV in EEPROM data.
- Info: Fetching http://10.0.0.129/onie-installer ...
- Connecting to 10.0.0.129 (10.0.0.129:80)
- installer 100% |*******************************| 322M 0:00:00 ETA
- ONIE: Executing installer: http://10.0.0.129/onie-installer
- installer: computing checksum of original archive
- installer: checksum is OK
- ...
-
-The installation will now start, and then ONL will boot culminating in::
-
- Open Network Linux OS ONL-wedge100bf-32qs, 2020-11-04.19:44-64100e9
-
- localhost login:
-
-The default ONL login is::
-
- username: root
- password: onl
-
-If you login, you can verify that the switch is getting it's IP address via
-DHCP::
-
- root@localhost:~# ip addr
- ...
- 3: ma1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
- link/ether 00:90:fb:5c:e1:97 brd ff:ff:ff:ff:ff:ff
- inet 10.0.0.130/25 brd 10.0.0.255 scope global ma1
- ...
-
-
-Post-ONL Configuration
-""""""""""""""""""""""
-
-A ``terraform`` user must be created on the switches to allow them to be
-configured.
-
-This is done using Ansible. Verify that your inventory (Created earlier from the
-``inventory/example-aether.ini`` file) includes an ``[aetherfabric]`` section
-that has all the names and IP addresses of the compute nodes in it.
-
-Then run a ping test::
-
- ansible -i inventory/sitename.ini -m ping aetherfabric
-
-This may fail with the error::
-
- "msg": "Using a SSH password instead of a key is not possible because Host Key checking is enabled and sshpass does not support this. Please add this host's fingerprint to your known_hosts file to manage this host."
-
-Comment out the ``ansible_ssh_pass="onl"`` line, then rerun the ping test. It
-may ask you about authorized keys - answer ``yes`` for each host to trust the
-keys::
-
- The authenticity of host '10.0.0.138 (<no hostip for proxy command>)' can't be established.
- ECDSA key fingerprint is SHA256:...
- Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
-
-Once you've trusted the host keys, the ping test should succeed::
-
- spine1.role1.site | SUCCESS => {
- "changed": false,
- "ping": "pong"
- }
- leaf1.role1.site | SUCCESS => {
- "changed": false,
- "ping": "pong"
- }
- ...
-
-Then run the playbook to create the ``terraform`` user::
-
- ansible-playbook -i inventory/sitename.ini playbooks/aetherfabric-playbook.yml
-
-Once completed, the switch should now be ready for TOST runtime install.
-
-VPN
----
-
-This section walks you through how to set up a VPN between ACE and Aether Central in GCP.
-We will be using GitOps based Aether CD pipeline for this,
-so we just need to create a patch to **aether-pod-configs** repository.
-Note that some of the steps described here are not directly related to setting up a VPN,
-but rather are a prerequisite for adding a new ACE.
-
-.. attention::
-
- If you are adding another ACE to an existing VPN connection, go to
- :ref:`Add ACE to an existing VPN connection <add_ace_to_vpn>`
-
-Before you begin
-""""""""""""""""
-
-* Make sure firewall in front of ACE allows UDP port 500, UDP port 4500, and ESP packets
- from **gcpvpn1.infra.aetherproject.net(35.242.47.15)** and **gcpvpn2.infra.aetherproject.net(34.104.68.78)**
-* Make sure that the external IP on ACE side is owned by or routed to the management node
-
-To help your understanding, the following sample ACE environment will be used in the rest of this section.
-Make sure to replace the sample values when you actually create a review request.
-
-+-----------------------------+----------------------------------+
-| Management node external IP | 128.105.144.189 |
-+-----------------------------+----------------------------------+
-| ASN | 65003 |
-+-----------------------------+----------------------------------+
-| GCP BGP IP address | Tunnel 1: 169.254.0.9/30 |
-| +----------------------------------+
-| | Tunnel 2: 169.254.1.9/30 |
-+-----------------------------+----------------------------------+
-| ACE BGP IP address | Tunnel 1: 169.254.0.10/30 |
-| +----------------------------------+
-| | Tunnel 2: 169.254.1.10/30 |
-+-----------------------------+----------------------------------+
-| PSK | UMAoZA7blv6gd3IaArDqgK2s0sDB8mlI |
-+-----------------------------+----------------------------------+
-| Management Subnet | 10.91.0.0/24 |
-+-----------------------------+----------------------------------+
-| K8S Subnet | Pod IP: 10.66.0.0/17 |
-| +----------------------------------+
-| | Cluster IP: 10.66.128.0/17 |
-+-----------------------------+----------------------------------+
-
-Download aether-pod-configs repository
-""""""""""""""""""""""""""""""""""""""
-
-.. code-block:: shell
-
- $ cd $WORKDIR
- $ git clone "ssh://[username]@gerrit.opencord.org:29418/aether-pod-configs"
-
-.. _update_global_resource:
-
-Update global resource maps
-"""""""""""""""""""""""""""
-
-Add a new ACE information at the end of the following global resource maps.
-
-* user_map.tfvars
-* cluster_map.tfvars
-* vpn_map.tfvars
-
-As a note, you can find several other global resource maps under the `production` directory.
-Resource definitions that need to be shared among clusters or are better managed in a
-single file to avoid configuration conflicts are maintained in this way.
-
-.. code-block:: diff
-
- $ cd $WORKDIR/aether-pod-configs/production
- $ vi user_map.tfvars
-
- # Add the new cluster admin user at the end of the map
- $ git diff user_map.tfvars
- --- a/production/user_map.tfvars
- +++ b/production/user_map.tfvars
- @@ user_map = {
- username = "menlo"
- password = "changeme"
- global_roles = ["user-base", "catalogs-use"]
- + },
- + test_admin = {
- + username = "test"
- + password = "changeme"
- + global_roles = ["user-base", "catalogs-use"]
- }
- }
-
-.. code-block:: diff
-
- $ cd $WORKDIR/aether-pod-configs/production
- $ vi cluster_map.tfvars
-
- # Add the new K8S cluster information at the end of the map
- $ git diff cluster_map.tfvars
- --- a/production/cluster_map.tfvars
- +++ b/production/cluster_map.tfvars
- @@ cluster_map = {
- kube_dns_cluster_ip = "10.53.128.10"
- cluster_domain = "prd.menlo.aetherproject.net"
- calico_ip_detect_method = "can-reach=www.google.com"
- + },
- + ace-test = {
- + cluster_name = "ace-test"
- + management_subnets = ["10.91.0.0/24"]
- + k8s_version = "v1.18.8-rancher1-1"
- + k8s_pod_range = "10.66.0.0/17"
- + k8s_cluster_ip_range = "10.66.128.0/17"
- + kube_dns_cluster_ip = "10.66.128.10"
- + cluster_domain = "prd.test.aetherproject.net"
- + calico_ip_detect_method = "can-reach=www.google.com"
- }
- }
- }
-
-.. code-block:: diff
-
- $ cd $WORKDIR/aether-pod-configs/production
- $ vi vpn_map.tfvars
-
- # Add VPN and tunnel information at the end of the map
- $ git diff vpn_map.tfvars
- --- a/production/vpn_map.tfvars
- +++ b/production/vpn_map.tfvars
- @@ vpn_map = {
- bgp_peer_ip_address_1 = "169.254.0.6"
- bgp_peer_ip_range_2 = "169.254.1.5/30"
- bgp_peer_ip_address_2 = "169.254.1.6"
- + },
- + ace-test = {
- + peer_name = "production-ace-test"
- + peer_vpn_gateway_address = "128.105.144.189"
- + tunnel_shared_secret = "UMAoZA7blv6gd3IaArDqgK2s0sDB8mlI"
- + bgp_peer_asn = "65003"
- + bgp_peer_ip_range_1 = "169.254.0.9/30"
- + bgp_peer_ip_address_1 = "169.254.0.10"
- + bgp_peer_ip_range_2 = "169.254.1.9/30"
- + bgp_peer_ip_address_2 = "169.254.1.10"
- }
- }
-
-.. note::
- Unless you have a specific requirement, set ASN and BGP addresses to the next available values in the map.
-
-
-Create ACE specific configurations
-""""""""""""""""""""""""""""""""""
-
-In this step, we will create a directory under `production` with the same name as ACE,
-and add several Terraform configurations and Ansible inventory needed to configure a VPN connection.
-Throughout the deployment procedure, this directory will contain all ACE specific configurations.
-
-Run the following commands to auto-generate necessary files under the target ACE directory.
-
-.. code-block:: shell
-
- $ cd $WORKDIR/aether-pod-configs/tools
- $ cp ace_env /tmp/ace_env
- $ vi /tmp/ace_env
- # Set environment variables
-
- $ source /tmp/ace_env
- $ make vpn
- Created ../production/ace-test
- Created ../production/ace-test/main.tf
- Created ../production/ace-test/variables.tf
- Created ../production/ace-test/gcp_fw.tf
- Created ../production/ace-test/gcp_ha_vpn.tf
- Created ../production/ace-test/ansible
- Created ../production/ace-test/backend.tf
- Created ../production/ace-test/cluster_val.tfvars
- Created ../production/ace-test/ansible/hosts.ini
- Created ../production/ace-test/ansible/extra_vars.yml
-
-.. attention::
- The predefined templates are tailored to Pronto BOM. You'll need to fix `cluster_val.tfvars` and `ansible/extra_vars.yml`
- when using a different BOM.
-
-Create a review request
-"""""""""""""""""""""""
-
-.. code-block:: shell
-
- $ cd $WORKDIR/aether-pod-configs/production
- $ git status
- On branch tools
- Changes not staged for commit:
-
- modified: cluster_map.tfvars
- modified: user_map.tfvars
- modified: vpn_map.tfvars
-
- Untracked files:
- (use "git add <file>..." to include in what will be committed)
-
- ace-test/
-
- $ git add .
- $ git commit -m "Add test ACE"
- $ git review
-
-Once the review request is accepted and merged,
-CD pipeline will create VPN tunnels on both GCP and the management node.
-
-Verify VPN connection
-"""""""""""""""""""""
-
-You can verify the VPN connections after successful post-merge job
-by checking the routing table on the management node and trying to ping to one of the central cluster VMs.
-Make sure two tunnel interfaces, `gcp_tunnel1` and `gcp_tunnel2`, exist
-and three additional routing entries via one of the tunnel interfaces.
-
-.. code-block:: shell
-
- # Verify routings
- $ netstat -rn
- Kernel IP routing table
- Destination Gateway Genmask Flags MSS Window irtt Iface
- 0.0.0.0 128.105.144.1 0.0.0.0 UG 0 0 0 eno1
- 10.45.128.0 169.254.0.9 255.255.128.0 UG 0 0 0 gcp_tunnel1
- 10.52.128.0 169.254.0.9 255.255.128.0 UG 0 0 0 gcp_tunnel1
- 10.66.128.0 10.91.0.8 255.255.128.0 UG 0 0 0 eno1
- 10.91.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eno1
- 10.168.0.0 169.254.0.9 255.255.240.0 UG 0 0 0 gcp_tunnel1
- 128.105.144.0 0.0.0.0 255.255.252.0 U 0 0 0 eno1
- 169.254.0.8 0.0.0.0 255.255.255.252 U 0 0 0 gcp_tunnel1
- 169.254.1.8 0.0.0.0 255.255.255.252 U 0 0 0 gcp_tunnel2
-
- # Verify ACC VM access
- $ ping 10.168.0.6
-
- # Verify ACC K8S cluster access
- $ nslookup kube-dns.kube-system.svc.prd.acc.gcp.aetherproject.net 10.52.128.10
-
-You can further verify whether the ACE routes are propagated well to GCP
-by checking GCP dashboard **VPC Network > Routes > Dynamic**.
-
-
-Post VPN setup
-""""""""""""""
-
-Once you verify the VPN connections, please update `ansible` directory name to `_ansible` to prevent
-the ansible playbook from running again.
-Note that it is no harm to re-run the ansible playbook but not recommended.
-
-.. code-block:: shell
-
- $ cd $WORKDIR/aether-pod-configs/production/$ACE_NAME
- $ mv ansible _ansible
- $ git add .
- $ git commit -m "Mark ansible done for test ACE"
- $ git review
-
-.. _add_ace_to_vpn:
-
-Add another ACE to an existing VPN connection
-"""""""""""""""""""""""""""""""""""""""""""""
-
-VPN connections can be shared when there are multiple ACE clusters in a site.
-In order to add ACE to an existing VPN connection,
-you'll have to SSH into the management node and manually update BIRD configuration.
-
-.. note::
-
- This step needs improvements in the future.
-
-.. code-block:: shell
-
- $ sudo vi /etc/bird/bird.conf
- protocol static {
- ...
- route 10.66.128.0/17 via 10.91.0.10;
-
- # Add routings for the new ACE's K8S cluster IP range via cluster nodes
- # TODO: Configure iBGP peering with Calico nodes and dynamically learn these routings
- route <NEW-ACE-CLUSTER-IP> via <SERVER1>
- route <NEW-ACE-CLUSTER-IP> via <SERVER2>
- route <NEW-ACE-CLUSTER-IP> via <SERVER3>
- }
-
- filter gcp_tunnel_out {
- # Add the new ACE's K8S cluster IP range and the management subnet if required to the list
- if (net ~ [ 10.91.0.0/24, 10.66.128.0/17, <NEW-ACE-CLUSTER-IP-RANGE> ]) then accept;
- else reject;
- }
- # Save and exit
-
- $ sudo birdc configure
-
- # Confirm the static routes are added
- $ sudo birdc show route
-
diff --git a/pronto_deployment_guide/hw_installation.rst b/pronto_deployment_guide/hw_installation.rst
deleted file mode 100644
index cfee987..0000000
--- a/pronto_deployment_guide/hw_installation.rst
+++ /dev/null
@@ -1,654 +0,0 @@
-..
- SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
- SPDX-License-Identifier: Apache-2.0
-
-Hardware Installation
-=====================
-
-Once the hardware has been ordered, the installation can be planned and
-implemented. This document describes the installation of the servers and
-software.
-
-Installation of the fabric switch hardware is covered in :ref:`OS Installation
-- Switches <switch-install>`.
-
-Installation of the radio hardware is covered in :ref:`eNB Installation
-<enb-installation>`.
-
-Site Bookkeeping
-----------------
-
-The following items need to be added to `NetBox
-<https://netbox.readthedocs.io/en/stable>`_ to describe each edge site:
-
-1. Add a Site for the edge (if one doesn't already exist), which has the
- physical location and contact information for the edge.
-
-2. Add equipment Racks to the Site (if they don't already exist).
-
-3. Add a Tenant for the edge (who owns/manages it), assigned to the ``Pronto``
- or ``Aether`` Tenant Group.
-
-4. Add a VRF (Routing Table) for the edge site. This is usually just the name
- of the site. Make sure that ``Enforce unique space`` is checked, so that IP
- addresses within the VRF are forced to be unique, and that the Tenant Group
- and Tenant are set.
-
-5. Add a VLAN Group to the edge site, which groups the site's VLANs and
- requires that they have a unique VLAN number.
-
-6. Add VLANs for the edge site. These should be assigned a VLAN Group, the
- Site, and Tenant.
-
- There can be multiple of the same VLAN in NetBox (VLANs are layer 2, and
- local to the site), but not within the VLAN group.
-
- The minimal list of VLANs:
-
- * ADMIN 1
- * UPLINK 10
- * MGMT 800
- * FAB 801
-
- If you have multiple deployments at a site using the same management server,
- add additional VLANs incremented by 10 for the MGMT/FAB - for example:
-
- * DEVMGMT 810
- * DEVFAB 801
-
-7. Add IP Prefixes for the site. This should have the Tenant and VRF assigned.
-
- All edge IP prefixes fit into a ``/22`` sized block.
-
- The description of the Prefix contains the DNS suffix for all Devices that
- have IP addresses within this Prefix. The full DNS names are generated by
- combining the first ``<devname>`` component of the Device names with this
- suffix.
-
- An examples using the ``10.0.0.0/22`` block. There are 4 edge
- prefixes, with the following purposes:
-
- * ``10.0.0.0/25``
-
- * Has the Server BMC/LOM and Management Switch
- * Assign the ADMIN 1 VLAN
- * Set the description to ``admin.<deployment>.<site>.aetherproject.net`` (or
- ``prontoproject.net``).
-
- * ``10.0.0.128/25``
-
- * Has the Server Management plane, Fabric Switch Management/BMC
- * Assign MGMT 800 VLAN
- * Set the description to ``<deployment>.<site>.aetherproject.net`` (or
- ``prontoproject.net``).
-
- * ``10.0.1.0/25``
-
- * IP addresses of the qsfp0 port of the Compute Nodes to Fabric switches, devices
- connected to the Fabric like the eNB
- * Assign FAB 801 VLAN
- * Set the description to ``fab1.<deployment>.<site>.aetherproject.net`` (or
- ``prontoproject.net``).
-
- * ``10.0.1.128/25``
-
- * IP addresses of the qsfp1 port of the Compute Nodes to fabric switches
- * Assign FAB 801 VLAN
- * Set the description to ``fab2.<deployment>.<site>.aetherproject.net`` (or
- ``prontoproject.net``).
-
- There also needs to be a parent range of the two fabric ranges added:
-
- * ``10.0.1.0/24``
-
- * This is used to configure the correct routes, DNS, and TFTP servers
- provided by DHCP to the equipment that is connected to the fabric
- leaf switch that the management server (which provides those
- services) is not connected to.
-
- Additionally, these edge prefixes are used for Kubernetes but don't need to
- be created in NetBox:
-
- * ``10.0.2.0/24``
-
- * Kubernetes Pod IP's
-
- * ``10.0.3.0/24``
-
- * Kubernetes Cluster IP's
-
-8. Add Devices to the site, for each piece of equipment. These are named with a
- scheme similar to the DNS names used for the pod, given in this format::
-
- <devname>.<deployment>.<site>
-
- Examples::
-
- mgmtserver1.ops1.tucson
- node1.stage1.menlo
-
- Note that these names are transformed into DNS names using the Prefixes, and
- may have additional components - ``admin`` or ``fabric`` may be added after
- the ``<devname>`` for devices on those networks.
-
- Set the following fields when creating a device:
-
- * Site
- * Tenant
- * Rack & Rack Position
- * Serial number
-
- If a specific Device Type doesn't exist for the device, it must be created,
- which is detailed in the NetBox documentation, or ask the OPs team for help.
-
- See `Rackmount of Equipment`_ below for guidance on how equipment should be
- mounted in the Rack.
-
-9. Add Services to the management server:
-
- * name: ``dns``
- protocol: UDP
- port: 53
-
- * name: ``tftp``
- protocol: UDP
- port: 69
-
- These are used by the DHCP and DNS config to know which servers offer
- DNS or TFTP service.
-
-10. Set the MAC address for the physical interfaces on the device.
-
- You may also need to add physical network interfaces if aren't already
- created by the Device Type. An example would be if additional add-in
- network cards were installed.
-
-11. Add any virtual interfaces to the Devices. When creating a virtual
- interface, it should have it's ``label`` field set to the physical network
- interface that it is assigned
-
- These are needed are two cases for the Pronto deployment:
-
- 1. On the Management Server, there should bet (at least) two VLAN
- interfaces created attached to the ``eno2`` network port, which
- are used to provide connectivity to the management plane and fabric.
- These should be named ``<name of vlan><vlan ID>``, so the MGMT 800 VLAN
- would become a virtual interface named ``mgmt800``, with the label
- ``eno2``.
-
- 2. On the Fabric switches, the ``eth0`` port is shared between the OpenBMC
- interface and the ONIE/ONL installation. Add a ``bmc`` virtual
- interface with a label of ``eth0`` on each fabric switch, and check the
- ``OOB Management`` checkbox.
-
-12. Create IP addresses for the physical and virtual interfaces. These should
- have the Tenant and VRF set.
-
- The Management Server should always have the first IP address in each
- range, and they should be incremental, in this order. Examples are given as
- if there was a single instance of each device - adding additional devices
- would increment the later IP addresses.
-
- * Management Server
-
- * ``eno1`` - site provided public IP address, or blank if DHCP
- provided
-
- * ``eno2`` - 10.0.0.1/25 (first of ADMIN) - set as primary IP
- * ``bmc`` - 10.0.0.2/25 (next of ADMIN)
- * ``mgmt800`` - 10.0.0.129/25 (first of MGMT)
- * ``fab801`` - 10.0.1.1/25 (first of FAB)
-
- * Management Switch
-
- * ``gbe1`` - 10.0.0.3/25 (next of ADMIN) - set as primary IP
-
- * Fabric Switch
-
- * ``eth0`` - 10.0.0.130/25 (next of MGMT), set as primary IP
- * ``bmc`` - 10.0.0.131/25
-
- * Compute Server
-
- * ``eth0`` - 10.0.0.132/25 (next of MGMT), set as primary IP
- * ``bmc`` - 10.0.0.4/25 (next of ADMIN)
- * ``qsfp0`` - 10.0.1.2/25 (next of FAB)
- * ``qsfp1`` - 10.0.1.3/25
-
- * Other Fabric devices (eNB, etc.)
-
- * ``eth0`` or other primary interface - 10.0.1.4/25 (next of FAB)
-
-13. Add DHCP ranges to the IP Prefixes for IP's that aren't reserved. These are
- done like any other IP Address, but with the ``Status`` field is set to
- ``DHCP``, and they'll consume the entire range of IP addresses given in the
- CIDR mask.
-
- For example ``10.0.0.32/27`` as a DHCP block would take up 1/4 of the ADMIN
- prefix.
-
-14. Add router IP reservations to the IP Prefix for both Fabric prefixes. These
- are IP addresses used by ONOS to route traffic to the other leaf, and have
- the following attributes:
-
- - Have the last usable address in range (in the ``/25`` fabric examples
- above, these would be ``10.0.1.126/25`` and ``10.0.1.254/25``)
-
- - Have a ``Status`` of ``Reserved``, and the VRF, Tenant Group, and Tenant
- set.
-
- - The Description must start with the word ``router``, such as: ``router
- for leaf1 Fabric``
-
- - A custom field named ``RFC3442 Routes`` is set to the CIDR IP address of
- the opposite leaf - if the leaf's prefix is ``10.0.1.0/25`` and the
- router IP is ``10.0.1.126/25`` then ``RFC3442 Routes`` should be set to
- ``10.0.1.128\25`` (and the reverse - on ``10.0.1.254/25`` the ``RFC3442
- Routes`` would be set to be ``10.0.1.0/25``). This creates an `RFC3442
- Classless Static Route Option <https://tools.ietf.org/html/rfc3442>`_
- for the subnet in DHCP.
-
-15. Add Cables between physical interfaces on the devices
-
- The topology needs to match the logical diagram presented in the
- :ref:`network_cable_plan`. Note that many of the management interfaces
- need to be located either on the MGMT or ADMIN VLANs, and the management
- switch is
- used to provide that separation.
-
-Rackmount of Equipment
-----------------------
-
-Most of the Pronto equipment has a 19" rackmount form factor.
-
-Guidelines for mounting this equipment:
-
-- The EdgeCore Wedge Switches have a front-to-back (aka "port-to-power") fan
- configuration, so hot air exhaust is out the back of the switch near the
- power inlets, away from the 32 QSFP network ports on the front of the switch.
-
-- The full-depth 1U and 2U Supermicro servers also have front-to-back airflow
- but have most of their ports on the rear of the device.
-
-- Airflow through the rack should be in one direction to avoid heat being
- pulled from one device into another. This means that to connect the QSFP
- network ports from the servers to the switches, cabling should be routed
- through the rack from front (switch) to back (server). Empty rack spaces
- should be reserved for this purpose.
-
-- The short-depth management HP Switch and 1U Supermicro servers should be
- mounted on the rear of the rack. They both don't generate an appreciable
- amount of heat, so the airflow direction isn't a significant factor in
- racking them.
-
-Inventory
----------
-
-Once equipment arrives, any device needs to be recorded in inventory if it:
-
-1. Connects to the network (has a MAC address)
-2. Has a serial number
-3. Isn't a subcomponent (disk, add-in card, linecard, etc.) of a larger device.
-
-The following information should be recorded for every device:
-
-- Manufacturer
-- Model
-- Serial Number
-- MAC address (for the primary and any management/BMC/IPMI interfaces)
-
-This information should be be added to the corresponding Devices ONF NetBox
-instance. The accuracy of this information is very important as it is used in
-bootstrapping the systems.
-
-Once inventory has been completed, let the Infra team know, and the pxeboot
-configuration will be generated to have the OS preseed files corresponding to the
-new servers based on their serial numbers.
-
-Management Switch Bootstrap
----------------------------
-
-The current Pronto deployment uses an HP/Aruba 2540 24G PoE+ 4SFP+ JL356A
-switch to run the management network and other VLAN's that are used internally.
-
-By default the switch will pull an IP address via DHCP and ``http://<switch IP>``
-will display a management webpage for the switch. You need to be able to access
-this webpage before you can update the configuration.
-
-Loading the Management Switch Configuration
-"""""""""""""""""""""""""""""""""""""""""""
-
-1. Obtain a copy of the Management switch configuration file (this ends in ``.pcc``).
-
-2. Open the switch web interface at ``http://<switch IP>``. You may be
- prompted to login - the default credentials are both ``admin``:
-
- .. image:: images/pswi-000.png
- :alt: User Login for switch
- :scale: 50%
-
-3. Go to the "Management" section at bottom left:
-
- .. image:: images/pswi-001.png
- :alt: Update upload
- :scale: 50%
-
- In the "Update" section at left, drag the configuration file into the upload
- area, or click Browse and select it.
-
-4. In the "Select switch configuration file to update" section, select
- "config1", so it overwrites the default configuration.
-
-5. In the "Select switch configuration file to update" section, select
- "config1", so it overwrites the default configuration. Click "Update".
- You'll be prompted to reboot the switch, which you can do with the power
- symbol at top right. You may be prompted to select an image used to reboot -
- the "Previously Selected" is the correct one to use:
-
- .. image:: images/pswi-003.png
- :alt: Switch Image Select
- :scale: 30%
-
-6. Wait for the switch to reboot:
-
- .. image:: images/pswi-004.png
- :alt: Switch Reboot
- :scale: 50%
-
- The switch is now configured with the correct VLANs for Pronto Use. If you
- go to Interfaces > VLANs should see a list of VLANs configured on the
- switch:
-
- .. image:: images/pswi-005.png
- :alt: Mgmt VLANs
- :scale: 50%
-
-Software Bootstrap
-------------------
-
-Management Server Bootstrap
-"""""""""""""""""""""""""""
-
-The management server is bootstrapped into a customized version of the standard
-Ubuntu 18.04 OS installer.
-
-The `iPXE boot firmware <https://ipxe.org/>`_. is used to start this process
-and is built using the steps detailed in the `ipxe-build
-<https://gerrit.opencord.org/plugins/gitiles/ipxe-build>`_. repo, which
-generates both USB and PXE chainloadable boot images.
-
-Once a system has been started using these images started, these images will
-download a customized script from an external webserver to continue the boot
-process. This iPXE to webserver connection is secured with mutual TLS
-authentication, enforced by the nginx webserver.
-
-The iPXE scripts are created by the `pxeboot
-<https://gerrit.opencord.org/plugins/gitiles/ansible/role/pxeboot>`_ role,
-which creates both a boot menu, downloads the appropriate binaries for
-bootstrapping an OS installation, and creates per-node installation preseed files.
-
-The preseed files contain configuration steps to install the OS from the
-upstream Ubuntu repos, as well as customization of packages and creating the
-``onfadmin`` user.
-
-Creating a bootable USB drive
-'''''''''''''''''''''''''''''
-
-1. Get a USB key. Can be tiny as the uncompressed image is floppy sized
- (1.4MB). Download the USB image file (``<date>_onf_ipxe.usb.zip``) on the
- system you're using to write the USB key, and unzip it.
-
-2. Put a USB key in the system you're using to create the USB key, then
- determine which USB device file it's at in ``/dev``. You might look at the
- end of the ``dmesg`` output on Linux/Unix or the output of ``diskutil
- list`` on macOS.
-
- Be very careful about this, as if you accidentally overwrite some other disk in
- your system that would be highly problematic.
-
-3. Write the image to the device::
-
- $ dd if=/path/to/20201116_onf_ipxe.usb of=/dev/sdg
- 2752+0 records in
- 2752+0 records out
- 1409024 bytes (1.4 MB, 1.3 MiB) copied, 2.0272 s, 695 kB/s
-
- You may need to use `sudo` for this.
-
-Boot and Image Management Server
-''''''''''''''''''''''''''''''''
-
-1. Connect a USB keyboard and VGA monitor to the management node. Put the USB
- Key in one of the management node's USB ports (port 2 or 3):
-
- .. image:: images/mgmtsrv-000.png
- :alt: Management Server Ports
- :scale: 50%
-
-2. Turn on the management node, and press the F11 key as it starts to get into
- the Boot Menu:
-
- .. image:: images/mgmtsrv-001.png
- :alt: Management Server Boot Menu
- :scale: 50%
-
-3. Select the USB key (in this case "PNY USB 2.0", your options may vary) and press return. You should see iPXE load:
-
- .. image:: images/mgmtsrv-002.png
- :alt: iPXE load
- :scale: 50%
-
-4. A menu will appear which displays the system information and DHCP discovered
- network settings (your network must provide the IP address to the management
- server via DHCP):
-
- Use the arrow keys to select "Ubuntu 18.04 Installer (fully automatic)":
-
- .. image:: images/mgmtsrv-003.png
- :alt: iPXE Menu
- :scale: 50%
-
- There is a 10 second default timeout if left untouched (it will continue the
- system boot process) so restart the system if you miss the 10 second window.
-
-5. The Ubuntu 18.04 installer will be downloaded and booted:
-
- .. image:: images/mgmtsrv-004.png
- :alt: Ubuntu Boot
- :scale: 50%
-
-6. Then the installer starts and takes around 10 minutes to install (depends on
- your connection speed):
-
- .. image:: images/mgmtsrv-005.png
- :alt: Ubuntu Install
- :scale: 50%
-
-
-7. At the end of the install, the system will restart and present you with a
- login prompt:
-
- .. image:: images/mgmtsrv-006.png
- :alt: Ubuntu Install Complete
- :scale: 50%
-
-
-Management Server Configuration
-'''''''''''''''''''''''''''''''
-
-Once the OS is installed on the management server, Ansible is used to remotely
-install software on the management server.
-
-To checkout the ONF ansible repo and enter the virtualenv with the tooling::
-
- mkdir infra
- cd infra
- repo init -u ssh://<your gerrit username>@gerrit.opencord.org:29418/infra-manifest
- repo sync
- cd ansible
- make galaxy
- source venv_onfansible/bin/activate
-
-Obtain the ``undionly.kpxe`` iPXE artifact for bootstrapping the compute
-servers, and put it in the ``playbook/files`` directory.
-
-Next, create an inventory file to access the NetBox API. An example is given
-in ``inventory/example-netbox.yml`` - duplicate this file and modify it. Fill
-in the ``api_endpoint`` address and ``token`` with an API key you get out of
-the NetBox instance. List the IP Prefixes used by the site in the
-``ip_prefixes`` list.
-
-Next, run the ``scripts/netbox_edgeconfig.py`` to generate a host_vars file for
-the management server. Assuming that the management server in the edge is
-named ``mgmtserver1.stage1.menlo``, you'd run::
-
- python scripts/netbox_edgeconfig.py inventory/my-netbox.yml > inventory/host_vars/mgmtserver1.stage1.menlo.yml
-
-One manual change needs to be made to this output - edit the
-``inventory/host_vars/mgmtserver1.stage1.menlo.yml`` file and add the following
-to the bottom of the file, replacing the IP addresses with the management
-server IP address for each segment.
-
-In the case of the Fabric that has two leaves and IP ranges, add the Management
-server IP address used for the leaf that it is connected to, and then add a
-route for the other IP address range for the non-Management-connected leaf that
-is via the Fabric router address in the connected leaf range.
-
-This configures the `netplan <https://netplan.io>`_ on the management server,
-and creates a SNAT rule for the UE range route, and will be automated away
-soon::
-
- # added manually
- netprep_netplan:
- ethernets:
- eno2:
- addresses:
- - 10.0.0.1/25
- vlans:
- mgmt800:
- id: 800
- link: eno2
- addresses:
- - 10.0.0.129/25
- fabr801:
- id: 801
- link: eno2
- addresses:
- - 10.0.1.129/25
- routes:
- - to: 10.0.1.0/25
- via: 10.0.1.254
- metric: 100
-
- netprep_nftables_nat_postrouting: >
- ip saddr 10.0.1.0/25 ip daddr 10.168.0.0/20 counter snat to 10.0.1.129;
-
-
-Using the ``inventory/example-aether.ini`` as a template, create an
-:doc:`ansible inventory <ansible:user_guide/intro_inventory>` file for the
-site. Change the device names, IP addresses, and ``onfadmin`` password to match
-the ones for this site. The management server's configuration is in the
-``[aethermgmt]`` and corresponding ``[aethermgmt:vars]`` section.
-
-Then, to configure a management server, run::
-
- ansible-playbook -i inventory/sitename.ini playbooks/aethermgmt-playbook.yml
-
-This installs software with the following functionality:
-
-- VLANs on second Ethernet port to provide connectivity to the rest of the pod.
-- Firewall with NAT for routing traffic
-- DHCP and TFTP for bootstrapping servers and switches
-- DNS for host naming and identification
-- HTTP server for serving files used for bootstrapping switches
-- Downloads the Tofino switch image
-- Creates user accounts for administrative access
-
-Compute Server Bootstrap
-""""""""""""""""""""""""
-
-Once the management server has finished installation, it will be set to offer
-the same iPXE bootstrap file to the computer.
-
-Each node will be booted, and when iPXE loads select the ``Ubuntu 18.04
-Installer (fully automatic)`` option.
-
-The nodes can be controlled remotely via their BMC management interfaces - if
-the BMC is at ``10.0.0.3`` a remote user can SSH into them with::
-
- ssh -L 2443:10.0.0.3:443 onfadmin@<mgmt server ip>
-
-And then use their web browser to access the BMC at::
-
- https://localhost:2443
-
-The default BMC credentials for the Pronto nodes are::
-
- login: ADMIN
- password: Admin123
-
-The BMC will also list all of the MAC addresses for the network interfaces
-(including BMC) that are built into the logic board of the system. Add-in
-network cards like the 40GbE ones used in compute servers aren't listed.
-
-To prepare the compute nodes, software must be installed on them. As they
-can't be accessed directly from your local system, a :ref:`jump host
-<ansible:use_ssh_jump_hosts>` configuration is added, so the SSH connection
-goes through the management server to the compute systems behind it. Doing this
-requires a few steps:
-
-First, configure SSH to use Agent forwarding - create or edit your
-``~/.ssh/config`` file and add the following lines::
-
- Host <management server IP>
- ForwardAgent yes
-
-Then try to login to the management server, then the compute node::
-
- $ ssh onfadmin@<management server IP>
- Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-54-generic x86_64)
- ...
- onfadmin@mgmtserver1:~$ ssh onfadmin@10.0.0.138
- Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-54-generic x86_64)
- ...
- onfadmin@node2:~$
-
-Being able to login to the compute nodes from the management node means that
-SSH Agent forwarding is working correctly.
-
-Verify that your inventory (Created earlier from the
-``inventory/example-aether.ini`` file) includes an ``[aethercompute]`` section
-that has all the names and IP addresses of the compute nodes in it.
-
-Then run a ping test::
-
- ansible -i inventory/sitename.ini -m ping aethercompute
-
-It may ask you about authorized keys - answer ``yes`` for each host to trust the keys::
-
- The authenticity of host '10.0.0.138 (<no hostip for proxy command>)' can't be established.
- ECDSA key fingerprint is SHA256:...
- Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
-
-You should then see a success message for each host::
-
- node1.stage1.menlo | SUCCESS => {
- "changed": false,
- "ping": "pong"
- }
- node2.stage1.menlo | SUCCESS => {
- "changed": false,
- "ping": "pong"
- }
- ...
-
-Once you've seen this, run the playbook to install the prerequisites (Terraform
-user, Docker)::
-
- ansible-playbook -i inventory/sitename.ini playbooks/aethercompute-playbook.yml
-
-Note that Docker is quite large and may take a few minutes for installation
-depending on internet connectivity.
-
-Now that these compute nodes have been brought up, the rest of the installation
-can continue.
diff --git a/pronto_deployment_guide/acceptance_test_specification.rst b/testing/acceptance_specification.rst
similarity index 96%
rename from pronto_deployment_guide/acceptance_test_specification.rst
rename to testing/acceptance_specification.rst
index da12424..c4c72e1 100644
--- a/pronto_deployment_guide/acceptance_test_specification.rst
+++ b/testing/acceptance_specification.rst
@@ -2,23 +2,27 @@
SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
SPDX-License-Identifier: Apache-2.0
-=============================
-Acceptance Test Specification
-=============================
+Acceptance Specification
+========================
Objectives
-==========
+----------
-The purpose of this document is to create an end-user test object list (TOL) for Aether Connected Edge (ACE).
-This document will focus on the connectivity services end-user testing.
-In the future, this document will extend to other services offered through ACE.
+The purpose of this document is to create an end-user test object list (TOL)
+for Aether Connected Edge (ACE).
+
+This document will focus on the connectivity services end-user testing. In the
+future, this document will extend to other services offered through ACE.
The Automated continuous testing framework for the platform is out of the scope of this document.
Integration Test (eNB-LTE Core)
-===============================
-Before we start to test End-to-End connectivity, we have to check the connection (called S1-MME/S1-C interface)
-between eNB in an edge and MME in a public cloud.
+-------------------------------
+
+Before we start to test End-to-End connectivity, we have to check the
+connection (called S1-MME/S1-C interface) between eNB in an edge and MME in a
+public cloud.
+
In order to verify this connectivity, the following test cases should be passed.
Note that all the following test/verification cases have some assumptions:
@@ -29,6 +33,7 @@
IT-TOL01 Fabric Test 1: the connectivity test within the edge
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
In order to test the fabric test, please see the following steps:
@@ -216,7 +221,7 @@
:alt: Example (the eNB ID is 19)
Connectivity Services
-=====================
+---------------------
Aether provides only data connectivity for end-user devices and systems.
So the voice service over LTE is not available. However, users can use
@@ -313,14 +318,14 @@
Monitoring Services
-===================
+-------------------
ACE uses the Grafana dashboard for monitoring services.
Each ACE will be provided with Read-Only Access to our centralized monitoring platform.
Application Services
-====================
+--------------------
Aether uses Rancher to onboard applications to ACE.
Each ACE host will be provided with access to rancher to onboard applications on their ACE cluster.
diff --git a/pronto_deployment_guide/images/it-tol04.png b/testing/images/it-tol04.png
similarity index 100%
rename from pronto_deployment_guide/images/it-tol04.png
rename to testing/images/it-tol04.png
Binary files differ
diff --git a/pronto_deployment_guide/images/it-tol05.png b/testing/images/it-tol05.png
similarity index 100%
rename from pronto_deployment_guide/images/it-tol05.png
rename to testing/images/it-tol05.png
Binary files differ
diff --git a/pronto_deployment_guide/images/it-tol06.png b/testing/images/it-tol06.png
similarity index 100%
rename from pronto_deployment_guide/images/it-tol06.png
rename to testing/images/it-tol06.png
Binary files differ
