Update VPN and runtime section in consideration of dev cluster

Change-Id: I763f88f9b6bb02232e0846112a53ff8defb1f049
diff --git a/pronto_deployment_guide/bootstrapping.rst b/pronto_deployment_guide/bootstrapping.rst
index 08d0977..1870190 100644
--- a/pronto_deployment_guide/bootstrapping.rst
+++ b/pronto_deployment_guide/bootstrapping.rst
@@ -14,6 +14,11 @@
 Note that some of the steps described here are not directly related to setting up a VPN,
 but rather are a prerequisite for adding a new ACE.
 
+.. attention::
+
+   If you are adding another ACE to an existing VPN connection, go to
+   :ref:`Add ACE to an existing VPN connection <add_ace_to_vpn>`
+
 Before you begin
 ----------------
 * Make sure firewall in front of ACE allows UDP port 500, UDP port 4500, and ESP packets
@@ -45,7 +50,6 @@
 |                             | Cluster IP: 10.66.128.0/17       |
 +-----------------------------+----------------------------------+
 
-
 Download aether-pod-configs repository
 --------------------------------------
 .. code-block:: shell
@@ -211,6 +215,7 @@
 
 .. code-block:: shell
 
+   # Verify routings
    $ netstat -rn
    Kernel IP routing table
    Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
@@ -224,15 +229,15 @@
    169.254.0.8     0.0.0.0         255.255.255.252 U         0 0          0 gcp_tunnel1
    169.254.1.8     0.0.0.0         255.255.255.252 U         0 0          0 gcp_tunnel2
 
-   $ ping 10.168.0.6 -c 3
-   PING 10.168.0.6 (10.168.0.6) 56(84) bytes of data.
-   64 bytes from 35.235.67.169: icmp_seq=1 ttl=56 time=67.9 ms
-   64 bytes from 35.235.67.169: icmp_seq=2 ttl=56 time=67.4 ms
-   64 bytes from 35.235.67.169: icmp_seq=3 ttl=56 time=67.1 ms
+   # Verify ACC VM access
+   $ ping 10.168.0.6
 
-   --- 10.168.0.6 ping statistics ---
-   3 packets transmitted, 3 received, 0% packet loss, time 2002ms
-   rtt min/avg/max/mdev = 67.107/67.502/67.989/0.422 ms
+   # Verify ACC K8S cluster access
+   $ nslookup kube-dns.kube-system.svc.prd.acc.gcp.aetherproject.net 10.52.128.10
+
+You can further verify whether the ACE routes are propagated well to GCP
+by checking GCP dashboard **VPC Network > Routes > Dynamic**.
+
 
 Post VPN setup
 --------------
@@ -248,6 +253,43 @@
    $ git commit -m "Mark ansible done for test ACE"
    $ git review
 
+.. _add_ace_to_vpn:
+
+Add another ACE to an existing VPN connection
+---------------------------------------------
+VPN connections can be shared when there are multiple ACE clusters in a site.
+In order to add ACE to an existing VPN connection,
+you'll have to SSH into the management node and manually update BIRD configuration.
+
+.. note::
+
+   This step needs improvements in the future.
+
+.. code-block:: shell
+
+   $ sudo vi /etc/bird/bird.conf
+   protocol static {
+      ...
+      route 10.66.128.0/17 via 10.91.0.10;
+
+      # Add routings for the new ACE's K8S cluster IP range via cluster nodes
+      # TODO: Configure iBGP peering with Calico nodes and dynamically learn these routings
+      route <NEW-ACE-CLUSTER-IP> via <SERVER1>
+      route <NEW-ACE-CLUSTER-IP> via <SERVER2>
+      route <NEW-ACE-CLUSTER-IP> via <SERVER3>
+   }
+
+   filter gcp_tunnel_out {
+      # Add the new ACE's K8S cluster IP range and the management subnet if required to the list
+      if (net ~ [ 10.91.0.0/24, 10.66.128.0/17, <NEW-ACE-CLUSTER-IP-RANGE> ]) then accept;
+      else reject;
+   }
+   # Save and exit
+
+   $ sudo birdc configure
+
+   # Confirm the static routes are added
+   $ sudo birdc show route
 
 OS Installation - Switches
 ==========================
diff --git a/pronto_deployment_guide/run_time_deployment.rst b/pronto_deployment_guide/run_time_deployment.rst
index 69b73cb..f693936 100644
--- a/pronto_deployment_guide/run_time_deployment.rst
+++ b/pronto_deployment_guide/run_time_deployment.rst
@@ -9,10 +9,6 @@
 We will be using GitOps based Aether CD pipeline for this,
 so we just need to create a patch to **aether-pod-configs** repository.
 
-Before you begin
-================
-Make sure :ref:`Update Global Resources Map <update_global_resource>` section is completed.
-
 Download aether-pod-configs repository
 ======================================
 Download aether-pod-configs repository if you don't have it already in your develop machine.
@@ -22,6 +18,70 @@
    $ cd $WORKDIR
    $ git clone "ssh://[username]@gerrit.opencord.org:29418/aether-pod-configs"
 
+Update global resource maps
+===========================
+.. attention::
+
+   Skip this section if you have already done the same step in the
+   :ref:`Update Global Resources Map for VPN <update_global_resource>` section.
+
+Add a new ACE information at the end of the following global resource maps.
+
+* user_map.tfvars
+* cluster_map.tfvars
+
+As a note, you can find several other global resource maps under the `production` directory.
+Resource definitions that need to be shared among clusters or are better managed in a
+single file to avoid configuration conflicts are maintained in this way.
+
+.. code-block:: diff
+
+   $ cd $WORKDIR/aether-pod-configs/production
+   $ vi user_map.tfvars
+
+   # Add the new cluster admin user at the end of the map
+   $ git diff user_map.tfvars
+   --- a/production/user_map.tfvars
+   +++ b/production/user_map.tfvars
+   @@ user_map = {
+      username      = "menlo"
+      password      = "changeme"
+      global_roles  = ["user-base", "catalogs-use"]
+   +  },
+   +  test_admin = {
+   +    username      = "test"
+   +    password      = "changeme"
+   +    global_roles  = ["user-base", "catalogs-use"]
+      }
+   }
+
+.. code-block:: diff
+
+   $ cd $WORKDIR/aether-pod-configs/production
+   $ vi cluster_map.tfvars
+
+   # Add the new K8S cluster information at the end of the map
+   $ git diff cluster_map.tfvars
+   --- a/production/cluster_map.tfvars
+   +++ b/production/cluster_map.tfvars
+   @@ cluster_map = {
+         kube_dns_cluster_ip     = "10.53.128.10"
+         cluster_domain          = "prd.menlo.aetherproject.net"
+         calico_ip_detect_method = "can-reach=www.google.com"
+   +    },
+   +    ace-test = {
+   +      cluster_name            = "ace-test"
+   +      management_subnets      = ["10.91.0.0/24"]
+   +      k8s_version             = "v1.18.8-rancher1-1"
+   +      k8s_pod_range           = "10.66.0.0/17"
+   +      k8s_cluster_ip_range    = "10.66.128.0/17"
+   +      kube_dns_cluster_ip     = "10.66.128.10"
+   +      cluster_domain          = "prd.test.aetherproject.net"
+   +      calico_ip_detect_method = "can-reach=www.google.com"
+         }
+      }
+   }
+
 Create runtime configurations
 =============================
 In this step, we will add several Terraform configurations and overriding values for the managed applications.
@@ -36,10 +96,15 @@
 
    $ source /tmp/ace_env
    $ make runtime
+   Created ../production/ace-test
    Created ../production/ace-test/main.tf
    Created ../production/ace-test/variables.tf
+   Created ../production/ace-test/gcp_fw.tf
    Created ../production/ace-test/cluster.tf
    Created ../production/ace-test/alerts.tf
+   Created ../production/ace-test/backend.tf
+   Created ../production/ace-test/cluster_val.tfvars
+   Created ../production/ace-test/app_values
    Created ../production/ace-test/app_values/ace-coredns.yml
    Created ../production/ace-test/app_values/omec-upf-pfcp-agent.yml