Merge "Update VPN and runtime section in consideration of dev cluster"
diff --git a/pronto_deployment_guide/bootstrapping.rst b/pronto_deployment_guide/bootstrapping.rst
index 08d0977..1870190 100644
--- a/pronto_deployment_guide/bootstrapping.rst
+++ b/pronto_deployment_guide/bootstrapping.rst
@@ -14,6 +14,11 @@
Note that some of the steps described here are not directly related to setting up a VPN,
but rather are a prerequisite for adding a new ACE.
+.. attention::
+
+ If you are adding another ACE to an existing VPN connection, go to
+ :ref:`Add ACE to an existing VPN connection <add_ace_to_vpn>`
+
Before you begin
----------------
* Make sure firewall in front of ACE allows UDP port 500, UDP port 4500, and ESP packets
@@ -45,7 +50,6 @@
| | Cluster IP: 10.66.128.0/17 |
+-----------------------------+----------------------------------+
-
Download aether-pod-configs repository
--------------------------------------
.. code-block:: shell
@@ -211,6 +215,7 @@
.. code-block:: shell
+ # Verify routings
$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
@@ -224,15 +229,15 @@
169.254.0.8 0.0.0.0 255.255.255.252 U 0 0 0 gcp_tunnel1
169.254.1.8 0.0.0.0 255.255.255.252 U 0 0 0 gcp_tunnel2
- $ ping 10.168.0.6 -c 3
- PING 10.168.0.6 (10.168.0.6) 56(84) bytes of data.
- 64 bytes from 35.235.67.169: icmp_seq=1 ttl=56 time=67.9 ms
- 64 bytes from 35.235.67.169: icmp_seq=2 ttl=56 time=67.4 ms
- 64 bytes from 35.235.67.169: icmp_seq=3 ttl=56 time=67.1 ms
+ # Verify ACC VM access
+ $ ping 10.168.0.6
- --- 10.168.0.6 ping statistics ---
- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms
- rtt min/avg/max/mdev = 67.107/67.502/67.989/0.422 ms
+ # Verify ACC K8S cluster access
+ $ nslookup kube-dns.kube-system.svc.prd.acc.gcp.aetherproject.net 10.52.128.10
+
+You can further verify whether the ACE routes are propagated well to GCP
+by checking GCP dashboard **VPC Network > Routes > Dynamic**.
+
Post VPN setup
--------------
@@ -248,6 +253,43 @@
$ git commit -m "Mark ansible done for test ACE"
$ git review
+.. _add_ace_to_vpn:
+
+Add another ACE to an existing VPN connection
+---------------------------------------------
+VPN connections can be shared when there are multiple ACE clusters in a site.
+In order to add ACE to an existing VPN connection,
+you'll have to SSH into the management node and manually update BIRD configuration.
+
+.. note::
+
+ This step needs improvements in the future.
+
+.. code-block:: shell
+
+ $ sudo vi /etc/bird/bird.conf
+ protocol static {
+ ...
+ route 10.66.128.0/17 via 10.91.0.10;
+
+ # Add routings for the new ACE's K8S cluster IP range via cluster nodes
+ # TODO: Configure iBGP peering with Calico nodes and dynamically learn these routings
+ route <NEW-ACE-CLUSTER-IP> via <SERVER1>
+ route <NEW-ACE-CLUSTER-IP> via <SERVER2>
+ route <NEW-ACE-CLUSTER-IP> via <SERVER3>
+ }
+
+ filter gcp_tunnel_out {
+ # Add the new ACE's K8S cluster IP range and the management subnet if required to the list
+ if (net ~ [ 10.91.0.0/24, 10.66.128.0/17, <NEW-ACE-CLUSTER-IP-RANGE> ]) then accept;
+ else reject;
+ }
+ # Save and exit
+
+ $ sudo birdc configure
+
+ # Confirm the static routes are added
+ $ sudo birdc show route
OS Installation - Switches
==========================
diff --git a/pronto_deployment_guide/run_time_deployment.rst b/pronto_deployment_guide/run_time_deployment.rst
index 69b73cb..f693936 100644
--- a/pronto_deployment_guide/run_time_deployment.rst
+++ b/pronto_deployment_guide/run_time_deployment.rst
@@ -9,10 +9,6 @@
We will be using GitOps based Aether CD pipeline for this,
so we just need to create a patch to **aether-pod-configs** repository.
-Before you begin
-================
-Make sure :ref:`Update Global Resources Map <update_global_resource>` section is completed.
-
Download aether-pod-configs repository
======================================
Download aether-pod-configs repository if you don't have it already in your develop machine.
@@ -22,6 +18,70 @@
$ cd $WORKDIR
$ git clone "ssh://[username]@gerrit.opencord.org:29418/aether-pod-configs"
+Update global resource maps
+===========================
+.. attention::
+
+ Skip this section if you have already done the same step in the
+ :ref:`Update Global Resources Map for VPN <update_global_resource>` section.
+
+Add a new ACE information at the end of the following global resource maps.
+
+* user_map.tfvars
+* cluster_map.tfvars
+
+As a note, you can find several other global resource maps under the `production` directory.
+Resource definitions that need to be shared among clusters or are better managed in a
+single file to avoid configuration conflicts are maintained in this way.
+
+.. code-block:: diff
+
+ $ cd $WORKDIR/aether-pod-configs/production
+ $ vi user_map.tfvars
+
+ # Add the new cluster admin user at the end of the map
+ $ git diff user_map.tfvars
+ --- a/production/user_map.tfvars
+ +++ b/production/user_map.tfvars
+ @@ user_map = {
+ username = "menlo"
+ password = "changeme"
+ global_roles = ["user-base", "catalogs-use"]
+ + },
+ + test_admin = {
+ + username = "test"
+ + password = "changeme"
+ + global_roles = ["user-base", "catalogs-use"]
+ }
+ }
+
+.. code-block:: diff
+
+ $ cd $WORKDIR/aether-pod-configs/production
+ $ vi cluster_map.tfvars
+
+ # Add the new K8S cluster information at the end of the map
+ $ git diff cluster_map.tfvars
+ --- a/production/cluster_map.tfvars
+ +++ b/production/cluster_map.tfvars
+ @@ cluster_map = {
+ kube_dns_cluster_ip = "10.53.128.10"
+ cluster_domain = "prd.menlo.aetherproject.net"
+ calico_ip_detect_method = "can-reach=www.google.com"
+ + },
+ + ace-test = {
+ + cluster_name = "ace-test"
+ + management_subnets = ["10.91.0.0/24"]
+ + k8s_version = "v1.18.8-rancher1-1"
+ + k8s_pod_range = "10.66.0.0/17"
+ + k8s_cluster_ip_range = "10.66.128.0/17"
+ + kube_dns_cluster_ip = "10.66.128.10"
+ + cluster_domain = "prd.test.aetherproject.net"
+ + calico_ip_detect_method = "can-reach=www.google.com"
+ }
+ }
+ }
+
Create runtime configurations
=============================
In this step, we will add several Terraform configurations and overriding values for the managed applications.
@@ -36,10 +96,15 @@
$ source /tmp/ace_env
$ make runtime
+ Created ../production/ace-test
Created ../production/ace-test/main.tf
Created ../production/ace-test/variables.tf
+ Created ../production/ace-test/gcp_fw.tf
Created ../production/ace-test/cluster.tf
Created ../production/ace-test/alerts.tf
+ Created ../production/ace-test/backend.tf
+ Created ../production/ace-test/cluster_val.tfvars
+ Created ../production/ace-test/app_values
Created ../production/ace-test/app_values/ace-coredns.yml
Created ../production/ace-test/app_values/omec-upf-pfcp-agent.yml