VOL-112: Document the backup and restore strategy as well as the steps required to backup and restore Consul data
Change-Id: I59a2cd98f55928fc0fff355ab634650dd74b0a27
diff --git a/docker/Dockerfile.tools b/docker/Dockerfile.tools
index 08956a2..5fd0510 100755
--- a/docker/Dockerfile.tools
+++ b/docker/Dockerfile.tools
@@ -28,10 +28,14 @@
RUN useradd -b /home -d /home/tools tools -s /bin/bash
RUN mkdir /home/tools
COPY docker/config/bashrc /home/tools/.bashrc
+COPY install/install_consul_cli.sh /home/tools
RUN chown -R tools.tools /home/tools
RUN echo "tools:tools" | chpasswd
RUN apt-get update && apt-get -y upgrade && apt-get -y install openssh-server kafkacat iputils-ping vim manpages iproute2 net-tools moreutils
RUN mkdir /var/run/sshd
+RUN chmod +x /home/tools/install_consul_cli.sh
+RUN /home/tools/install_consul_cli.sh
+RUN rm /home/tools/install_consul_cli.sh
RUN echo 'root:screencast' | chpasswd
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
diff --git a/docs/manuals/user/backup_restore/backup-restore.md b/docs/manuals/user/backup_restore/backup-restore.md
new file mode 100644
index 0000000..987e0d0
--- /dev/null
+++ b/docs/manuals/user/backup_restore/backup-restore.md
@@ -0,0 +1,101 @@
+## Backup and Restore
+The strategy for backup and restore for the entire Voltha cluster is
+depicted in the diagram below.
+
+ 
+
+ Implementation of this strategy would be done post-PoC3.
+
+For PoC-3, the strategy is to backup and restore Consul data manually.
+
+#### Backing Up Consul Data
+
+There are two sets of data that could be backup and restored from Consul:
+
+* The KV store using Consul's export/import commands
+* All data using Consul's snapshot command
+
+The focus is on the KV store as this is the only Consul data that needs to be
+backed up for Voltha. We will initiate the backup and restore operations
+from the voltha/tools container.
+
+The steps below show the basic backup and restore procedure. Consul provides
+ additional options, e.g. authentication, to use during these operations.
+
+##### Backup
+
+###### Prerequisites
+* Consul is running
+
+###### Steps
+
+* Get the IP and port number of one of the running consul node.
+
+* Enter the voltha/tools container on one of the cluster node:
+```angular2html
+docker exec -it <containerid> bash
+cd /home/tools
+```
+* Initiate the backup as a json file (consul_backup.json is just an example).
+```angular2html
+consul kv export -http-addr=http://<Consul_IP>:<Consul_PORT></Consul_PORT>
+service/voltha > consul_backup.json
+```
+
+File consul_backup.json contains all the data in json format from the KV
+store with prefix 'service/voltha'. The above command can be added as a
+Cron job to backup the consul data on a regular basis (obviously the
+back_file will need to be uniquely identified for each backup,
+preferably using a timestamp)
+
+
+##### Restore
+
+###### Prerequisites
+* The consul backup file exists and accesible from the voltha/tools container
+
+###### Steps
+
+Restoring a Consul backup implies that the current consul data needs to be
+overwritten. This is typically a disaster recovery scenario.
+
+* Stop all the running voltha vcore instances as well as the consul
+instances. This should be performed from a docker swarm manager node.
+```angular2html
+docker service rm vcore_vcore
+docker service rm consul_consul
+```
+* Start the consul service
+
+```angular2html
+docker stack deploy -c /cord/incubator/voltha/compose/docker-compose-consul-cluster.yml consul
+```
+* Ensure all consul agents are running. There should be 3/3 instances running
+```angular2html
+docker service ls consul_consul
+```
+
+* Get the IP and port number of one of the running consul node.
+
+* Enter the voltha/tools container on one of the cluster machine:
+```angular2html
+docker exec -it <containerid> bash
+cd /home/tools
+```
+* Initiate the restore from a json file (consul_backup.json is just an
+example).
+```angular2html
+consul kv import -http-addr=http://<Consul_IP>:<Consul_PORT></Consul_PORT> @consul_backup.json
+```
+
+* The backup data has been restored into Consul
+
+* Start the voltha instances. This should be performed from a docker swarm
+manager node.
+```angular2html
+docker stack deploy -c /cord/incubator/voltha/compose/docker-compose-voltha-swarm.yml vcore
+```
+
+* At this point the cluster should be up and running with the restored data.
+ Please allow some time for all the data to sync up with the existing PON
+ environment before making requests to the cluster.
diff --git a/docs/manuals/user/backup_restore/backup_restore.png b/docs/manuals/user/backup_restore/backup_restore.png
new file mode 100644
index 0000000..ff89e3e
--- /dev/null
+++ b/docs/manuals/user/backup_restore/backup_restore.png
Binary files differ
diff --git a/install/install_consul_cli.sh b/install/install_consul_cli.sh
new file mode 100644
index 0000000..05d7ff1
--- /dev/null
+++ b/install/install_consul_cli.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+apt-get install -y curl unzip
+
+mkdir -p /var/lib/consul
+mkdir -p /usr/share/consul
+mkdir -p /etc/consul/conf.d
+
+consulVersion='0.9.2'
+
+curl -OL https://releases.hashicorp.com/consul/${consulVersion}/consul_${consulVersion}_linux_amd64.zip
+unzip consul_${consulVersion}_linux_amd64.zip
+mv consul /usr/local/bin/consul
