Merge branch 'master' of github.com:open-cloud/xos
diff --git a/xos/configurations/cord/README-VTN.md b/xos/configurations/cord/README-VTN.md
index 38fc2b4..be31fad 100644
--- a/xos/configurations/cord/README-VTN.md
+++ b/xos/configurations/cord/README-VTN.md
@@ -2,12 +2,18 @@
see also: https://github.com/hyunsun/documentations/wiki/Neutron-ONOS-Integration-for-CORD-VTN#onos-setup
-inside the xos container:
+VTN doesn't seem to like cloudlab's networks (flat-net-1, ext-net, etc). I've placed a script in xos/scripts/ called destroy-all-networks.sh that will automate tearing down all of cloudlab's neutron networks.
+
+ cd xos/scripts
+ ./destroy-all-networks.sh
+
+inside the xos container, update the configuration. Make sure to restart Openstack Synchronizer afterward. Might be a good idea to restart the XOS UI as well:
python /opt/xos/tosca/run.py padmin@vicci.org /opt/xos/tosca/samples/vtn.yaml
emacs /opt/xos/xos_configuration/xos_common_config
[networking]
use_vtn=True
+ supervisorctl restart observer
ctl node:
@@ -53,8 +59,6 @@
# 4) delete any existing br-int bridge
# 5) [nm only] turn off neutron-dhcp-agent
-VTN doesn't seem to like cloudlab's networks (flat-net-1, ext-net, etc). You might have to delete them all. I've placed a script in xos/scripts/ called destroy-all-networks.sh that will automate tearing down all of cloudlab's neutron networks.
-
For development, I suggest using the bash configuration (remember to start the ONOS observer manually) so that
there aren't a bunch of preexisting Neutron networks and nova instances to get in the way.
@@ -64,3 +68,16 @@
also listening on 6640.
* Adding use_vtn=True to the [networking] section in the XOS config file has two effects: 1) it sets the gateway in sync_controller_networks, and 2) it disables automatic creation of nat-net for new slices. This is because VTN will fail if there is no gateway on a network, and because we don't have nat-net under the VTN configuration.
* When using of-vfctl to look at flow rules, if you get a protocol error, try "ovs-ofctl show -O OpenFlow13 br-int "
+
+There is no management network yet, so no way to SSH into the slices. I've been setting up a VNC tunnel, like this:
+
+ # on compute node, run the following and note the IP address and port number
+ virsh vncdisplay <instance-id>
+
+ # from home
+ ssh -o "GatewayPorts yes" -L <port+5900>:<IP>:<port+5900> <username>@<compute_node_hostname>
+
+ # example
+ ssh -o "GatewayPorts yes" -L 5901:192.168.0.7:5901 smbaker@cp-1.smbaker-xos3.xos-pg0.clemson.cloudlab.us
+
+Then open a VNC session to the local port on your local machine. You'll have a console on the Instance. The username is "Ubuntu" and the password can be obtained from your cloudlab experiment description
diff --git a/xos/configurations/vtn/Makefile b/xos/configurations/vtn/Makefile
new file mode 100644
index 0000000..8f77036
--- /dev/null
+++ b/xos/configurations/vtn/Makefile
@@ -0,0 +1,43 @@
+SETUPDIR:=../setup
+MYIP:=$(shell hostname -i)
+
+cloudlab: common_cloudlab xos
+
+devstack: upgrade_pkgs common_devstack xos
+
+xos: vtn_network_cfg_json
+ sudo MYIP=$(MYIP) docker-compose up -d
+ bash ../common/wait_for_xos.sh
+ sudo MYIP=$(MYIP) docker-compose run xos python /opt/xos/tosca/run.py padmin@vicci.org /opt/xos/configurations/common/base.yaml
+ sudo MYIP=$(MYIP) docker-compose run xos python /opt/xos/tosca/run.py padmin@vicci.org /root/setup/nodes.yaml
+ sudo MYIP=$(MYIP) docker-compose run xos python /opt/xos/tosca/run.py padmin@vicci.org /opt/xos/tosca/samples/vtn.yaml
+
+common_cloudlab:
+ make -C ../common -f Makefile.cloudlab
+
+common_devstack:
+ make -C ../common -f Makefile.devstack
+
+vtn_network_cfg_json:
+ export SETUPDIR=$(SETUPDIR); bash ../cord/make-vtn-networkconfig-json.sh
+
+stop:
+ sudo MYIP=$(MYIP) docker-compose stop
+
+showlogs:
+ sudo MYIP=$(MYIP) docker-compose logs
+
+rm: stop
+ sudo MYIP=$(MYIP) docker-compose rm
+
+ps:
+ sudo MYIP=$(MYIP) docker-compose ps
+
+enter-xos:
+ sudo docker exec -it vtn_xos_1 bash
+
+enter-synchronizer:
+ sudo docker exec -it vtn_xos_synchronizer_openstack_1 bash
+
+upgrade_pkgs:
+ sudo pip install httpie --upgrade
diff --git a/xos/configurations/vtn/README.md b/xos/configurations/vtn/README.md
new file mode 100644
index 0000000..5d8eb12
--- /dev/null
+++ b/xos/configurations/vtn/README.md
@@ -0,0 +1,68 @@
+# XOS+VTN development environment
+
+This goal is to use this configuration to do basic end-to-end development of XOS and VTN.
+It launches
+XOS in three Docker containers (development GUI, Synchronizer, database) and configures XOS
+to talk to an OpenStack backend. *docker-compose* is used to manage the containers.
+
+See [the VTN README](../cord/README-VTN.md) for more information.
+
+## How to run it
+
+The configuration can be either run on [CloudLab](http://cloudlab.us) (controlling
+an OpenStack backend set up by a CloudLab profile) or used with a basic
+[DevStack](http://docs.openstack.org/developer/devstack/) configuration.
+
+### CloudLab
+
+To get started on CloudLab:
+* Create an experiment using the *OpenStack* profile. Choose *Kilo* and
+disable security groups.
+* Wait until you get an email from CloudLab with title "OpenStack Instance Finished Setting Up".
+* Login to the *ctl* node of your experiment and run:
+```
+ctl:~$ git clone https://github.com/open-cloud/xos.git
+ctl:~$ cd xos/xos/configurations/devel/
+ctl:~/xos/xos/configurations/devel$ make cloudlab
+```
+
+### DevStack
+
+The following instructions can be used to install DevStack and XOS together
+on a single node. This setup has been run successfully in a VirtualBox VM
+with 2 CPUs and 4096 GB RAM.
+
+First, if you happen to be installing DevStack on a CloudLab node, you can
+configure about 1TB of unallocated disk space for DevStack as follows:
+```
+~$ sudo mkdir -p /opt/stack
+~$ sudo /usr/testbed/bin/mkextrafs /opt/stack
+```
+
+To install DevStack and XOS:
+
+```
+~$ git clone https://github.com/open-cloud/xos.git
+~$ git clone https://git.openstack.org/openstack-dev/devstack
+~$ cd devstack
+~/devstack$ cp ../xos/xos/configurations/common/devstack/local.conf .
+~/devstack$ ./stack.sh
+~/devstack$ cd ../xos/xos/configurations/devel/
+~/xos/xos/configurations/devel$ make devstack
+```
+
+## Docker Helpers
+
+Stop the containers: `make stop`
+
+Restart the containers: `make stop; make [cloudlab|devstack]`
+
+Delete the containers and relaunch them: `make rm; make [cloudlab|devstack]`
+
+View logs: `make showlogs`
+
+See what containers are running: `make ps`
+
+Open a shell on the XOS container: `make enter-xos`
+
+Open a shell on the Synchronizer container: `make enter-synchronizer`
diff --git a/xos/configurations/vtn/docker-compose.yml b/xos/configurations/vtn/docker-compose.yml
new file mode 100644
index 0000000..406ac80
--- /dev/null
+++ b/xos/configurations/vtn/docker-compose.yml
@@ -0,0 +1,54 @@
+xos_db:
+ image: xosproject/xos-postgres
+ expose:
+ - "5432"
+
+xos_synchronizer_openstack:
+ image: xosproject/xos-synchronizer-openstack
+ command: bash -c "python /opt/xos/xos-observer.py"
+ labels:
+ org.xosproject.kind: synchronizer
+ org.xosproject.target: openstack
+ links:
+ - xos_db
+ extra_hosts:
+ - ctl:${MYIP}
+ volumes:
+ - ../setup:/root/setup:ro
+ - ../common/xos_common_config:/opt/xos/xos_configuration/xos_common_config:ro
+ - ./xos_vtn_config:/opt/xos/xos_configuration/xos_vtn_config:ro
+
+
+xos_synchronizer_onos:
+ image: xosproject/xos-synchronizer-openstack
+ command: bash -c "python /opt/xos/observers/onos/onos-observer.py -C /opt/xos/observers/onos/onos_observer_config"
+ labels:
+ org.xosproject.kind: synchronizer
+ org.xosproject.target: onos
+ links:
+ - xos_db
+ extra_hosts:
+ - ctl:${MYIP}
+ volumes:
+ - ../setup:/root/setup:ro
+ - ../setup/id_rsa:/opt/xos/observers/onos/onos_key:ro # private key
+
+# FUTURE
+#xos_swarm_synchronizer:
+# image: xosproject/xos-swarm-synchronizer
+# labels:
+# org.xosproject.kind: synchronizer
+# org.xosproject.target: swarm
+
+xos:
+ image: xosproject/xos
+ command: python /opt/xos/manage.py runserver 0.0.0.0:8000 --insecure --makemigrations
+ ports:
+ - "9999:8000"
+ links:
+ - xos_db
+ volumes:
+ - ../setup:/root/setup:ro
+ - ../common/xos_common_config:/opt/xos/xos_configuration/xos_common_config:ro
+ - ./xos_vtn_config:/opt/xos/xos_configuration/xos_vtn_config:ro
+ - ../setup/id_rsa.pub:/opt/xos/observers/onos/onos_key.pub:ro
diff --git a/xos/configurations/vtn/xos_vtn_config b/xos/configurations/vtn/xos_vtn_config
new file mode 100644
index 0000000..5dfd459
--- /dev/null
+++ b/xos/configurations/vtn/xos_vtn_config
@@ -0,0 +1,2 @@
+[networking]
+use_vtn=True
diff --git a/xos/xos-observer.py b/xos/xos-observer.py
index b85b29b..1b85518 100644
--- a/xos/xos-observer.py
+++ b/xos/xos-observer.py
@@ -4,6 +4,10 @@
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "xos.settings")
from observer.backend import Backend
from xos.config import Config, DEFAULT_CONFIG_FN
+from core.models import Instance
+from util.logger import Logger, logging, logger
+from django.db import ProgrammingError
+import time
try:
from django import setup as django_setup # django 1.7
@@ -54,6 +58,18 @@
if django_setup: # 1.7
django_setup()
+ models_active = False
+ wait = False
+ while not models_active:
+ try:
+ _ = Instance.objects.first()
+ models_active = True
+ except ProgrammingError:
+ logger.info('Waiting for data model to come up before starting...')
+ wait = True
+
+ if (wait):
+ time.sleep(60) # Safety factor, seeing that we stumbled waiting for the data model to come up.
backend = Backend()
backend.run()