Update container ubuntu version, and switch to ansible PPA
remove vestigal Dockerfile
updated Dockerfile to LTS versions of Django, jQuery. Removed use of easy_install (using pip instead)
apply changes made to Dockerfile to devel/templ dockerfiles
have to quote version specification with commas, whitespace
"null has no effect on ManyToManyField" per django 1.8
Have to supply fields = '__all__' in class Meta in 1.8
more additions of default __all__ fields in Meta
more fields = '__all__'
another fields = '__all__'
increase keysize to 2048
stop including event_manager.py which was commented out. This removes the fofum dep
change from nova_compute to os_server
let os_* ansible modules work
Make compatible with django 1.8
rename openstack due to module name conflict with upstream
s/openstack/openstack_xos/
s/openstack/openstack_xos/ module renaming
don't specify ancient openstack client versions
specify keystone API 2.0
_ not .
update to novaclient v2
s/quantum/neutron/ and fix client
fix get_query_set v. get_queryset per django change
revert testing change
Fixme: NovaDB disables, so disable the code that relies on it and throws runtime errors
tracebacks when a RuntimeWarning: DateTimeField xxxx received a naive datet
ime
more commenting out of nonfunctional code
don't include gateway if not set
sort imports, switch to xos.logger
comment out defective function
fix django deprecation warning
remove unused import which was causing a deprecation warning
fix more deprecation warnings
Django 1.8 requires Meta to have fields specified
make creation of tenant it's own step
fix order
os_ ansible module fix
turn off django DEBUG
switch to os_* modules for networking
os_* module-ify user creation
syntax
use legacy keystone_user to create roles
use dict keys appropriate for os_* module results
Make admin role name a config variable as it varies between openstack deployments
set 'body_format' properly, as it's an Ansible 2.0 feature that was being ignored earlier and was set incorrectly for json data
s/queryset/get_queryset/ in admin.py of services using proxy models
fix wget > curl subsitution, duplicate pip package
django 1.8 requires fields in Meta for ModelForm children
fields = '__all__' for 1.8 compat
work around no-gateway options that were hacked into custom Ansible
download/checksum jQuery UI
add missing '.min' component to jquery-ui.min.css path
remove PhantomJS
set REQUESTS_CA_BUNDLE in script that runs ansible
change close_connection() to close_old_connections() per 1.8
same close_old_connections fix
use the CA cert for more OS clients
fix sync_ports to use CA cert
More SSL compat fixes
cleanup, fix instance sync
fields attribute to Meta
more fields attribute
note about not placing new services in the services directory
Fixed ui routing conflict with syncNotification
Fixed ui routing conflict with syncNotification
Removed backbone deps and views
Started E2E UI Tests with selenium
Test running locally, still broken in container because of phantomjs
Removed backone templates
download services (WIP)
Makefile for downloading services
download services for frontend config
update test-standalone to download services
update cord-pod to download services
fix service directories
rename old service directories
fix service directory name
add the onos service back in
fix missing keys
updating .gitreview
download services before bootstrapping
rename MySite to mysite in test infrastructure
Add install_dependencies
use separate project for bootstrap
prevent dirs being created as root in cord_pod config
remove onboard directory that was removed in master
look up capitalization issue of 'admin' vs 'Admin' in OpenStack
remove --no-gateway option as it breaks the VTN
turn on DEBUG, as in master
cleanup of onboarding synchronizer Dockerfile
extend ssh timeout in ansbile to work around initial login name lookup failure on vsg container
move 'timeout' directive in ansible config
ansible 2.1 fixes, logic fix on subnet without gateways
revert sync_controller_users.yaml change until we get a sane way to deal with openstack roles
Change-Id: If79756113ec40ea4bff686ac33c9994b4a31b1d1
diff --git a/containers/onboarding_synchronizer/Dockerfile b/containers/onboarding_synchronizer/Dockerfile
index b86cbb1..e17cfd0 100644
--- a/containers/onboarding_synchronizer/Dockerfile
+++ b/containers/onboarding_synchronizer/Dockerfile
@@ -1,4 +1,4 @@
-FROM xosproject/xos-synchronizer-openstack
+FROM xosproject/xos-synchronizer-openstack
# Install docker-in-docker (dind). See https://hub.docker.com/_/docker/. The docker git repo
# currently only has 1.10 and 1.11, but it's possible to get the dockerfiles for earlier
@@ -7,27 +7,21 @@
# alias dfimage="docker run -v /var/run/docker.sock:/var/run/docker.sock --rm centurylink/dockerfile-from-image"
# dgimage <name of image>
-# This container must be started in privileged mode.
+# This container must be started in privileged mode.
-RUN apt-get install -y curl
+# Install docker
+ENV DOCKER_URL https://get.docker.com/builds/Linux/x86_64/docker-1.10.3
+ENV DOCKER_SHA256 d0df512afa109006a450f41873634951e19ddabf8c7bd419caeb5a526032d86d
-# XXX version 1.10.3
-ENV DOCKER_BUCKET get.docker.com
-ENV DOCKER_VERSION 1.10.3
-ENV DOCKER_SHA256 d0df512afa109006a450f41873634951e19ddabf8c7bd419caeb5a526032d86d
-RUN curl -fSL "https://${DOCKER_BUCKET}/builds/Linux/x86_64/docker-$DOCKER_VERSION" -o /usr/local/bin/docker && echo "${DOCKER_SHA256} /usr/local/bin/docker" | sha256sum -c - && chmod +x /usr/local/bin/docker
+RUN curl -fLsS $DOCKER_URL -o docker && \
+ echo "${DOCKER_SHA256} docker" | sha256sum -c - && \
+ mv docker /usr/local/bin/docker && \
+ chmod +x /usr/local/bin/docker
-# XXX vserioin 1.8.3
-#ENV DOCKER_BUCKET=get.docker.com
-#ENV DOCKER_VERSION=1.8.3
-#ENV DOCKER_SHA256=f024bc65c45a3778cf07213d26016075e8172de8f6e4b5702bedde06c241650f
-#RUN curl -fSL "https://${DOCKER_BUCKET}/builds/Linux/x86_64/docker-$DOCKER_VERSION" -o /usr/local/bin/docker && echo "${DOCKER_SHA256} /usr/local/bin/docker" | sha256sum -c - && chmod +x /usr/local/bin/docker
-
-# XXX version 1.8.3
# XXX uncomment the following 6 lines to run docker-in-docker
# comment them out if using the docker socket in a volume instead
#ENV DIND_COMMIT=3b5fac462d21ca164b3778647420016315289034
-#RUN wget "https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind" -O /usr/local/bin/dind && chmod +x /usr/local/bin/dind
+#RUN wget "https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind" -O /usr/local/bin/dind && chmod +x /sr/local/bin/dind
#COPY start-dockerd.sh /usr/local/bin/
#VOLUME /var/lib/docker
#EXPOSE 2375
@@ -38,11 +32,15 @@
# - /var/run/docker.sock:/var/run/docker.sock
# This is more convenient, allowing us to build directly into our
# parent's docker build system, making the images available for
-# instantiation on the parent.
+# instantiation on the parent.
-# Now install docker-compose
+# Install docker-compose
+ENV DOCKER_COMPOSE_URL https://github.com/docker/compose/releases/download/1.5.2/docker-compose-Linux-x86_64
+ENV DOCKER_COMPOSE_SHA256 b6b975badc5389647ef1c16fe8a33bdc5935c61f6afd5a15a28ff765427d01e3
-RUN bash -c "curl -L https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose"
-RUN chmod +x /usr/local/bin/docker-compose
+RUN curl -fLsS $DOCKER_COMPOSE_URL -o docker-compose && \
+ echo "${DOCKER_COMPOSE_SHA256} docker-compose" | sha256sum -c - && \
+ mv docker-compose /usr/local/bin/docker-compose && \
+ chmod +x /usr/local/bin/docker-compose
-CMD update-ca-certificates && /usr/bin/supervisord -c /etc/supervisor/conf.d/synchronizer.conf
+CMD /usr/bin/supervisord -c /etc/supervisor/conf.d/synchronizer.conf
diff --git a/containers/xos/Dockerfile.base b/containers/xos/Dockerfile.base
index e653b5a..27d4c52 100644
--- a/containers/xos/Dockerfile.base
+++ b/containers/xos/Dockerfile.base
@@ -1,7 +1,7 @@
# Dockerfile.base
# This image isn't used, but installs the prereqs for the other XOS images
-FROM ubuntu:14.04.3
+FROM ubuntu:14.04.4
# XXX Workaround for docker bug:
# https://github.com/docker/docker/issues/6345
@@ -33,6 +33,7 @@
python-httplib2 \
python-jinja2 \
python-keystoneclient \
+ python-netaddr \
python-neutronclient \
python-novaclient \
python-paramiko \
@@ -42,61 +43,86 @@
python-setuptools \
python-yaml \
rsync \
+ software-properties-common \
supervisor \
tar \
+ unzip \
wget \
&& rm -rf /var/lib/apt/lists/*
# Install python pacakges with pip
RUN pip install \
- django==1.7 \
+ ca-certs-locater \
+ "django>=1.8,<1.8.99" \
django-bitfield \
django-crispy-forms \
django-encrypted-fields \
django-extensions \
- django-filter==0.11.0 \
+ django-filter \
django-geoposition \
django-ipware \
- django_rest_swagger \
- django-suit==0.3a1 \
+ django-suit==0.3a3 \
django-timezones \
+ django_rest_swagger \
djangorestframework==3.3.3 \
dnslib \
+ google-api-python-client \
+ httplib2 \
jinja2 \
lxml \
markdown \
netaddr \
- pyOpenSSL \
psycopg2 \
+ pyOpenSSL \
+ pygraphviz \
python-ceilometerclient \
python-dateutil \
+ python-gflags \
python-keyczar \
python-logstash \
- pygraphviz \
pytz \
pyyaml \
requests
-# Upgrade jinja2
-RUN pip install -U \
- jinja2
+# Required for ansible 2.x OpenStack os_* modules
+RUN pip install --upgrade pbr && \
+ pip install --upgrade six && \
+ pip install shade
-# Installs with Easy install (should be incorporated into pip?)
-RUN easy_install --upgrade httplib2
+# Install ansible 2.x (has to be done after installing `software-properties-common`)
+RUN add-apt-repository ppa:ansible/ansible
+RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y \
+ ansible \
+ && rm -rf /var/lib/apt/lists/*
-RUN easy_install \
- python_gflags \
- google_api_python_client \
- httplib2.ca_certs_locater
+COPY ansible-hosts /etc/ansible/hosts
-# jQuery download w/checksum
-ENV JQUERY_VERSION jquery-1.9.1.min.js
-ENV JQUERY_DL_URL http://code.jquery.com/jquery-1.9.1.min.js
-ENV JQUERY_SHA256 c12f6098e641aaca96c60215800f18f5671039aecf812217fab3c0d152f6adb4
+# jQuery
+ENV JQUERY_DL_URL http://code.jquery.com/jquery-1.12.4.min.js
+ENV JQUERY_SHA256 668b046d12db350ccba6728890476b3efee53b2f42dbb84743e5e9f1ae0cc404
-RUN wget $JQUERY_DL_URL && \
- echo "$JQUERY_SHA256 $JQUERY_VERSION" | sha256sum -c - && \
- mv $JQUERY_VERSION /usr/local/lib/python2.7/dist-packages/suit/static/suit/js/
+RUN curl -fLsS $JQUERY_DL_URL -o jquery.min.js && \
+ echo "$JQUERY_SHA256 jquery.min.js" | sha256sum -c - && \
+ mv jquery.min.js /usr/local/lib/python2.7/dist-packages/suit/static/suit/js/
+
+# jQueryUI
+ENV JQUERYUI_DL_URL https://jqueryui.com/resources/download/jquery-ui-1.11.4.zip
+ENV JQUERYUI_SHA256 503e4c0f109bf627aff87a424edc760608ec15e4a6e37f217a083ca682543e32
+
+RUN curl -fLsS $JQUERYUI_DL_URL -o jquery-ui.zip && \
+ echo "$JQUERYUI_SHA256 jquery-ui.zip" | sha256sum -c - && \
+ unzip jquery-ui.zip && \
+ mv jquery-ui-*/jquery-ui.min.js /usr/local/lib/python2.7/dist-packages/suit/static/suit/js/ && \
+ rm -rf jquery-ui.zip jquery-ui-*
+
+ENV JQUERYUI_THEMES_DL_URL http://jqueryui.com/resources/download/jquery-ui-themes-1.11.4.zip
+ENV JQUERYUI_THEMES_SHA256 df2b9cb084095ea24129a6a54587a1d9d7ae4bcd68bf5ea2957eb3d4d18fe884
+
+RUN curl -fLsS $JQUERYUI_THEMES_DL_URL -o jquery-ui-themes.zip && \
+ echo "$JQUERYUI_THEMES_SHA256 jquery-ui-themes.zip" | sha256sum -c - && \
+ unzip jquery-ui-themes.zip && \
+ mv jquery-ui-themes-*/themes/smoothness/jquery-ui.min.css /usr/local/lib/python2.7/dist-packages/suit/static/suit/css/ && \
+ rm -rf jquery-ui-themes.zip jquery-ui-themes-*
# Install heat-translator for TOSCA support
ENV HT_REPO_URL https://github.com/openstack/heat-translator.git
@@ -110,33 +136,3 @@
echo > /opt/tosca/translator/__init__.py && \
rm -rf /tmp/heat-translator
-# Install custom Ansible
-RUN \
- git clone -b release1.8.2 git://github.com/ansible/ansible.git /opt/ansible && \
- git clone -b release1.8.2 git://github.com/ansible/ansible-modules-extras.git /opt/ansible/lib/ansible/modules/extras && \
- git clone -b release1.8.2 git://github.com/ansible/ansible-modules-extras.git /opt/ansible/v2/ansible/modules/extras && \
- git clone git://github.com/sb98052/ansible-modules-core.git /opt/ansible/lib/ansible/modules/core && \
- git clone git://github.com/sb98052/ansible-modules-core.git /opt/ansible/v2/ansible/modules/core && \
- # git clone uses cached copy, doesn't pick up latest
- git -C /opt/ansible pull && \
- git -C /opt/ansible/lib/ansible/modules/core pull && \
- git -C /opt/ansible/v2/ansible/modules/core pull && \
- mkdir -p /usr/local/share /bin /etc/ansible
-
-COPY ansible-hosts /etc/ansible/hosts
-
-# For Synchronizer
-# ENV PHANTOMJS_DL_URL http://phantomjs.googlecode.com/files/phantomjs-1.7.0-linux-x86_64.tar.bz2
-# ENV PHANTOMJS_SHA256 a7658f5f2d9464f86891afdb979eb60b754d5f404801db624368ac11e16724d4
-
-# RUN curl -fLsS $PHANTOMJS_DL_URL -o phantomjs.tar.bz2 && \
-# echo "$PHANTOMJS_SHA256 phantomjs.tar.bz2" | sha256sum -c - && \
-# tar -C /usr/local/share -xjf phantomjs.tar.bz2 && \
-# ln -s /usr/local/share/phantomjs-* /usr/local/share/phantomjs && \
-# ln -s /usr/local/share/phantomjs/bin/phantomjs /bin/phantomjs && \
-# rm phantomjs.tar.bz2
-
-#RUN git clone git://git.planet-lab.org/fofum.git /tmp/fofum && \
-# cd /tmp/fofum; python setup.py install && \
-# rm -rf /tmp/fofum
-
diff --git a/xos/configurations-deprecated/acord/xos_cord_config b/xos/configurations-deprecated/acord/xos_cord_config
index a5448f7..4d2d13b 100644
--- a/xos/configurations-deprecated/acord/xos_cord_config
+++ b/xos/configurations-deprecated/acord/xos_cord_config
@@ -4,3 +4,7 @@
branding_icon=/static/cord-logo.png
branding_favicon=/static/cord-favicon.png
branding_bg=/static/cord-bg.jpg
+
+[nova]
+admin_role=Admin
+
diff --git a/xos/configurations-deprecated/common/Dockerfile.common b/xos/configurations-deprecated/common/Dockerfile.common
deleted file mode 100644
index aedd245..0000000
--- a/xos/configurations-deprecated/common/Dockerfile.common
+++ /dev/null
@@ -1,144 +0,0 @@
-FROM ubuntu:14.04.2
-MAINTAINER Andy Bavier <acb@cs.princeton.edu>
-
-# XXX Workaround for docker bug:
-# https://github.com/docker/docker/issues/6345
-# Kernel 3.15 breaks docker, uss the line below as a workaround
-# until there is a fix
-RUN ln -s -f /bin/true /usr/bin/chfn
-# XXX End workaround
-
-# Install.
-RUN apt-get update && apt-get install -y \
- git \
- postgresql \
- python-psycopg2 \
- graphviz \
- graphviz-dev \
- libxslt1.1 \
- libxslt1-dev \
- python-pip \
- tar \
- gcc \
- python-httplib2 \
- geoip-database \
- libgeoip1 \
- wget \
- curl \
- python-dev \
- libyaml-dev \
- pkg-config \
- python-pycurl
-
-RUN pip install django==1.7
-RUN pip install djangorestframework==2.4.4
-RUN pip install markdown # Markdown support for the browseable API.
-RUN pip install pyyaml # YAML content-type support.
-RUN pip install django-filter==0.11.0 # Filtering support
-RUN pip install lxml # XML manipulation library
-RUN pip install netaddr # IP Addr library
-RUN pip install pytz
-RUN pip install django-timezones
-RUN pip install requests
-RUN pip install python-logstash
-RUN pip install django-crispy-forms
-RUN pip install django-geoposition
-RUN pip install django-extensions
-RUN pip install django-suit==0.3a1
-RUN pip install django-bitfield
-RUN pip install django-ipware
-RUN pip install django-encrypted-fields
-RUN pip install python-keyczar
-RUN pip install pygraphviz --install-option="--include-path=/usr/include/graphviz" --install-option="--library-path=/usr/lib/graphviz/"
-RUN pip install dnslib
-
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-keystoneclient
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-novaclient
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-neutronclient
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-glanceclient
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-ceilometerclient
-
-RUN pip install django_rest_swagger
-
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-setuptools
-RUN easy_install python_gflags
-RUN easy_install --upgrade httplib2
-RUN easy_install google_api_python_client
-RUN easy_install httplib2.ca_certs_locater
-
-# Install custom Ansible
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-crypto
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-yaml
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y openssh-client
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-paramiko
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-jinja2
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-httplib2
-RUN git clone -b release1.8.2 git://github.com/ansible/ansible.git /opt/ansible
-RUN git clone -b release1.8.2 git://github.com/ansible/ansible-modules-extras.git /opt/ansible/lib/ansible/modules/extras
-RUN git clone -b release1.8.2 git://github.com/ansible/ansible-modules-extras.git /opt/ansible/v2/ansible/modules/extras
-RUN git clone git://github.com/sb98052/ansible-modules-core.git /opt/ansible/lib/ansible/modules/core
-RUN git clone git://github.com/sb98052/ansible-modules-core.git /opt/ansible/v2/ansible/modules/core
-ADD ansible-hosts /etc/ansible/hosts
-
-ADD http://code.jquery.com/jquery-1.9.1.min.js /usr/local/lib/python2.7/dist-packages/suit/static/suit/js/
-
-# For Observer
-RUN git clone git://git.planet-lab.org/fofum.git /tmp/fofum
-RUN cd /tmp/fofum; python setup.py install
-RUN rm -rf /tmp/fofum
-
-RUN mkdir -p /usr/local/share /bin
-ADD http://phantomjs.googlecode.com/files/phantomjs-1.7.0-linux-x86_64.tar.bz2 /usr/local/share/
-RUN tar jxvf /usr/local/share/phantomjs-1.7.0-linux-x86_64.tar.bz2 -C /usr/local/share/
-RUN rm -f /usr/local/share/phantomjs-1.7.0-linux-x86_64.tar.bz2
-RUN ln -s /usr/local/share/phantomjs-1.7.0-linux-x86_64 /usr/local/share/phantomjs
-RUN ln -s /usr/local/share/phantomjs/bin/phantomjs /bin/phantomjs
-
-# Supervisor
-RUN DEBIAN_FRONTEND=noninteractive apt-get install -y supervisor
-ADD observer.conf /etc/supervisor/conf.d/
-
-# Get XOS
-ADD xos /opt/xos
-
-# Initscript is broken in Ubuntu
-#ADD observer-initscript /etc/init.d/xosobserver
-
-RUN chmod +x /opt/xos/tools/xos-manage
-RUN /opt/xos/tools/xos-manage genkeys
-
-# Workaround for AUFS issue
-# https://github.com/docker/docker/issues/783#issuecomment-56013588
-RUN mkdir /etc/ssl/private-copy; mv /etc/ssl/private/* /etc/ssl/private-copy/; rm -r /etc/ssl/private; mv /etc/ssl/private-copy /etc/ssl/private; chmod -R 0700 /etc/ssl/private; chown -R postgres /etc/ssl/private
-
-# Set postgres password to match default value in settings.py
-RUN service postgresql start; sudo -u postgres psql -c "alter user postgres with password 'password';"
-
-# Turn DEBUG on so that devel server will serve static files
-# (not necessary if --insecure is passed to 'manage.py runserver')
-# RUN sed -i 's/DEBUG = False/DEBUG = True/' /opt/xos/xos/settings.py
-
-# Cruft to workaround problems with migrations, should go away...
-RUN /opt/xos/tools/xos-manage remigrate
-
-# git clone uses cached copy, doesn't pick up latest
-RUN git -C /opt/ansible pull
-RUN git -C /opt/ansible/lib/ansible/modules/core pull
-RUN git -C /opt/ansible/v2/ansible/modules/core pull
-
-# install Tosca engine
-RUN apt-get install -y m4
-RUN pip install python-dateutil
-RUN bash /opt/xos/tosca/install_tosca.sh
-
-EXPOSE 8000
-
-# Set environment variables.
-ENV HOME /root
-
-# Define working directory.
-WORKDIR /root
-
-# Define default command.
-#CMD ["/bin/bash"]
-#CMD /opt/xos/tools/docker_start_xos
diff --git a/xos/configurations-deprecated/common/xos_common_config b/xos/configurations-deprecated/common/xos_common_config
index 76ba747..3a6e464 100644
--- a/xos/configurations-deprecated/common/xos_common_config
+++ b/xos/configurations-deprecated/common/xos_common_config
@@ -29,7 +29,7 @@
default_image=None
default_flavor=m1.small
default_security_group=default
-ca_ssl_cert=/etc/ssl/certs/ca-certificates.crt
+ca_ssl_cert=/usr/local/share/ca-certificates/local_certs.crt
[observer]
pretend=False
diff --git a/xos/configurations-deprecated/mcord/xos/Dockerfile.templ b/xos/configurations-deprecated/mcord/xos/Dockerfile.templ
index 25270a6..3d33821 100644
--- a/xos/configurations-deprecated/mcord/xos/Dockerfile.templ
+++ b/xos/configurations-deprecated/mcord/xos/Dockerfile.templ
@@ -1,4 +1,4 @@
-FROM ubuntu:14.04.3
+FROM ubuntu:14.04.4
# XXX Workaround for docker bug:
# https://github.com/docker/docker/issues/6345
@@ -29,43 +29,45 @@
python-setuptools \
tar \
wget \
-##### observer dependencies
+##### synchronizer dependencies
python-keystoneclient \
python-novaclient \
python-neutronclient \
python-glanceclient \
python-ceilometerclient
-RUN pip install -U \
- django==1.7 \
+RUN pip install \
+ ca-certs-locater \
+ "django>=1.8,<1.8.99" \
django-bitfield \
django-crispy-forms \
django-encrypted-fields \
- django_evolution \
django-extensions \
- django-filter==0.11.0 \
+ django-filter \
django-geoposition \
django-ipware \
django_rest_swagger \
- django-suit==0.3a1 \
+ django-suit==0.3a3 \
django-timezones \
djangorestframework==2.4.4 \
dnslib \
- google_api_python_client \
+ google-api-python-client \
httplib2 \
- httplib2.ca_certs_locater \
lxml \
markdown \
netaddr \
+ pyOpenSSL \
+ psycopg2 \
+ python-ceilometerclient \
python-dateutil \
- python_gflags \
+ python-gflags \
python-keyczar \
pygraphviz \
pytz \
pyyaml \
requests
-ADD http://code.jquery.com/jquery-1.9.1.min.js /usr/local/lib/python2.7/dist-packages/suit/static/suit/js/
+ADD http://code.jquery.com/jquery-1.12.3.min.js /usr/local/lib/python2.7/dist-packages/suit/static/suit/js/
# Install XOS
RUN git clone XOS_GIT_REPO -b XOS_GIT_BRANCH /tmp/xos && \
diff --git a/xos/core/admin.py b/xos/core/admin.py
index cc24d76..ab34183 100644
--- a/xos/core/admin.py
+++ b/xos/core/admin.py
@@ -12,7 +12,7 @@
ReadOnlyPasswordHashField)
from django.contrib.auth.models import Group
from django.contrib.auth.signals import user_logged_in
-from django.contrib.contenttypes import generic
+from django.contrib.contenttypes.admin import GenericTabularInline
from django.core.exceptions import PermissionDenied, ValidationError
from django.core.urlresolvers import NoReverseMatch, resolve, reverse
from django.forms.utils import flatatt, to_current_timezone
@@ -21,7 +21,7 @@
from django.utils.html import conditional_escape, format_html
from django.utils.safestring import mark_safe
from django.utils.text import capfirst
-from openstack.manager import OpenStackManager
+from openstack_xos.manager import OpenStackManager
from suit.widgets import LinkedSelect
# thread locals necessary to work around a django-suit issue
@@ -415,7 +415,7 @@
backend_status_icon.short_description = ""
-class PlStackGenericTabularInline(generic.GenericTabularInline):
+class PlStackGenericTabularInline(GenericTabularInline):
def has_add_permission(self, request):
return not request.user.isReadOnlyUser()
@@ -723,6 +723,7 @@
class Meta:
model = Deployment
many_to_many = ["flavors", ]
+ fields = '__all__'
def __init__(self, *args, **kwargs):
request = kwargs.pop('request', None)
@@ -857,6 +858,7 @@
class Meta:
model = Controller
+ fields = '__all__'
def __init__(self, *args, **kwargs):
request = kwargs.pop('request', None)
@@ -1215,6 +1217,7 @@
widgets = {
'service': LinkedSelect
}
+ fields = '__all__'
def clean(self):
cleaned_data = super(SliceForm, self).clean()
@@ -1443,6 +1446,7 @@
'site': LinkedSelect,
'deployment': LinkedSelect
}
+ fields = '__all__'
def __init__(self, *args, **kwargs):
request = kwargs.pop('request', None)
@@ -1512,6 +1516,7 @@
'node': LinkedSelect,
'image': LinkedSelect
}
+ fields = '__all__'
class TagAdmin(XOSBaseAdmin):
@@ -1724,6 +1729,7 @@
class Meta:
model = User
widgets = {'public_key': UploadTextareaWidget, }
+ fields = '__all__'
def clean_password(self):
# Regardless of what the user provides, return the initial value.
@@ -1758,6 +1764,7 @@
class Meta:
app_label = "core"
+ fields = '__all__'
# The forms to add and change user instances
form = UserChangeForm
@@ -1920,6 +1927,7 @@
widgets = {
'slice': LinkedSelect
}
+ fields = '__all__'
class ReservationAddForm(forms.ModelForm):
@@ -1944,6 +1952,7 @@
widgets = {
'slice': LinkedSelect
}
+ fields = '__all__'
class ReservationAddRefreshForm(ReservationAddForm):
@@ -2109,6 +2118,7 @@
'topologyParameters': UploadTextareaWidget,
'controllerParameters': UploadTextareaWidget,
}
+ fields = '__all__'
class NetworkAdmin(XOSBaseAdmin):
@@ -2334,6 +2344,7 @@
'messages': forms.Textarea(attrs={'rows': 20, 'cols': 80, 'class': 'input-xxlarge'}),
'output': forms.Textarea(attrs={'rows': 3, 'cols': 80, 'class': 'input-xxlarge'})
}
+ fields = '__all__'
class ProgramAdmin(XOSBaseAdmin):
@@ -2374,6 +2385,7 @@
widgets = {
'addresses': UploadTextareaWidget(attrs={'rows': 20, 'cols': 80, 'class': "input-xxlarge"}),
}
+ fields = '__all__'
class AddressPoolAdmin(XOSBaseAdmin):
diff --git a/xos/core/models/billing.py b/xos/core/models/billing.py
index 6e517b4..48c8a38 100644
--- a/xos/core/models/billing.py
+++ b/xos/core/models/billing.py
@@ -5,7 +5,6 @@
from core.models import PlCoreBase, Site, Slice, Instance, Deployment
from core.models.plcorebase import StrippedCharField
from django.contrib.contenttypes.models import ContentType
-from django.contrib.contenttypes import generic
from django.db.models import Sum
from django.utils import timezone
diff --git a/xos/core/models/dashboard.py b/xos/core/models/dashboard.py
index 330a0d6..9b80ba8 100644
--- a/xos/core/models/dashboard.py
+++ b/xos/core/models/dashboard.py
@@ -3,7 +3,6 @@
from core.models import PlCoreBase, Controller, Deployment
from core.models.plcorebase import StrippedCharField
from core.models.site import ControllerLinkManager, ControllerLinkDeletionManager
-from django.contrib.contenttypes import generic
class DashboardView(PlCoreBase):
name = StrippedCharField(max_length=200, unique=True, help_text="Name of the View")
diff --git a/xos/core/models/flavor.py b/xos/core/models/flavor.py
index 8251eb1..693a5f8 100644
--- a/xos/core/models/flavor.py
+++ b/xos/core/models/flavor.py
@@ -4,7 +4,6 @@
from core.models import PlCoreBase, Deployment
from core.models.plcorebase import StrippedCharField
from django.contrib.contenttypes.models import ContentType
-from django.contrib.contenttypes import generic
class Flavor(PlCoreBase):
name = StrippedCharField(max_length=32, help_text="name of this flavor, as displayed to users")
diff --git a/xos/core/models/instance.py b/xos/core/models/instance.py
index 8cd889a..6e73af8 100644
--- a/xos/core/models/instance.py
+++ b/xos/core/models/instance.py
@@ -13,7 +13,7 @@
from core.models import User
from core.models import Tag
from core.models import Flavor
-from django.contrib.contenttypes import generic
+from django.contrib.contenttypes.fields import GenericRelation
from xos.config import Config
from django.core.exceptions import PermissionDenied, ValidationError
@@ -95,7 +95,7 @@
node = models.ForeignKey(Node, related_name='instances')
numberCores = models.IntegerField(verbose_name="Number of Cores", help_text="Number of cores for instance", default=0)
flavor = models.ForeignKey(Flavor, help_text="Flavor of this instance", default=get_default_flavor)
- tags = generic.GenericRelation(Tag)
+ tags = GenericRelation(Tag)
userData = models.TextField(blank=True, null=True, help_text="user_data passed to instance during creation")
isolation = models.CharField(null=False, blank=False, max_length=30, choices=ISOLATION_CHOICES, default="vm")
volumes = models.TextField(null=True, blank=True, help_text="Comma-separated list of directories to expose to parent context")
diff --git a/xos/core/models/network.py b/xos/core/models/network.py
index 37f4cfe..b5269ad 100644
--- a/xos/core/models/network.py
+++ b/xos/core/models/network.py
@@ -5,7 +5,7 @@
from core.models import PlCoreBase, Site, Slice, Instance, Controller, Service
from core.models import ControllerLinkManager,ControllerLinkDeletionManager
from django.contrib.contenttypes.models import ContentType
-from django.contrib.contenttypes import generic
+from django.contrib.contenttypes.fields import GenericForeignKey
from django.core.exceptions import ValidationError
from django.db.models import Q
@@ -347,7 +347,7 @@
# The required fields to do a ObjectType lookup, and object_id assignment
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
- content_object = generic.GenericForeignKey('content_type', 'object_id')
+ content_object = GenericForeignKey('content_type', 'object_id')
def __unicode__(self):
return self.parameter.name
diff --git a/xos/core/models/node.py b/xos/core/models/node.py
index f3ea303..cefb48d 100644
--- a/xos/core/models/node.py
+++ b/xos/core/models/node.py
@@ -4,7 +4,7 @@
from core.models.plcorebase import StrippedCharField
from core.models.site import Site, SiteDeployment, SitePrivilege
from core.models import Tag
-from django.contrib.contenttypes import generic
+from django.contrib.contenttypes.fields import GenericRelation
# Create your models here.
@@ -12,7 +12,7 @@
name = StrippedCharField(max_length=200, unique=True, help_text="Name of the Node")
site_deployment = models.ForeignKey(SiteDeployment, related_name='nodes')
site = models.ForeignKey(Site, null=True, blank=True, related_name='nodes')
- tags = generic.GenericRelation(Tag)
+ tags = GenericRelation(Tag)
def __unicode__(self): return u'%s' % (self.name)
diff --git a/xos/core/models/role.py b/xos/core/models/role.py
index 353139e..b259410 100644
--- a/xos/core/models/role.py
+++ b/xos/core/models/role.py
@@ -2,7 +2,6 @@
import datetime
from django.db import models
from django.contrib.contenttypes.models import ContentType
-from django.contrib.contenttypes import generic
from core.models import PlCoreBase
from core.models.plcorebase import StrippedCharField
diff --git a/xos/core/models/serviceclass.py b/xos/core/models/serviceclass.py
index ccc3180..a51b476 100644
--- a/xos/core/models/serviceclass.py
+++ b/xos/core/models/serviceclass.py
@@ -16,7 +16,7 @@
membershipFee = models.IntegerField(default=0)
membershipFeeMonths = models.IntegerField(default=12)
upgradeRequiresApproval = models.BooleanField(default=False)
- upgradeFrom = models.ManyToManyField('self', blank=True, null=True)
+ upgradeFrom = models.ManyToManyField('self', blank=True)
class Meta(PlCoreBase.Meta):
verbose_name_plural = "Service classes"
diff --git a/xos/core/models/site.py b/xos/core/models/site.py
index 77b96ac..31e69a6 100644
--- a/xos/core/models/site.py
+++ b/xos/core/models/site.py
@@ -1,7 +1,7 @@
import os
from django.db import models
from django.db.models import Q
-from django.contrib.contenttypes import generic
+from django.contrib.contenttypes.fields import GenericRelation
from django.core.exceptions import PermissionDenied
from geoposition.fields import GeopositionField
from core.models import PlCoreBase,PlCoreBaseManager,PlCoreBaseDeletionManager
@@ -110,7 +110,7 @@
#deployments = models.ManyToManyField('Deployment', blank=True, related_name='sites')
deployments = models.ManyToManyField('Deployment', through='SiteDeployment', blank=True, help_text="Select which sites are allowed to host nodes in this deployment", related_name='sites')
- tags = generic.GenericRelation(Tag)
+ tags = GenericRelation(Tag)
def __unicode__(self): return u'%s' % (self.name)
diff --git a/xos/core/models/slice.py b/xos/core/models/slice.py
index 73fa121..33e5521 100644
--- a/xos/core/models/slice.py
+++ b/xos/core/models/slice.py
@@ -9,7 +9,7 @@
from core.models import ServiceClass
#from core.models.serviceclass import get_default_serviceclass
from core.models import Tag
-from django.contrib.contenttypes import generic
+from django.contrib.contenttypes.fields import GenericRelation
from core.models import Service
from core.models import Controller
from core.models.node import Node
@@ -34,7 +34,7 @@
service = models.ForeignKey(Service, related_name='slices', null=True, blank=True)
network = models.CharField(null=True, blank=True, max_length=256, choices=NETWORK_CHOICES)
exposed_ports = models.CharField(null=True, blank=True, max_length=256)
- tags = generic.GenericRelation(Tag)
+ tags = GenericRelation(Tag)
serviceClass = models.ForeignKey(ServiceClass, related_name = "slices", null=True, blank=True) # DEPRECATED
creator = models.ForeignKey(User, related_name='slices', blank=True, null=True)
diff --git a/xos/core/models/tag.py b/xos/core/models/tag.py
index 76a4e2e..7bd7aea 100644
--- a/xos/core/models/tag.py
+++ b/xos/core/models/tag.py
@@ -4,7 +4,7 @@
from core.models import Service
from core.models.plcorebase import StrippedCharField
from django.contrib.contenttypes.models import ContentType
-from django.contrib.contenttypes import generic
+from django.contrib.contenttypes.fields import GenericForeignKey
# Create your models here.
@@ -18,7 +18,7 @@
# The required fields to do a ObjectType lookup, and object_id assignment
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
- content_object = generic.GenericForeignKey('content_type', 'object_id')
+ content_object = GenericForeignKey('content_type', 'object_id')
def __unicode__(self):
return self.name
diff --git a/xos/core/models/user.py b/xos/core/models/user.py
index 715c670..d85c58c 100644
--- a/xos/core/models/user.py
+++ b/xos/core/models/user.py
@@ -85,7 +85,11 @@
class DeletedUserManager(UserManager):
def get_queryset(self):
- return super(UserManager, self).get_query_set().filter(deleted=True)
+ parent = super(UserManager, self)
+ if hasattr(parent, "get_queryset"):
+ return parent.get_queryset().filter(deleted=True)
+ else:
+ return parent.get_query_set().filter(deleted=True)
# deprecated in django 1.7 in favor of get_queryset()
def get_query_set(self):
@@ -294,7 +298,7 @@
def can_update_root(self):
"""
- Return True if user has root (global) write access.
+ Return True if user has root (global) write access.
"""
if self.is_readonly:
return False
@@ -386,15 +390,15 @@
return readable_objects
def get_permissions(self, filter_by=None):
- """ Return a list of objects for which the user has read or read/write
- access. The object will be an instance of a django model object.
+ """ Return a list of objects for which the user has read or read/write
+ access. The object will be an instance of a django model object.
Permissions will be either 'r' or 'rw'.
e.g.
[{'object': django_object_instance, 'permissions': 'rw'}, ...]
Returns:
- list of dicts
+ list of dicts
"""
from core.models import Deployment, Flavor, Image, Network, NetworkTemplate, Node, PlModelMixIn, Site, Slice, SliceTag, Instance, Tag, User, DeploymentPrivilege, SitePrivilege, SlicePrivilege
diff --git a/xos/openstack/__init__.py b/xos/openstack_xos/__init__.py
similarity index 100%
rename from xos/openstack/__init__.py
rename to xos/openstack_xos/__init__.py
diff --git a/xos/openstack/client.py b/xos/openstack_xos/client.py
similarity index 75%
rename from xos/openstack/client.py
rename to xos/openstack_xos/client.py
index c500204..d1383f4 100644
--- a/xos/openstack/client.py
+++ b/xos/openstack_xos/client.py
@@ -1,10 +1,12 @@
import urlparse
try:
+ from keystoneauth1.identity import v2 as keystoneauth_v2
+ from keystoneauth1 import session as keystone_session
from keystoneclient.v2_0 import client as keystone_client
#from glance import client as glance_client
import glanceclient
- from novaclient.v1_1 import client as nova_client
- from neutronclient.v2_0 import client as quantum_client
+ from novaclient.v2 import client as nova_client
+ from neutronclient.v2_0 import client as neutron_client
has_openstack = True
except:
has_openstack = False
@@ -62,7 +64,10 @@
if endpoint:
self.endpoint = endpoint
- self.cacert = cacert
+ if cacert:
+ self.cacert = cacert
+ else:
+ self.cacert = getattr(Config(), "nova_ca_ssl_cert", "None")
#if '@' in self.username:
# self.username = self.username[:self.username.index('@')]
@@ -71,11 +76,13 @@
def __init__(self, *args, **kwds):
Client.__init__(self, *args, **kwds)
if has_openstack:
- self.client = keystone_client.Client(username=self.username,
- password=self.password,
- tenant_name=self.tenant,
- auth_url=self.url
- )
+ auth = keystoneauth_v2.Password(username=self.username,
+ password=self.password,
+ tenant_name=self.tenant,
+ auth_url=self.url,
+ )
+ sess = keystone_session.Session(auth=auth, verify=self.cacert, )
+ self.client = keystone_client.Client(session=sess)
@require_enabled
def connect(self, *args, **kwds):
@@ -94,7 +101,9 @@
username=self.username,
password=self.password,
tenant=self.tenant,
- auth_url=self.url)
+ auth_url=self.url,
+ cacert=self.cacert
+ )
@require_enabled
def __getattr__(self, name):
return getattr(self.client, name)
@@ -117,15 +126,18 @@
def __init__(self, *args, **kwds):
Client.__init__(self, *args, **kwds)
if has_openstack:
- self.client = nova_client.Client(username=self.username,
- api_key=self.password,
- project_id=self.tenant,
- auth_url=self.url,
- region_name='',
- extensions=[],
- service_type='compute',
- service_name='',
- )
+ self.client = nova_client.client.Client(
+ "2",
+ username=self.username,
+ api_key=self.password,
+ project_id=self.tenant,
+ auth_url=self.url,
+ region_name='',
+ extensions=[],
+ service_type='compute',
+ service_name='',
+ cacert=self.cacert
+ )
@require_enabled
def connect(self, *args, **kwds):
@@ -152,15 +164,16 @@
def __getattr__(self, name):
return getattr(self.client, name)
-class QuantumClient(Client):
+class NeutronClient(Client):
def __init__(self, *args, **kwds):
Client.__init__(self, *args, **kwds)
if has_openstack:
- self.client = quantum_client.Client(username=self.username,
+ self.client = neutron_client.Client(username=self.username,
password=self.password,
tenant_name=self.tenant,
auth_url=self.url,
- ca_cert=self.cacert)
+ ca_cert=self.cacert
+ )
@require_enabled
def connect(self, *args, **kwds):
self.__init__(*args, **kwds)
@@ -181,12 +194,11 @@
url_parsed = urlparse.urlparse(self.keystone.url)
hostname = url_parsed.netloc.split(':')[0]
token = self.keystone.client.tokens.authenticate(username=self.keystone.username, password=self.keystone.password, tenant_name=self.keystone.tenant)
- glance_endpoint = self.keystone.service_catalog.url_for(service_type='image', endpoint_type='publicURL')
-
- self.glanceclient = GlanceClient('1', endpoint=glance_endpoint, token=token.id, **kwds)
+# glance_endpoint = self.keystone.client.service_catalog.url_for(service_type='image', endpoint_type='publicURL')
+# self.glanceclient = GlanceClient('1', endpoint=glance_endpoint, token=token.id, **kwds)
self.nova = NovaClient(*args, **kwds)
# self.nova_db = NovaDB(*args, **kwds)
- self.quantum = QuantumClient(*args, **kwds)
+ self.neutron = NeutronClient(*args, **kwds)
@require_enabled
diff --git a/xos/openstack/driver.py b/xos/openstack_xos/driver.py
similarity index 82%
rename from xos/openstack/driver.py
rename to xos/openstack_xos/driver.py
index 15668b9..3064eb2 100644
--- a/xos/openstack/driver.py
+++ b/xos/openstack_xos/driver.py
@@ -4,7 +4,7 @@
from core.models import Controller
try:
- from openstack.client import OpenStackClient
+ from openstack_xos.client import OpenStackClient
has_openstack = True
except:
has_openstack = False
@@ -51,7 +51,7 @@
driver = OpenStackDriver(client=client)
driver.admin_user = client.keystone.users.find(name=controller.admin_user)
driver.controller = controller
- return driver
+ return driver
def create_role(self, name):
roles = self.shell.keystone.roles.findall(name=name)
@@ -70,17 +70,17 @@
return 1
def create_tenant(self, tenant_name, enabled, description):
- """Create keystone tenant. Suggested fields: name, description, enabled"""
+ """Create keystone tenant. Suggested fields: name, description, enabled"""
tenants = self.shell.keystone.tenants.findall(name=tenant_name)
if not tenants:
- fields = {'tenant_name': tenant_name, 'enabled': enabled,
- 'description': description}
+ fields = {'tenant_name': tenant_name, 'enabled': enabled,
+ 'description': description}
tenant = self.shell.keystone.tenants.create(**fields)
else:
tenant = tenants[0]
- # always give the admin user the admin role to any tenant created
- # by the driver.
+ # always give the admin user the admin role to any tenant created
+ # by the driver.
self.add_user_role(self.admin_user.id, tenant.id, 'admin')
return tenant
@@ -88,17 +88,20 @@
return self.shell.keystone.tenants.update(id, **kwds)
def delete_tenant(self, id):
- ctx = self.shell.nova_db.ctx
+ # FIXME: nova_db is commented out in clients.py, throws errors.
+ # Commenting this out for the time being until actually fixed
+
+ #ctx = self.shell.nova_db.ctx
tenants = self.shell.keystone.tenants.findall(id=id)
for tenant in tenants:
# nova does not automatically delete the tenant's instances
- # so we manually delete instances before deleteing the tenant
- instances = self.shell.nova_db.instance_get_all_by_filters(ctx,
- {'project_id': tenant.id}, 'id', 'asc')
- client = OpenStackClient(tenant=tenant.name)
- driver = OpenStackDriver(client=client)
- for instance in instances:
- driver.destroy_instance(instance.id)
+ # so we manually delete instances before deleting the tenant
+ #instances = self.shell.nova_db.instance_get_all_by_filters(ctx,
+ # {'project_id': tenant.id}, 'id', 'asc')
+ #client = OpenStackClient(tenant=tenant.name)
+ #driver = OpenStackDriver(client=client)
+ #for instance in instances:
+ # driver.destroy_instance(instance.id)
self.shell.keystone.tenants.delete(tenant)
return 1
@@ -108,7 +111,7 @@
fields = {'name': name, 'email': email, 'password': password,
'enabled': enabled}
user = self.shell.keystone.users.create(**fields)
- else:
+ else:
user = users[0]
return user
@@ -129,7 +132,7 @@
if roles:
role = roles[0]
break
- return role
+ return role
def add_user_role(self, kuser_id, tenant_id, role_name):
user = self.shell.keystone.users.find(id=kuser_id)
@@ -139,8 +142,8 @@
if role_name.lower() == 'admin':
role = self.get_admin_role()
else:
- # look up non admin role or force exception when admin role isnt found
- role = self.shell.keystone.roles.find(name=role_name)
+ # look up non admin role or force exception when admin role isnt found
+ role = self.shell.keystone.roles.find(name=role_name)
role_found = False
user_roles = user.list_roles(tenant.id)
@@ -171,63 +174,63 @@
if role_found:
tenant.remove_user(user, role)
- return 1
+ return 1
def update_user(self, id, fields):
if 'password' in fields:
self.shell.keystone.users.update_password(id, fields['password'])
if 'enabled' in fields:
- self.shell.keystone.users.update_enabled(id, fields['enabled'])
- return 1
+ self.shell.keystone.users.update_enabled(id, fields['enabled'])
+ return 1
def create_router(self, name, set_gateway=True):
- routers = self.shell.quantum.list_routers(name=name)['routers']
+ routers = self.shell.neutron.list_routers(name=name)['routers']
if routers:
router = routers[0]
else:
- router = self.shell.quantum.create_router({'router': {'name': name}})['router']
+ router = self.shell.neutron.create_router({'router': {'name': name}})['router']
# add router to external network
if set_gateway:
- nets = self.shell.quantum.list_networks()['networks']
+ nets = self.shell.neutron.list_networks()['networks']
for net in nets:
- if net['router:external'] == True:
- self.shell.quantum.add_gateway_router(router['id'],
+ if net['router:external'] == True:
+ self.shell.neutron.add_gateway_router(router['id'],
{'network_id': net['id']})
-
+
return router
def delete_router(self, id):
- routers = self.shell.quantum.list_routers(id=id)['routers']
+ routers = self.shell.neutron.list_routers(id=id)['routers']
for router in routers:
- self.shell.quantum.delete_router(router['id'])
+ self.shell.neutron.delete_router(router['id'])
# remove router form external network
- #nets = self.shell.quantum.list_networks()['networks']
+ #nets = self.shell.neutron.list_networks()['networks']
#for net in nets:
# if net['router:external'] == True:
- # self.shell.quantum.remove_gateway_router(router['id'])
+ # self.shell.neutron.remove_gateway_router(router['id'])
def add_router_interface(self, router_id, subnet_id):
- router = self.shell.quantum.show_router(router_id)['router']
- subnet = self.shell.quantum.show_subnet(subnet_id)['subnet']
+ router = self.shell.neutron.show_router(router_id)['router']
+ subnet = self.shell.neutron.show_subnet(subnet_id)['subnet']
if router and subnet:
- self.shell.quantum.add_interface_router(router_id, {'subnet_id': subnet_id})
+ self.shell.neutron.add_interface_router(router_id, {'subnet_id': subnet_id})
def delete_router_interface(self, router_id, subnet_id):
- router = self.shell.quantum.show_router(router_id)
- subnet = self.shell.quantum.show_subnet(subnet_id)
+ router = self.shell.neutron.show_router(router_id)
+ subnet = self.shell.neutron.show_subnet(subnet_id)
if router and subnet:
- self.shell.quantum.remove_interface_router(router_id, {'subnet_id': subnet_id})
-
+ self.shell.neutron.remove_interface_router(router_id, {'subnet_id': subnet_id})
+
def create_network(self, name, shared=False):
- nets = self.shell.quantum.list_networks(name=name)['networks']
- if nets:
+ nets = self.shell.neutron.list_networks(name=name)['networks']
+ if nets:
net = nets[0]
else:
- net = self.shell.quantum.create_network({'network': {'name': name, 'shared': shared}})['network']
+ net = self.shell.neutron.create_network({'network': {'name': name, 'shared': shared}})['network']
return net
-
+
def delete_network(self, id):
- nets = self.shell.quantum.list_networks()['networks']
+ nets = self.shell.neutron.list_networks()['networks']
for net in nets:
if net['id'] == id:
# delete_all ports
@@ -235,18 +238,18 @@
# delete all subnets:
for subnet_id in net['subnets']:
self.delete_subnet(subnet_id)
- self.shell.quantum.delete_network(net['id'])
+ self.shell.neutron.delete_network(net['id'])
return 1
def delete_network_ports(self, network_id):
- ports = self.shell.quantum.list_ports()['ports']
+ ports = self.shell.neutron.list_ports()['ports']
for port in ports:
if port['network_id'] == network_id:
- self.shell.quantum.delete_port(port['id'])
- return 1
+ self.shell.neutron.delete_port(port['id'])
+ return 1
def delete_subnet_ports(self, subnet_id):
- ports = self.shell.quantum.list_ports()['ports']
+ ports = self.shell.neutron.list_ports()['ports']
for port in ports:
delete = False
for fixed_ip in port['fixed_ips']:
@@ -254,17 +257,17 @@
delete=True
break
if delete:
- self.shell.quantum.delete_port(port['id'])
+ self.shell.neutron.delete_port(port['id'])
return 1
-
+
def create_subnet(self, name, network_id, cidr_ip, ip_version, start, end):
- #nets = self.shell.quantum.list_networks(name=network_name)['networks']
+ #nets = self.shell.neutron.list_networks(name=network_name)['networks']
#if not nets:
- # raise Exception, "No such network: %s" % network_name
+ # raise Exception, "No such network: %s" % network_name
#net = nets[0]
- subnet = None
- subnets = self.shell.quantum.list_subnets()['subnets']
+ subnet = None
+ subnets = self.shell.neutron.list_subnets()['subnets']
for snet in subnets:
if snet['cidr'] == cidr_ip and snet['network_id'] == network_id:
subnet = snet
@@ -282,22 +285,22 @@
'host_routes': [{'destination':'169.254.169.254/32','nexthop':metadata_ip}],
'gateway_ip': None,
'allocation_pools': allocation_pools}}
- subnet = self.shell.quantum.create_subnet(subnet)['subnet']
+ subnet = self.shell.neutron.create_subnet(subnet)['subnet']
# self.add_external_route(subnet)
return subnet
def update_subnet(self, id, fields):
- return self.shell.quantum.update_subnet(id, fields)
+ return self.shell.neutron.update_subnet(id, fields)
def delete_subnet(self, id):
- #return self.shell.quantum.delete_subnet(id=id)
+ #return self.shell.neutron.delete_subnet(id=id)
# inefficient but fault tolerant
- subnets = self.shell.quantum.list_subnets()['subnets']
+ subnets = self.shell.neutron.list_subnets()['subnets']
for subnet in subnets:
if subnet['id'] == id:
self.delete_subnet_ports(subnet['id'])
- self.shell.quantum.delete_subnet(id)
+ self.shell.neutron.delete_subnet(id)
self.delete_external_route(subnet)
return 1
@@ -309,8 +312,8 @@
def add_external_route(self, subnet, routes=[]):
if not routes:
routes = self.get_external_routes()
-
- ports = self.shell.quantum.list_ports()['ports']
+
+ ports = self.shell.neutron.list_ports()['ports']
gw_ip = subnet['gateway_ip']
subnet_id = subnet['id']
@@ -325,7 +328,7 @@
if fixed_ip['subnet_id'] == subnet_id and fixed_ip['ip_address'] == gw_ip:
gw_port = port
router_id = gw_port['device_id']
- router = self.shell.quantum.show_router(router_id)['router']
+ router = self.shell.neutron.show_router(router_id)['router']
if router and router.get('external_gateway_info'):
ext_net = router['external_gateway_info']['network_id']
for port in ports:
@@ -347,7 +350,7 @@
return 1
def delete_external_route(self, subnet):
- ports = self.shell.quantum.list_ports()['ports']
+ ports = self.shell.neutron.list_ports()['ports']
gw_ip = subnet['gateway_ip']
subnet_id = subnet['id']
@@ -362,7 +365,7 @@
if fixed_ip['subnet_id'] == subnet_id and fixed_ip['ip_address'] == gw_ip:
gw_port = port
router_id = gw_port['device_id']
- router = self.shell.quantum.show_router(router_id)['router']
+ router = self.shell.neutron.show_router(router_id)['router']
ext_net = router['external_gateway_info']['network_id']
for port in ports:
if port['device_id'] == router_id and port['network_id'] == ext_net:
@@ -371,14 +374,14 @@
if ip_address:
cmd = "route delete -net %s" % (subnet['cidr'])
commands.getstatusoutput(cmd)
-
+
return 1
-
+
def create_keypair(self, name, public_key):
keys = self.shell.nova.keypairs.findall(name=name)
if keys:
key = keys[0]
- # update key
+ # update key
if key.public_key != public_key:
self.delete_keypair(key.id)
key = self.shell.nova.keypairs.create(name=name, public_key=public_key)
@@ -389,7 +392,7 @@
def delete_keypair(self, id):
keys = self.shell.nova.keypairs.findall(id=id)
for key in keys:
- self.shell.nova.keypairs.delete(key)
+ self.shell.nova.keypairs.delete(key)
return 1
def get_private_networks(self, tenant=None):
@@ -397,24 +400,24 @@
tenant = self.shell.nova.tenant
tenant = self.shell.keystone.tenants.find(name=tenant)
search_opts = {"tenant_id": tenant.id, "shared": False}
- private_networks = self.shell.quantum.list_networks(**search_opts)
+ private_networks = self.shell.neutron.list_networks(**search_opts)
return private_networks
def get_shared_networks(self):
search_opts = {"shared": True}
- shared_networks = self.shell.quantum.list_networks(**search_opts)
+ shared_networks = self.shell.neutron.list_networks(**search_opts)
return shared_networks
def get_network_subnet(self, network_id):
subnet_id = None
subnet = None
if network_id:
- os_networks = self.shell.quantum.list_networks(id=network_id)["networks"]
+ os_networks = self.shell.neutron.list_networks(id=network_id)["networks"]
if os_networks:
os_network = os_networks[0]
if os_network['subnets']:
subnet_id = os_network['subnets'][0]
- os_subnets = self.shell.quantum.list_subnets(id=subnet_id)['subnets']
+ os_subnets = self.shell.neutron.list_subnets(id=subnet_id)['subnets']
if os_subnets:
subnet = os_subnets[0]['cidr']
@@ -433,12 +436,12 @@
#if pubkeys:
# files["/root/.ssh/authorized_keys"] = "\n".join(pubkeys).encode('base64')
hints = {}
-
- # determine availability zone and compute host
+
+ # determine availability zone and compute host
availability_zone_filter = None
if availability_zone is None or not availability_zone:
availability_zone_filter = 'nova'
- else:
+ else:
availability_zone_filter = availability_zone
if hostname:
availability_zone_filter += ':%s' % hostname
diff --git a/xos/openstack/get_token.sh b/xos/openstack_xos/get_token.sh
similarity index 100%
rename from xos/openstack/get_token.sh
rename to xos/openstack_xos/get_token.sh
diff --git a/xos/openstack/instanceagent.py b/xos/openstack_xos/instanceagent.py
similarity index 96%
rename from xos/openstack/instanceagent.py
rename to xos/openstack_xos/instanceagent.py
index 8e50cf3..e3fd58d 100644
--- a/xos/openstack/instanceagent.py
+++ b/xos/openstack_xos/instanceagent.py
@@ -3,7 +3,7 @@
#os.environ.setdefault("DJANGO_SETTINGS_MODULE", "xos.settings")
import time
from core.models.instance import Instance
-from openstack.manager import OpenStackManager
+from openstack_xos.manager import OpenStackManager
class InstanceAgent:
diff --git a/xos/openstack/manager.py b/xos/openstack_xos/manager.py
similarity index 97%
rename from xos/openstack/manager.py
rename to xos/openstack_xos/manager.py
index ba163c7..193ec75 100644
--- a/xos/openstack/manager.py
+++ b/xos/openstack_xos/manager.py
@@ -12,8 +12,8 @@
from core.models import *
from xos.config import Config
try:
- from openstack.client import OpenStackClient
- from openstack.driver import OpenStackDriver
+ from openstack_xos.client import OpenStackClient
+ from openstack_xos.driver import OpenStackDriver
has_openstack = True
except:
has_openstack = False
@@ -240,7 +240,7 @@
self.driver.delete_tenant(slice.tenant_id)
# delete external route
subnet = None
- subnets = self.driver.shell.quantum.list_subnets()['subnets']
+ subnets = self.driver.shell.neutron.list_subnets()['subnets']
for snet in subnets:
if snet['id'] == slice.subnet_id:
subnet = snet
@@ -269,7 +269,7 @@
def get_next_subnet(self):
# limit ourself to 10.0.x.x for now
valid_subnet = lambda net: net.startswith('10.0')
- subnets = self.driver.shell.quantum.list_subnets()['subnets']
+ subnets = self.driver.shell.neutron.list_subnets()['subnets']
ints = [int(IPNetwork(subnet['cidr']).ip) for subnet in subnets \
if valid_subnet(subnet['cidr'])]
ints.sort()
@@ -281,13 +281,13 @@
@require_enabled
def save_subnet(self, subnet):
if not subnet.subnet_id:
- quantum_subnet = self.driver.create_subnet(name= subnet.slice.name,
+ neutron_subnet = self.driver.create_subnet(name= subnet.slice.name,
network_id=subnet.slice.network_id,
cidr_ip = subnet.cidr,
ip_version=subnet.ip_version,
start = subnet.start,
end = subnet.end)
- subnet.subnet_id = quantum_subnet['id']
+ subnet.subnet_id = neutron_subnet['id']
# add subnet as interface to slice's router
self.driver.add_router_interface(subnet.slice.router_id, subnet.subnet_id)
#add_route = 'route add -net %s dev br-ex gw 10.100.0.5' % self.cidr
@@ -473,7 +473,7 @@
def save_network_template(self, template):
if (template.shared_network_name) and (not template.shared_network_id):
- os_networks = self.driver.shell.quantum.list_networks(name=template.shared_network_name)['networks']
+ os_networks = self.driver.shell.neutron.list_networks(name=template.shared_network_name)['networks']
if os_networks:
template.shared_network_id = os_networks[0]["id"]
@@ -522,7 +522,7 @@
# Get a list of all shared networks in OS
- os_networks = self.driver.shell.quantum.list_networks()['networks']
+ os_networks = self.driver.shell.neutron.list_networks()['networks']
os_networks_by_name = {}
os_networks_by_id = {}
for os_network in os_networks:
diff --git a/xos/openstack/reservationagent.py b/xos/openstack_xos/reservationagent.py
similarity index 100%
rename from xos/openstack/reservationagent.py
rename to xos/openstack_xos/reservationagent.py
diff --git a/xos/services/hpc/admin.py b/xos/services/hpc/admin.py
index d9f28c6..19f5190 100644
--- a/xos/services/hpc/admin.py
+++ b/xos/services/hpc/admin.py
@@ -8,7 +8,6 @@
from django.contrib.auth.forms import ReadOnlyPasswordHashField
from django.contrib.auth.signals import user_logged_in
from django.utils import timezone
-from django.contrib.contenttypes import generic
from suit.widgets import LinkedSelect
from core.admin import ServiceAppAdmin,SliceInline,ServiceAttrAsTabInline, ReadOnlyAwareAdmin, XOSTabularInline, SliderWidget, ServicePrivilegeInline
from core.middleware import get_request
@@ -125,6 +124,7 @@
widgets = {
'serviceProvider' : LinkedSelect
}
+ fields = '__all__'
def __init__(self, *args, **kwargs):
request = kwargs.pop('request', None)
@@ -180,6 +180,7 @@
widgets = {
'contentProvider' : LinkedSelect
}
+ fields = '__all__'
class CDNPrefixAdmin(HPCAdmin):
form = CDNPrefixForm
diff --git a/xos/services/mcord/admin.py b/xos/services/mcord/admin.py
index ee19c7e..7e20f70 100644
--- a/xos/services/mcord/admin.py
+++ b/xos/services/mcord/admin.py
@@ -113,6 +113,7 @@
class Meta:
model = VBBUComponent
+ fields = '__all__'
# Class to represent the form to add and edit tenants.
# We need to define this instead of just using an admin like we did for the
@@ -160,13 +161,12 @@
class Meta:
model = VPGWCComponent
-
+ fields = '__all__'
# Define the admin form for the tenant. This uses a similar structure as the
# service but uses HelloWorldTenantCompleteForm to change the python behavior.
-
class VBBUComponentAdmin(ReadOnlyAwareAdmin):
verbose_name = "vBBU Component"
verbose_name_plural = "vBBU Components"
diff --git a/xos/services/requestrouter/admin.py b/xos/services/requestrouter/admin.py
index 6d9e2fe..a17bc44 100644
--- a/xos/services/requestrouter/admin.py
+++ b/xos/services/requestrouter/admin.py
@@ -8,7 +8,6 @@
from django.contrib.auth.forms import ReadOnlyPasswordHashField
from django.contrib.auth.signals import user_logged_in
from django.utils import timezone
-from django.contrib.contenttypes import generic
from suit.widgets import LinkedSelect
from core.admin import ServiceAppAdmin,SliceInline,ServiceAttrAsTabInline, ReadOnlyAwareAdmin, ServicePrivilegeInline
diff --git a/xos/services/syndicate_storage/admin.py b/xos/services/syndicate_storage/admin.py
index 779f83b..afe4de9 100644
--- a/xos/services/syndicate_storage/admin.py
+++ b/xos/services/syndicate_storage/admin.py
@@ -8,7 +8,6 @@
from django.contrib.auth.forms import ReadOnlyPasswordHashField
from django.contrib.auth.signals import user_logged_in
from django.utils import timezone
-from django.contrib.contenttypes import generic
from suit.widgets import LinkedSelect
from core.admin import ReadOnlyAwareAdmin,ServiceAppAdmin,SliceInline,ServiceAttrAsTabInline,XOSBaseAdmin, XOSTabularInline
from suit.widgets import LinkedSelect
diff --git a/xos/synchronizers/base/ansible.py b/xos/synchronizers/base/ansible.py
index b6f1ca2..56a37ea 100644
--- a/xos/synchronizers/base/ansible.py
+++ b/xos/synchronizers/base/ansible.py
@@ -204,6 +204,7 @@
f.write('pipelining = True\n')
f.write('\n[defaults]\n')
f.write('host_key_checking = False\n')
+ f.write('timeout = 30\n')
f.close()
f = open(hosts_pathname, "w")
diff --git a/xos/synchronizers/base/backend.py.bak b/xos/synchronizers/base/backend.py.bak
deleted file mode 100644
index 6e46b85..0000000
--- a/xos/synchronizers/base/backend.py.bak
+++ /dev/null
@@ -1,34 +0,0 @@
-import threading
-import time
-from observer.event_loop import XOSObserver
-from observer.event_manager import EventListener
-from util.logger import Logger, logging
-from model_policy import run_policy
-from xos.config import Config
-
-logger = Logger(level=logging.INFO)
-
-class Backend:
-
- def run(self):
- # start the openstack observer
- observer = XOSObserver()
- observer_thread = threading.Thread(target=observer.run)
- observer_thread.start()
-
- # start model policies thread
- observer_name = getattr(Config(), "observer_name", "")
- if (not observer_name):
- model_policy_thread = threading.Thread(target=run_policy)
- model_policy_thread.start()
- else:
- print "Skipping model policies thread for service observer."
-
- model_policy_thread = threading.Thread(target=run_policy)
- model_policy_thread.start()
-
- # start event listene
- event_manager = EventListener(wake_up=observer.wake_up)
- event_manager_thread = threading.Thread(target=event_manager.run)
- event_manager_thread.start()
-
diff --git a/xos/synchronizers/base/controller.diff b/xos/synchronizers/base/controller.diff
deleted file mode 100644
index 865b83e..0000000
--- a/xos/synchronizers/base/controller.diff
+++ /dev/null
@@ -1,37 +0,0 @@
-diff -up xos/model_policies/model_policy_Controller.py.orig xos/model_policies/model_policy_Controller.py
---- xos/model_policies/model_policy_Controller.py.orig 2015-01-19 20:09:13.000000000 +0000
-+++ xos/model_policies/model_policy_Controller.py 2015-04-07 21:48:51.462215000 +0000
-@@ -1,6 +1,6 @@
-
- def handle(controller):
-- from core.models import Controller, Site, ControllerSite, Slice, ControllerSlice, User, ControllerUser
-+ from core.models import Controller, Site, ControllerSite, Slice, ControllerSlice, User, ControllerUser, ControllerImages, ControllerNetwork
- from collections import defaultdict
-
- # relations for all sites
-@@ -36,3 +36,25 @@ def handle(controller):
- controller not in ctrls_by_user[user]:
- controller_user = ControllerUser(controller=controller, user=user)
- controller_user.save()
-+ # relations for all networks
-+ ctrls_by_network = defaultdict(list)
-+ ctrl_networks = ControllerNetwork.objects.all()
-+ for ctrl_network in ctrl_networks:
-+ ctrls_by_network[ctrl_network.network].append(ctrl_network.controller)
-+ networks = Network.objects.all()
-+ for network in networks:
-+ if network not in ctrls_by_network or \
-+ controller not in ctrls_by_network[network]:
-+ controller_network = ControllerNetwork(controller=controller, network=network)
-+ controller_network.save()
-+ # relations for all images
-+ ctrls_by_image = defaultdict(list)
-+ ctrl_images = ControllerImages.objects.all()
-+ for ctrl_image in ctrl_images:
-+ ctrls_by_image[ctrl_image.image].append(ctrl_image.controller)
-+ images = Image.objects.all()
-+ for image in images:
-+ if image not in ctrls_by_image or \
-+ controller not in ctrls_by_image[image]:
-+ controller_image = ControllerImages(controller=controller, image=image)
-+ controller_image.save()
diff --git a/xos/synchronizers/base/event_loop.py b/xos/synchronizers/base/event_loop.py
index ae97329..7c71d2b 100644
--- a/xos/synchronizers/base/event_loop.py
+++ b/xos/synchronizers/base/event_loop.py
@@ -18,8 +18,8 @@
from django.db.models import F, Q
from django.db import connection
from django.db import reset_queries
-#from openstack.manager import OpenStackManager
-from openstack.driver import OpenStackDriver
+#from openstack_xos.manager import OpenStackManager
+from openstack_xos.driver import OpenStackDriver
from xos.logger import Logger, logging, logger
#from timeout import timeout
from xos.config import Config, XOS_DIR
@@ -477,7 +477,7 @@
try:
# if db.connection:
# db.connection.close()
- db.close_connection()
+ db.close_old_connections()
except:
logger.log_exc("XXX we failed to fix the failure")
else:
diff --git a/xos/synchronizers/base/run_ansible b/xos/synchronizers/base/run_ansible
index a504ec3..662f798 100755
--- a/xos/synchronizers/base/run_ansible
+++ b/xos/synchronizers/base/run_ansible
@@ -1,4 +1,4 @@
#!/bin/bash
-source /opt/ansible/hacking/env-setup >> /dev/null
+export REQUESTS_CA_BUNDLE=/usr/local/share/ca-certificates/local_certs.crt
ansible-playbook -v "$@"
diff --git a/xos/synchronizers/base/run_ansible_verbose b/xos/synchronizers/base/run_ansible_verbose
index d17cad7..d72b12d 100755
--- a/xos/synchronizers/base/run_ansible_verbose
+++ b/xos/synchronizers/base/run_ansible_verbose
@@ -1,4 +1,4 @@
#!/bin/bash
-source /opt/ansible/hacking/env-setup >> /dev/null
+export REQUESTS_CA_BUNDLE=/usr/local/share/ca-certificates/local_certs.crt
ansible-playbook -vvv "$@"
diff --git a/xos/synchronizers/base/steps/sync_controller_networks.py b/xos/synchronizers/base/steps/sync_controller_networks.py
index 990cb87..f209f01 100644
--- a/xos/synchronizers/base/steps/sync_controller_networks.py
+++ b/xos/synchronizers/base/steps/sync_controller_networks.py
@@ -11,7 +11,7 @@
from core.models.instance import Instance
from xos.logger import observer_logger as logger
from synchronizers.base.ansible import *
-from openstack.driver import OpenStackDriver
+from openstack_xos.driver import OpenStackDriver
from xos.config import Config
import json
diff --git a/xos/synchronizers/base/steps/sync_controller_slices.py b/xos/synchronizers/base/steps/sync_controller_slices.py
index 97d8c29..2f36441 100644
--- a/xos/synchronizers/base/steps/sync_controller_slices.py
+++ b/xos/synchronizers/base/steps/sync_controller_slices.py
@@ -8,7 +8,7 @@
from synchronizers.base.syncstep import *
from core.models import *
from synchronizers.base.ansible import *
-from openstack.driver import OpenStackDriver
+from openstack_xos.driver import OpenStackDriver
from xos.logger import observer_logger as logger
import json
diff --git a/xos/synchronizers/base/steps/sync_controller_slices.py.bak b/xos/synchronizers/base/steps/sync_controller_slices.py.bak
deleted file mode 100644
index e04da8e..0000000
--- a/xos/synchronizers/base/steps/sync_controller_slices.py.bak
+++ /dev/null
@@ -1,95 +0,0 @@
-import os
-import base64
-from collections import defaultdict
-from netaddr import IPAddress, IPNetwork
-from django.db.models import F, Q
-from xos.config import Config
-from observer.openstacksyncstep import OpenStackSyncStep
-from core.models import *
-from observer.ansible import *
-from openstack.driver import OpenStackDriver
-from util.logger import observer_logger as logger
-import json
-
-class SyncControllerSlices(OpenStackSyncStep):
- provides=[Slice]
- requested_interval=0
- observes=ControllerSlice
-
- def fetch_pending(self, deleted):
- if (deleted):
- return ControllerSlice.deleted_objects.all()
- else:
- return ControllerSlice.objects.filter(Q(enacted__lt=F('updated')) | Q(enacted=None))
-
- def sync_record(self, controller_slice):
- logger.info("sync'ing slice controller %s" % controller_slice)
-
- controller_register = json.loads(controller_slice.controller.backend_register)
- if (controller_register.get('disabled',False)):
- raise Exception('Controller %s is disabled'%controller_slice.controller.name)
-
- if not controller_slice.controller.admin_user:
- logger.info("controller %r has no admin_user, skipping" % controller_slice.controller)
- return
-
- controller_users = ControllerUser.objects.filter(user=controller_slice.slice.creator,
- controller=controller_slice.controller)
- if not controller_users:
- raise Exception("slice createor %s has not accout at controller %s" % (controller_slice.slice.creator, controller_slice.controller.name))
- else:
- controller_user = controller_users[0]
- roles = ['Admin']
-
- max_instances=int(controller_slice.slice.max_slivers)
- tenant_fields = {'endpoint':controller_slice.controller.auth_url,
- 'admin_user': controller_slice.controller.admin_user,
- 'admin_password': controller_slice.controller.admin_password,
- 'admin_tenant': 'admin',
- 'tenant': controller_slice.slice.name,
- 'tenant_description': controller_slice.slice.description,
- 'roles':roles,
- 'name':controller_user.user.email,
- 'ansible_tag':'%s@%s'%(controller_slice.slice.name,controller_slice.controller.name),
- 'max_instances':max_instances}
-
- expected_num = len(roles)+1
- res = run_template('sync_controller_slices.yaml', tenant_fields, path='controller_slices', expected_num=expected_num)
- tenant_id = res[0]['id']
- if (not controller_slice.tenant_id):
- try:
- driver = OpenStackDriver().admin_driver(controller=controller_slice.controller)
- driver.shell.nova.quotas.update(tenant_id=controller_slice.tenant_id, instances=int(controller_slice.slice.max_slivers))
- except:
- logger.log_exc('Could not update quota for %s'%controller_slice.slice.name)
- raise Exception('Could not update quota for %s'%controller_slice.slice.name)
-
- controller_slice.tenant_id = tenant_id
- controller_slice.backend_status = '1 - OK'
- controller_slice.save()
-
-
- def delete_record(self, controller_slice):
- controller_register = json.loads(controller_slice.controller.backend_register)
- if (controller_register.get('disabled',False)):
- raise Exception('Controller %s is disabled'%controller_slice.controller.name)
-
- controller_users = ControllerUser.objects.filter(user=controller_slice.slice.creator,
- controller=controller_slice.controller)
- if not controller_users:
- raise Exception("slice createor %s has not accout at controller %s" % (controller_slice.slice.creator, controller_slice.controller.name))
- else:
- controller_user = controller_users[0]
-
- tenant_fields = {'endpoint':controller_slice.controller.auth_url,
- 'admin_user': controller_slice.controller.admin_user,
- 'admin_password': controller_slice.controller.admin_password,
- 'admin_tenant': 'admin',
- 'tenant': controller_slice.slice.name,
- 'tenant_description': controller_slice.slice.description,
- 'name':controller_user.user.email,
- 'ansible_tag':'%s@%s'%(controller_slice.slice.name,controller_slice.controller.name),
- 'delete': True}
-
- expected_num = 1
- res = run_template('sync_controller_slices.yaml', tenant_fields, path='controller_slices', expected_num=expected_num)
diff --git a/xos/synchronizers/base/steps/sync_instances.py b/xos/synchronizers/base/steps/sync_instances.py
index 335ca4a..2862474 100644
--- a/xos/synchronizers/base/steps/sync_instances.py
+++ b/xos/synchronizers/base/steps/sync_instances.py
@@ -75,7 +75,7 @@
#driver = self.driver.client_driver(caller=instance.creator, tenant=instance.slice.name, controller=instance.controllerNetwork)
driver = self.driver.admin_driver(tenant='admin', controller=instance.node.site_deployment.controller)
- nets = driver.shell.quantum.list_networks()['networks']
+ nets = driver.shell.neutron.list_networks()['networks']
for net in nets:
if net['name'] in network_templates:
nics.append(net['id'])
diff --git a/xos/synchronizers/base/steps/sync_instances.yaml b/xos/synchronizers/base/steps/sync_instances.yaml
index a61e5cf..70da32d 100644
--- a/xos/synchronizers/base/steps/sync_instances.yaml
+++ b/xos/synchronizers/base/steps/sync_instances.yaml
@@ -2,28 +2,26 @@
- hosts: 127.0.0.1
connection: local
tasks:
- - nova_compute:
- auth_url: {{ endpoint }}
- login_username: {{ admin_user }}
- login_password: {{ admin_password }}
- login_tenant_name: {{ admin_tenant }}
+ - os_server:
name: {{ name }}
+ auth:
+ auth_url: {{ endpoint }}
+ username: {{ admin_user }}
+ password: {{ admin_password }}
+ project_name: {{ project_name }}
{% if delete -%}
state: absent
{% else -%}
state: present
- availability_zone: {{ availability_zone }}
- image_name: {{ image_name }}
- wait_for: 200
- flavor_name: {{ flavor_name }}
- user_data: "{{ user_data }}"
+ availability_zone: "{{ availability_zone }}"
+ image: {{ image_name }}
+ flavor: {{ flavor_name }}
+ timeout: 200
+ userdata: "{{ user_data }}"
config_drive: yes
nics:
- {% for net in nics %}
- - net-id: {{ net }}
- {% endfor %}
- {% for port in ports %}
- - port-id: {{ port }}
+ {% for nic in nics %}
+ - {{ nic.kind }}-id: {{ nic.value }}
{% endfor %}
{% if meta %}
@@ -33,3 +31,4 @@
{% endfor %}
{% endif %}
{% endif %}
+
diff --git a/xos/synchronizers/base/steps/sync_object.py b/xos/synchronizers/base/steps/sync_object.py
index a6a939f..aaf2f25 100644
--- a/xos/synchronizers/base/steps/sync_object.py
+++ b/xos/synchronizers/base/steps/sync_object.py
@@ -7,7 +7,7 @@
from synchronizers.base.syncstep import *
from core.models import *
from synchronizers.base.ansible import *
-from openstack.driver import OpenStackDriver
+from openstack_xos.driver import OpenStackDriver
from xos.logger import observer_logger as logger
import json
diff --git a/xos/synchronizers/base/steps/sync_ports.py b/xos/synchronizers/base/steps/sync_ports.py
index 21376e5..8f2b66d 100644
--- a/xos/synchronizers/base/steps/sync_ports.py
+++ b/xos/synchronizers/base/steps/sync_ports.py
@@ -12,7 +12,7 @@
provides=[Port]
observes=Port
- # The way it works is to enumerate the all of the ports that quantum
+ # The way it works is to enumerate the all of the ports that neutron
# has, and then work backward from each port's network-id to determine
# which Network is associated from the port.
@@ -51,7 +51,7 @@
continue
try:
driver = self.driver.admin_driver(controller = controller)
- ports = driver.shell.quantum.list_ports()["ports"]
+ ports = driver.shell.neutron.list_ports()["ports"]
except:
logger.log_exc("failed to get ports from controller %s" % controller)
continue
@@ -63,7 +63,7 @@
# in the data model, so build up a list of which ids map to which network
# templates.
try:
- neutron_networks = driver.shell.quantum.list_networks()["networks"]
+ neutron_networks = driver.shell.neutron.list_networks()["networks"]
except:
print "failed to get networks from controller %s" % controller
continue
@@ -109,7 +109,7 @@
if network.template.shared_network_name:
# If it's a shared network template, then more than one network
- # object maps to the quantum network. We have to do a whole bunch
+ # object maps to the neutron network. We have to do a whole bunch
# of extra work to find the right one.
networks = network.template.network_set.all()
network = None
@@ -171,8 +171,8 @@
# A bunch of stuff to compensate for OpenStackDriver.client_driveR()
# not being in working condition.
- from openstack.client import OpenStackClient
- from openstack.driver import OpenStackDriver
+ from openstack_xos.client import OpenStackClient
+ from openstack_xos.driver import OpenStackDriver
caller = port.network.owner.creator
auth = {'username': caller.email,
'password': caller.remote_password,
@@ -180,7 +180,7 @@
client = OpenStackClient(controller=controller, **auth) # cacert=self.config.nova_ca_ssl_cert,
driver = OpenStackDriver(client=client)
- neutron_port = driver.shell.quantum.create_port({"port": {"network_id": cn.net_id}})["port"]
+ neutron_port = driver.shell.neutron.create_port({"port": {"network_id": cn.net_id}})["port"]
port.port_id = neutron_port["id"]
if neutron_port["fixed_ips"]:
port.ip = neutron_port["fixed_ips"][0]["ip_address"]
diff --git a/xos/synchronizers/base/steps/sync_slivers.py.bak b/xos/synchronizers/base/steps/sync_slivers.py.bak
deleted file mode 100644
index c4240fa..0000000
--- a/xos/synchronizers/base/steps/sync_slivers.py.bak
+++ /dev/null
@@ -1,179 +0,0 @@
-import os
-import base64
-import socket
-from django.db.models import F, Q
-from xos.config import Config
-from xos.settings import RESTAPI_HOSTNAME, RESTAPI_PORT
-from observer.openstacksyncstep import OpenStackSyncStep
-from core.models.sliver import Sliver
-from core.models.slice import Slice, SlicePrivilege, ControllerSlice
-from core.models.network import Network, NetworkSlice, ControllerNetwork
-from observer.ansible import *
-from util.logger import observer_logger as logger
-
-def escape(s):
- s = s.replace('\n',r'\n').replace('"',r'\"')
- return s
-
-class SyncSlivers(OpenStackSyncStep):
- provides=[Sliver]
- requested_interval=0
- observes=Sliver
-
- def get_userdata(self, sliver, pubkeys):
- userdata = '#cloud-config\n\nopencloud:\n slicename: "%s"\n hostname: "%s"\n restapi_hostname: "%s"\n restapi_port: "%s"\n' % (sliver.slice.name, sliver.node.name, RESTAPI_HOSTNAME, str(RESTAPI_PORT))
- userdata += 'ssh_authorized_keys:\n'
- for key in pubkeys:
- userdata += ' - %s\n' % key
- return userdata
-
- def sync_record(self, sliver):
- logger.info("sync'ing sliver:%s slice:%s controller:%s " % (sliver, sliver.slice.name, sliver.node.site_deployment.controller))
- controller_register = json.loads(sliver.node.site_deployment.controller.backend_register)
-
- if (controller_register.get('disabled',False)):
- raise Exception('Controller %s is disabled'%sliver.node.site_deployment.controller.name)
-
- metadata_update = {}
- if (sliver.numberCores):
- metadata_update["cpu_cores"] = str(sliver.numberCores)
-
- for tag in sliver.slice.tags.all():
- if tag.name.startswith("sysctl-"):
- metadata_update[tag.name] = tag.value
-
- # public keys
- slice_memberships = SlicePrivilege.objects.filter(slice=sliver.slice)
- pubkeys = set([sm.user.public_key for sm in slice_memberships if sm.user.public_key])
- if sliver.creator.public_key:
- pubkeys.add(sliver.creator.public_key)
-
- if sliver.slice.creator.public_key:
- pubkeys.add(sliver.slice.creator.public_key)
-
- if sliver.slice.service and sliver.slice.service.public_key:
- pubkeys.add(sliver.slice.service.public_key)
-
- if sliver.slice.service and sliver.slice.service.public_key:
- pubkeys.add(sliver.slice.service.public_key)
-
- nics = []
- networks = [ns.network for ns in NetworkSlice.objects.filter(slice=sliver.slice)]
- controller_networks = ControllerNetwork.objects.filter(network__in=networks,
- controller=sliver.node.site_deployment.controller)
-
- for controller_network in controller_networks:
- if controller_network.network.template.visibility == 'private' and \
- controller_network.network.template.translation == 'none':
- if not controller_network.net_id:
- raise Exception("Private Network %s has no id; Try again later" % controller_network.network.name)
- nics.append(controller_network.net_id)
-
- # now include network template
- network_templates = [network.template.shared_network_name for network in networks \
- if network.template.shared_network_name]
-
- #driver = self.driver.client_driver(caller=sliver.creator, tenant=sliver.slice.name, controller=sliver.controllerNetwork)
- driver = self.driver.admin_driver(tenant='admin', controller=sliver.node.site_deployment.controller)
- nets = driver.shell.quantum.list_networks()['networks']
- for net in nets:
- if net['name'] in network_templates:
- nics.append(net['id'])
-
- if (not nics):
- for net in nets:
- if net['name']=='public':
- nics.append(net['id'])
-
- image_id = None
- controller_images = sliver.image.controllerimages.filter(controller=sliver.node.site_deployment.controller)
- if controller_images:
- image_id = controller_images[0].glance_image_id
- logger.info("using image_id from ControllerImage object: " + str(image_id))
-
- if image_id is None:
- controller_driver = self.driver.admin_driver(controller=sliver.node.site_deployment.controller)
- image_id = None
- images = controller_driver.shell.glanceclient.images.list()
- for image in images:
- if image.name == sliver.image.name or not image_id:
- image_id = image.id
- logger.info("using image_id from glance: " + str(image_id))
-
- try:
- legacy = Config().observer_legacy
- except:
- legacy = False
-
- if (legacy):
- host_filter = sliver.node.name.split('.',1)[0]
- else:
- host_filter = sliver.node.name.strip()
-
- availability_zone_filter = 'nova:%s'%host_filter
- sliver_name = '%s-%d'%(sliver.slice.name,sliver.id)
-
- userData = self.get_userdata(sliver, pubkeys)
- if sliver.userData:
- userData = sliver.userData
-
- controller = sliver.node.site_deployment.controller
- tenant_fields = {'endpoint':controller.auth_url,
- 'admin_user': sliver.creator.email,
- 'admin_password': sliver.creator.remote_password,
- 'admin_tenant': sliver.slice.name,
- 'tenant': sliver.slice.name,
- 'tenant_description': sliver.slice.description,
- 'name':sliver_name,
- 'ansible_tag':sliver_name,
- 'availability_zone': availability_zone_filter,
- 'image_id':image_id,
- 'flavor_id':sliver.flavor.id,
- 'nics':nics,
- 'meta':metadata_update,
- 'user_data':r'%s'%escape(userData)}
-
- res = run_template('sync_slivers.yaml', tenant_fields,path='slivers', expected_num=1)
- sliver_id = res[0]['info']['OS-EXT-SRV-ATTR:instance_name']
- sliver_uuid = res[0]['id']
-
- try:
- hostname = res[0]['info']['OS-EXT-SRV-ATTR:hypervisor_hostname']
- ip = socket.gethostbyname(hostname)
- sliver.ip = ip
- except:
- pass
-
- sliver.instance_id = sliver_id
- sliver.instance_uuid = sliver_uuid
- sliver.instance_name = sliver_name
- sliver.save()
-
- def delete_record(self, sliver):
- controller_register = json.loads(sliver.node.site_deployment.controller.backend_register)
-
- if (controller_register.get('disabled',False)):
- raise Exception('Controller %s is disabled'%sliver.node.site_deployment.controller.name)
-
- sliver_name = '%s-%d'%(sliver.slice.name,sliver.id)
- controller = sliver.node.site_deployment.controller
- tenant_fields = {'endpoint':controller.auth_url,
- 'admin_user': sliver.creator.email,
- 'admin_password': sliver.creator.remote_password,
- 'admin_tenant': sliver.slice.name,
- 'tenant': sliver.slice.name,
- 'tenant_description': sliver.slice.description,
- 'name':sliver_name,
- 'ansible_tag':sliver_name,
- 'delete': True}
-
- try:
- res = run_template('sync_slivers.yaml', tenant_fields,path='slivers', expected_num=1)
- except Exception,e:
- print "Could not sync %s"%sliver_name
- #import traceback
- #traceback.print_exc()
- raise e
-
- if (len(res)!=1):
- raise Exception('Could not delete sliver %s'%sliver.slice.name)
diff --git a/xos/synchronizers/base/syncstep.py.bak b/xos/synchronizers/base/syncstep.py.bak
deleted file mode 100644
index a1f242b..0000000
--- a/xos/synchronizers/base/syncstep.py.bak
+++ /dev/null
@@ -1,203 +0,0 @@
-import os
-import base64
-from datetime import datetime
-from xos.config import Config
-from util.logger import Logger, logging
-from observer.steps import *
-from django.db.models import F, Q
-from core.models import *
-import json
-import time
-import pdb
-import traceback
-
-logger = Logger(level=logging.INFO)
-
-def f7(seq):
- seen = set()
- seen_add = seen.add
- return [ x for x in seq if not (x in seen or seen_add(x))]
-
-def elim_dups(backend_str):
- strs = backend_str.split(' // ')
- strs2 = f7(strs)
- return ' // '.join(strs2)
-
-def deepgetattr(obj, attr):
- return reduce(getattr, attr.split('.'), obj)
-
-
-class InnocuousException(Exception):
- pass
-
-class FailedDependency(Exception):
- pass
-
-class SyncStep(object):
- """ An XOS Sync step.
-
- Attributes:
- psmodel Model name the step synchronizes
- dependencies list of names of models that must be synchronized first if the current model depends on them
- """
- slow=False
- def get_prop(self, prop):
- try:
- sync_config_dir = Config().sync_config_dir
- except:
- sync_config_dir = '/etc/xos/sync'
- prop_config_path = '/'.join(sync_config_dir,self.name,prop)
- return open(prop_config_path).read().rstrip()
-
- def __init__(self, **args):
- """Initialize a sync step
- Keyword arguments:
- name -- Name of the step
- provides -- XOS models sync'd by this step
- """
- dependencies = []
- self.driver = args.get('driver')
- self.error_map = args.get('error_map')
-
- try:
- self.soft_deadline = int(self.get_prop('soft_deadline_seconds'))
- except:
- self.soft_deadline = 5 # 5 seconds
-
- return
-
- def fetch_pending(self, deletion=False):
- # This is the most common implementation of fetch_pending
- # Steps should override it if they have their own logic
- # for figuring out what objects are outstanding.
- main_obj = self.observes
- if (not deletion):
- objs = main_obj.objects.filter(Q(enacted__lt=F('updated')) | Q(enacted=None),Q(lazy_blocked=False))
- else:
- objs = main_obj.deleted_objects.all()
-
- return objs
- #return Sliver.objects.filter(ip=None)
-
- def check_dependencies(self, obj, failed):
- for dep in self.dependencies:
- peer_name = dep[0].lower() + dep[1:] # django names are camelCased with the first letter lower
-
- try:
- peer_object = deepgetattr(obj, peer_name)
- try:
- peer_objects = peer_object.all()
- except AttributeError:
- peer_objects = [peer_object]
- except:
- peer_objects = []
-
- if (hasattr(obj,'controller')):
- try:
- peer_objects = filter(lambda o:o.controller==obj.controller, peer_objects)
- except AttributeError:
- pass
-
- if (failed in peer_objects):
- if (obj.backend_status!=failed.backend_status):
- obj.backend_status = failed.backend_status
- obj.save(update_fields=['backend_status'])
- raise FailedDependency("Failed dependency for %s:%s peer %s:%s failed %s:%s" % (obj.__class__.__name__, str(getattr(obj,"pk","no_pk")), peer_object.__class__.__name__, str(getattr(peer_object,"pk","no_pk")), failed.__class__.__name__, str(getattr(failed,"pk","no_pk"))))
-
- def call(self, failed=[], deletion=False):
- pending = self.fetch_pending(deletion)
- for o in pending:
- sync_failed = False
- try:
- backoff_disabled = Config().observer_backoff_disabled
- except:
- backoff_disabled = 0
-
- try:
- scratchpad = json.loads(o.backend_register)
- if (scratchpad):
- next_run = scratchpad['next_run']
- if (not backoff_disabled and next_run>time.time()):
- sync_failed = True
- except:
- logger.log_exc("Exception while loading scratchpad")
- pass
-
- if (not sync_failed):
- try:
- for f in failed:
- self.check_dependencies(o,f) # Raises exception if failed
- if (deletion):
- self.delete_record(o)
- o.delete(purge=True)
- else:
- self.sync_record(o)
- o.enacted = datetime.now() # Is this the same timezone? XXX
- scratchpad = {'next_run':0, 'exponent':0}
- o.backend_register = json.dumps(scratchpad)
- o.backend_status = "1 - OK"
- o.save(update_fields=['enacted','backend_status','backend_register'])
- except (InnocuousException,Exception) as e:
- logger.log_exc("sync step failed!")
- try:
- if (o.backend_status.startswith('2 - ')):
- str_e = '%s // %r'%(o.backend_status[4:],e)
- str_e = elim_dups(str_e)
- else:
- str_e = '%r'%e
- except:
- str_e = '%r'%e
-
- try:
- error = self.error_map.map(str_e)
- except:
- error = '2 - %s'%str_e
-
- if isinstance(e, InnocuousException) and not force_error:
- o.backend_status = '1 - %s'%error
- else:
- o.backend_status = '3 - %s'%error
-
- try:
- scratchpad = json.loads(o.backend_register)
- scratchpad['exponent']
- except:
- logger.log_exc("Exception while updating scratchpad")
- scratchpad = {'next_run':0, 'exponent':0}
-
- # Second failure
- if (scratchpad['exponent']):
- delay = scratchpad['exponent'] * 600 # 10 minutes
- if (delay<1440):
- delay = 1440
- scratchpad['next_run'] = time.time() + delay
-
- scratchpad['exponent']+=1
-
- o.backend_register = json.dumps(scratchpad)
-
- # TOFIX:
- # DatabaseError: value too long for type character varying(140)
- if (o.pk):
- try:
- o.backend_status = o.backend_status[:1024]
- o.save(update_fields=['backend_status','backend_register','updated'])
- except:
- print "Could not update backend status field!"
- pass
- sync_failed = True
-
-
- if (sync_failed):
- failed.append(o)
-
- return failed
-
- def sync_record(self, o):
- return
-
- def delete_record(self, o):
- return
-
- def __call__(self, **args):
- return self.call(**args)
diff --git a/xos/synchronizers/model_policy.py b/xos/synchronizers/model_policy.py
index aa12092..2ad6966 100644
--- a/xos/synchronizers/model_policy.py
+++ b/xos/synchronizers/model_policy.py
@@ -1,17 +1,18 @@
+from core.models import *
+from datetime import datetime
+from django.db import reset_queries
+from django.db.models import F, Q
from django.db.models.signals import post_save
+from django.db.transaction import atomic
from django.dispatch import receiver
-import pdb
+from django.utils import timezone
from generate.dependency_walker import *
from synchronizers.openstack import model_policies
from xos.logger import Logger, logging
-from datetime import datetime
-from django.utils import timezone
+
+import pdb
import time
import traceback
-from core.models import *
-from django.db import reset_queries
-from django.db.transaction import atomic
-from django.db.models import F, Q
modelPolicyEnabled = True
bad_instances=[]
@@ -75,8 +76,6 @@
elif (sender_name in delete_policy_models):
walk_inv_deps(delete_if_inactive, instance)
-
-
try:
policy_handler = getattr(model_policies, policy_name, None)
logger.info("MODEL POLICY: handler %s %s" % (policy_name, policy_handler))
@@ -114,7 +113,7 @@
try:
# if db.connection:
# db.connection.close()
- db.close_connection()
+ db.close_old_connections()
except:
logger.log_exc("XXX we failed to fix the failure")
else:
diff --git a/xos/synchronizers/openstack/steps/sync_controller_networks.py b/xos/synchronizers/openstack/steps/sync_controller_networks.py
index cf855c6..b61ef7b 100644
--- a/xos/synchronizers/openstack/steps/sync_controller_networks.py
+++ b/xos/synchronizers/openstack/steps/sync_controller_networks.py
@@ -13,7 +13,7 @@
from core.models.instance import Instance
from xos.logger import observer_logger as logger
from synchronizers.base.ansible import *
-from openstack.driver import OpenStackDriver
+from openstack_xos.driver import OpenStackDriver
from xos.config import Config
import json
@@ -77,8 +77,8 @@
network_fields = {'endpoint':controller_network.controller.auth_url,
'endpoint_v3': controller_network.controller.auth_url_v3,
'admin_user':slice.creator.email,
- 'tenant_name':slice.name,
'admin_password':slice.creator.remote_password,
+ 'admin_project':slice.name,
'domain': controller_network.controller.domain,
'name':network_name,
'subnet_name':subnet_name,
@@ -93,8 +93,8 @@
return network_fields
def map_sync_outputs(self, controller_network,res):
- network_id = res[0]['id']
- subnet_id = res[1]['id']
+ network_id = res[0]['network']['id']
+ subnet_id = res[1]['subnet']['id']
controller_network.net_id = network_id
controller_network.subnet = self.cidr
controller_network.subnet_id = subnet_id
diff --git a/xos/synchronizers/openstack/steps/sync_controller_networks.yaml b/xos/synchronizers/openstack/steps/sync_controller_networks.yaml
index 6fb9a92..7b6075c 100644
--- a/xos/synchronizers/openstack/steps/sync_controller_networks.yaml
+++ b/xos/synchronizers/openstack/steps/sync_controller_networks.yaml
@@ -2,44 +2,57 @@
- hosts: 127.0.0.1
connection: local
tasks:
- - quantum_network:
- auth_url={{ endpoint }}
- login_username={{ admin_user }}
- login_tenant_name={{ tenant_name }}
- login_password={{ admin_password }}
- tenant_name={{ tenant_name }}
- name={{ name }}
- {% if delete %}
- state=absent
- {% else %}
- state=present
- {% endif %}
- shared=true
- {% if not delete %}
- - quantum_subnet:
- auth_url={{ endpoint }}
- login_username={{ admin_user }}
- login_tenant_name={{ tenant_name }}
- login_password={{ admin_password }}
- tenant_name={{ tenant_name }}
- name={{ subnet_name }}
- network_name={{ name }}
- {% if delete %}
- state=absent
- {% else %}
- state=present
- {% if use_vtn %}
- gateway_ip={{ gateway }}
- {% else %}
- no_gateway=true
- {% endif %}
- dns_nameservers=8.8.8.8
- cidr={{ cidr }}
- {% if start_ip %}
- allocation_pool_start={{ start_ip }}
- {% endif %}
- {% if end_ip %}
- allocation_pool_end={{ end_ip }}
- {% endif %}
- {% endif %}
- {% endif %}
+ - os_network:
+ name: {{ name }}
+ shared: true
+ {% if not delete -%}
+ state: present
+ {% else -%}
+ state: absent
+ {% endif -%}
+ auth:
+ auth_url: {{ endpoint }}
+ username: {{ admin_user }}
+ password: {{ admin_password }}
+ project_name: {{ admin_project }}
+
+{% if not delete %}
+ - os_subnet:
+ name: {{ subnet_name }}
+ network_name: {{ name }}
+ {% if not delete -%}
+ state: present
+ cidr: {{ cidr }}
+ dns_nameservers: 8.8.8.8
+ {% if use_vtn -%}
+ gateway_ip: {{ gateway }}
+ {% endif -%}
+ {% if start_ip -%}
+ allocation_pool_start: {{ start_ip }}
+ {% endif -%}
+ {% if end_ip -%}
+ allocation_pool_end: {{ end_ip }}
+ {% endif -%}
+ {% else -%}
+ state: absent
+ {% endif -%}
+ auth:
+ auth_url: {{ endpoint }}
+ username: {{ admin_user }}
+ password: {{ admin_password }}
+ project_name: {{ admin_project }}
+
+{% if not use_vtn -%}
+# until we get 'no-gateway-ip' arg to os_subnet, in Ansible 2.2
+# https://github.com/ansible/ansible-modules-core/pull/3736
+ - command:
+ neutron \
+ --os-auth-url {{ endpoint }} \
+ --os-username {{ admin_user }} \
+ --os-password {{ admin_password }} \
+ --os-tenant-name {{ admin_project }} \
+ subnet-update --no-gateway {{ subnet_name }}
+{% endif -%}
+
+{% endif %}
+
diff --git a/xos/synchronizers/openstack/steps/sync_controller_slices.py b/xos/synchronizers/openstack/steps/sync_controller_slices.py
index 97d8c29..0666230 100644
--- a/xos/synchronizers/openstack/steps/sync_controller_slices.py
+++ b/xos/synchronizers/openstack/steps/sync_controller_slices.py
@@ -8,7 +8,7 @@
from synchronizers.base.syncstep import *
from core.models import *
from synchronizers.base.ansible import *
-from openstack.driver import OpenStackDriver
+from openstack_xos.driver import OpenStackDriver
from xos.logger import observer_logger as logger
import json
@@ -31,7 +31,8 @@
raise Exception("slice createor %s has not accout at controller %s" % (controller_slice.slice.creator, controller_slice.controller.name))
else:
controller_user = controller_users[0]
- roles = ['admin']
+ driver = OpenStackDriver().admin_driver(controller=controller_slice.controller)
+ roles = [driver.get_admin_role().name]
max_instances=int(controller_slice.slice.max_instances)
tenant_fields = {'endpoint':controller_slice.controller.auth_url,
diff --git a/xos/synchronizers/openstack/steps/sync_controller_users.py b/xos/synchronizers/openstack/steps/sync_controller_users.py
index 9af48e5..c9de142 100644
--- a/xos/synchronizers/openstack/steps/sync_controller_users.py
+++ b/xos/synchronizers/openstack/steps/sync_controller_users.py
@@ -9,6 +9,7 @@
from core.models.user import User
from core.models.controlleruser import ControllerUser
from synchronizers.base.ansible import *
+from openstack_xos.driver import OpenStackDriver
from xos.logger import observer_logger as logger
import json
@@ -25,9 +26,11 @@
# All users will have at least the 'user' role at their home site/tenant.
# We must also check if the user should have the admin role
+
roles = ['user']
if controller_user.user.is_admin:
- roles.append('admin')
+ driver = OpenStackDriver().admin_driver(controller=controller_user.controller)
+ roles.append(driver.get_admin_role().name)
# setup user home site roles at controller
if not controller_user.user.site:
@@ -52,14 +55,14 @@
'admin_user': controller_user.controller.admin_user,
'admin_password': controller_user.controller.admin_password,
'ansible_tag':'%s@%s'%(controller_user.user.email.replace('@','-at-'),controller_user.controller.name),
- 'admin_tenant': controller_user.controller.admin_tenant,
+ 'admin_project': controller_user.controller.admin_tenant,
'roles':roles,
- 'tenant':controller_user.user.site.login_base
+ 'project':controller_user.user.site.login_base
}
return user_fields
def map_sync_outputs(self, controller_user, res):
- controller_user.kuser_id = res[0]['id']
+ controller_user.kuser_id = res[0]['user']['id']
controller_user.backend_status = '1 - OK'
controller_user.save()
diff --git a/xos/synchronizers/openstack/steps/sync_controller_users.yaml b/xos/synchronizers/openstack/steps/sync_controller_users.yaml
index 4f2db5e..5cb3cc9 100644
--- a/xos/synchronizers/openstack/steps/sync_controller_users.yaml
+++ b/xos/synchronizers/openstack/steps/sync_controller_users.yaml
@@ -2,15 +2,49 @@
- hosts: 127.0.0.1
connection: local
tasks:
- - keystone_user:
- endpoint={{ endpoint }}
- login_user={{ admin_user }}
- login_password={{ admin_password }}
- login_tenant_name={{ admin_tenant }}
- user="{{ name }}"
- email={{ email }}
- password={{ password }}
- tenant={{ tenant }}
- {% for role in roles %}
- - keystone_user: endpoint={{ endpoint}} login_user={{ admin_user }} login_password={{ admin_password }} login_tenant_name={{ admin_tenant }} user="{{ name }}" role={{ role }} tenant={{ tenant }}
- {% endfor %}
+
+ - name: Create user account for "{{ name }}"
+ os_user:
+ name: "{{ name }}"
+ email: "{{ email }}"
+ password: "{{ password }}"
+ auth:
+ auth_url: {{ endpoint }}
+ username: {{ admin_user }}
+ password: {{ admin_password }}
+ project_name: {{ admin_project }}
+
+ - name: Create project for "{{ project }}"
+ os_project:
+ name: "{{ project }}"
+ auth:
+ auth_url: {{ endpoint }}
+ username: {{ admin_user }}
+ password: {{ admin_password }}
+ project_name: {{ admin_project }}
+
+{% for role in roles %}
+ - name: Creating role "{{ role }}" for "{{ name }}" on "{{ project }}"
+ keystone_user:
+ user: "{{ name }}"
+ role: "{{ role }}"
+ tenant: "{{ project }}"
+ endpoint: {{ endpoint }}
+ login_user: {{ admin_user }}
+ login_password: {{ admin_password }}
+ login_tenant_name: {{ admin_project }}
+{% endfor %}
+
+# FIXME: the below should work in Ansible 2.1, once we get the Admin/admin and
+# Member/user role name issues straightened out.
+#
+# - name: Creating role "{{ role }}" for "{{ name }}" on "{{ project }}"
+# os_user_role:
+# user: "{{ name }}"
+# role: "{{ role }}"
+# project: "{{ project }}"
+# auth:
+# auth_url: {{ endpoint }}
+# username: {{ admin_user }}
+# password: {{ admin_password }}
+# project_name: {{ admin_project }}
diff --git a/xos/synchronizers/openstack/steps/sync_instances.py b/xos/synchronizers/openstack/steps/sync_instances.py
index faeb860..5cec50d 100644
--- a/xos/synchronizers/openstack/steps/sync_instances.py
+++ b/xos/synchronizers/openstack/steps/sync_instances.py
@@ -119,7 +119,7 @@
#driver = self.driver.client_driver(caller=instance.creator, tenant=instance.slice.name, controller=instance.controllerNetwork)
driver = self.driver.admin_driver(tenant='admin', controller=instance.node.site_deployment.controller)
- nets = driver.shell.quantum.list_networks()['networks']
+ nets = driver.shell.neutron.list_networks()['networks']
for net in nets:
if net['name'] in network_templates:
nics.append({"kind": "net", "value": net['id'], "network": None})
@@ -169,7 +169,7 @@
'domain': controller.domain,
'admin_user': instance.creator.email,
'admin_password': instance.creator.remote_password,
- 'admin_tenant': instance.slice.name,
+ 'project_name': instance.slice.name,
'tenant': instance.slice.name,
'tenant_description': instance.slice.description,
'name':instance_name,
@@ -184,11 +184,11 @@
def map_sync_outputs(self, instance, res):
- instance_id = res[0]['info']['OS-EXT-SRV-ATTR:instance_name']
+ instance_id = res[0]['openstack']['OS-EXT-SRV-ATTR:instance_name']
instance_uuid = res[0]['id']
try:
- hostname = res[0]['info']['OS-EXT-SRV-ATTR:hypervisor_hostname']
+ hostname = res[0]['openstack']['OS-EXT-SRV-ATTR:hypervisor_hostname']
ip = socket.gethostbyname(hostname)
instance.ip = ip
except:
diff --git a/xos/synchronizers/openstack/steps/sync_instances.yaml b/xos/synchronizers/openstack/steps/sync_instances.yaml
index 3e7182a..476890f 100644
--- a/xos/synchronizers/openstack/steps/sync_instances.yaml
+++ b/xos/synchronizers/openstack/steps/sync_instances.yaml
@@ -2,22 +2,24 @@
- hosts: 127.0.0.1
connection: local
tasks:
- - nova_compute:
- auth_url: {{ endpoint }}
- login_username: {{ admin_user }}
- login_password: {{ admin_password }}
- login_tenant_name: {{ admin_tenant }}
+ - os_server:
name: {{ name }}
+ auth:
+ auth_url: {{ endpoint }}
+ username: {{ admin_user }}
+ password: {{ admin_password }}
+ project_name: {{ project_name }}
{% if delete -%}
state: absent
{% else -%}
state: present
- availability_zone: {{ availability_zone }}
- image_name: {{ image_name }}
- wait_for: 200
- flavor_name: {{ flavor_name }}
- user_data: "{{ user_data }}"
+ availability_zone: "{{ availability_zone }}"
+ image: {{ image_name }}
+ flavor: {{ flavor_name }}
+ timeout: 200
+ userdata: "{{ user_data }}"
config_drive: yes
+ auto_ip: no
nics:
{% for nic in nics %}
- {{ nic.kind }}-id: {{ nic.value }}
@@ -30,3 +32,4 @@
{% endfor %}
{% endif %}
{% endif %}
+
diff --git a/xos/synchronizers/openstack/steps/sync_object.py b/xos/synchronizers/openstack/steps/sync_object.py
index a6a939f..aaf2f25 100644
--- a/xos/synchronizers/openstack/steps/sync_object.py
+++ b/xos/synchronizers/openstack/steps/sync_object.py
@@ -7,7 +7,7 @@
from synchronizers.base.syncstep import *
from core.models import *
from synchronizers.base.ansible import *
-from openstack.driver import OpenStackDriver
+from openstack_xos.driver import OpenStackDriver
from xos.logger import observer_logger as logger
import json
diff --git a/xos/synchronizers/openstack/steps/sync_ports.py b/xos/synchronizers/openstack/steps/sync_ports.py
index 4f6ce14..5e0ff04 100644
--- a/xos/synchronizers/openstack/steps/sync_ports.py
+++ b/xos/synchronizers/openstack/steps/sync_ports.py
@@ -12,7 +12,7 @@
provides=[Port]
observes=Port
- # The way it works is to enumerate the all of the ports that quantum
+ # The way it works is to enumerate the all of the ports that neutron
# has, and then work backward from each port's network-id to determine
# which Network is associated from the port.
@@ -28,17 +28,17 @@
# ports to instances if the port's tenant does not match
# the instance's tenant.
- # A bunch of stuff to compensate for OpenStackDriver.client_driveR()
+ # A bunch of stuff to compensate for OpenStackDriver.client_driver()
# not being in working condition.
- from openstack.client import OpenStackClient
- from openstack.driver import OpenStackDriver
+ from openstack_xos.client import OpenStackClient
+ from openstack_xos.driver import OpenStackDriver
controller = port.instance.node.site_deployment.controller
slice = port.instance.slice
caller = port.network.owner.creator
auth = {'username': caller.email,
'password': caller.remote_password,
'tenant': slice.name}
- client = OpenStackClient(controller=controller, **auth) # cacert=self.config.nova_ca_ssl_cert,
+ client = OpenStackClient(controller=controller, **auth)
driver = OpenStackDriver(client=client)
return driver
@@ -78,7 +78,7 @@
continue
try:
driver = self.driver.admin_driver(controller = controller)
- ports = driver.shell.quantum.list_ports()["ports"]
+ ports = driver.shell.neutron.list_ports()["ports"]
except:
logger.log_exc("failed to get ports from controller %s" % controller)
continue
@@ -90,7 +90,7 @@
# in the data model, so build up a list of which ids map to which network
# templates.
try:
- neutron_networks = driver.shell.quantum.list_networks()["networks"]
+ neutron_networks = driver.shell.neutron.list_networks()["networks"]
except:
print "failed to get networks from controller %s" % controller
continue
@@ -136,7 +136,7 @@
if network.template.shared_network_name:
# If it's a shared network template, then more than one network
- # object maps to the quantum network. We have to do a whole bunch
+ # object maps to the neutron network. We have to do a whole bunch
# of extra work to find the right one.
networks = network.template.network_set.all()
network = None
@@ -198,7 +198,7 @@
if neutron_port_name:
args["name"] = neutron_port_name
- neutron_port = driver.shell.quantum.create_port({"port": args})["port"]
+ neutron_port = driver.shell.neutron.create_port({"port": args})["port"]
port.port_id = neutron_port["id"]
if neutron_port["fixed_ips"]:
port.ip = neutron_port["fixed_ips"][0]["ip_address"]
@@ -220,7 +220,7 @@
logger.info("calling openstack to destroy port %s" % port.port_id)
try:
driver = self.get_driver(port)
- driver.shell.quantum.delete_port(port.port_id)
+ driver.shell.neutron.delete_port(port.port_id)
except:
logger.log_exc("failed to delete port %s from neutron" % port.port_id)
return
diff --git a/xos/templates/admin/base.html b/xos/templates/admin/base.html
index 4fd5f44..2ae846c 100644
--- a/xos/templates/admin/base.html
+++ b/xos/templates/admin/base.html
@@ -25,9 +25,9 @@
<link rel="stylesheet" type="text/css" href="{% block stylesheet_rtl %}{% static "admin/css/rtl.css" %}{% endblock %}"/>
{% endif %}
<script type="text/javascript">window.__admin_media_prefix__ = "{% filter escapejs %}{% static "admin/" %}{% endfilter %}";</script>
- <script src="{% static 'suit/js/jquery-1.9.1.min.js' %}"></script>
- <script src="http://code.jquery.com/ui/1.10.4/jquery-ui.js"></script>
- <link rel="stylesheet" href="http://code.jquery.com/ui/1.10.4/themes/smoothness/jquery-ui.css">
+ <script src="{% static 'suit/js/jquery.min.js' %}"></script>
+ <script src="{% static 'suit/js/jquery-ui.min.js' %}"></script>
+ <link rel="stylesheet" href="{% static 'suit/css/jquery-ui.min.css' %}">
<script type="text/javascript" src="{% static 'log4javascript-1.4.6/log4javascript.js' %}"></script>
<script type="text/javascript" src="{% static 'uploadTextarea.js' %}"></script>
diff --git a/xos/templates/admin/dashboard/dashboard_base.html b/xos/templates/admin/dashboard/dashboard_base.html
index 5d510b8..d8f5a22 100644
--- a/xos/templates/admin/dashboard/dashboard_base.html
+++ b/xos/templates/admin/dashboard/dashboard_base.html
@@ -9,15 +9,10 @@
{% if XOS_BRANDING_CSS %}
<link rel="stylesheet" type="text/css" href="{{ XOS_BRANDING_CSS }}">
{% endif %}
-<link rel="stylesheet" href="http://code.jquery.com/ui/1.10.4/themes/smoothness/jquery-ui.css">
<link rel="stylesheet" href="http://cdn.leafletjs.com/leaflet-0.7.2/leaflet.css" />
<script src="http://cdn.leafletjs.com/leaflet-0.7.2/leaflet.js"></script>
-
-
-<!-- no need to include jquery here as it's already included by base.html. Including it multiple times will break mtuity statistics. -->
-<!-- src="http://code.jquery.com/jquery-1.9.1.js" -->
-
-<script src="http://code.jquery.com/ui/1.10.4/jquery-ui.js"></script>
+<script src="{% static 'suit/js/jquery-ui.min.js' %}"></script>
+<link rel="stylesheet" href="{% static 'suit/css/jquery-ui.min.css' %}">
<script src="http://ajax.aspnetcdn.com/ajax/jquery.dataTables/1.9.4/jquery.dataTables.min.js"></script>
<script type="text/javascript" src="{% static 'log4javascript-1.4.6/log4javascript.js' %}"></script>
<script src="{% static 'js/Leaflet.MakiMarkers.js' %}" > </script>
diff --git a/xos/templates/admin/login.html b/xos/templates/admin/login.html
index 386d579..0f08665 100644
--- a/xos/templates/admin/login.html
+++ b/xos/templates/admin/login.html
@@ -6,8 +6,8 @@
{% if XOS_BRANDING_CSS %}
<link rel="stylesheet" type="text/css" href="{{ XOS_BRANDING_CSS }}">
{% endif %}
-<script src="{% static 'suit/js/jquery-1.9.1.min.js' %}"></script>
-<script src="http://code.jquery.com/ui/1.11.0/jquery-ui.js"></script>
+<script src="{% static 'suit/js/jquery.min.js' %}"></script>
+<script src="{% static 'suit/js/jquery-ui.min.js' %}"></script>
{% endblock %}
diff --git a/xos/templates/admin/wholePage.html b/xos/templates/admin/wholePage.html
index 156c8aa..4508e4e 100644
--- a/xos/templates/admin/wholePage.html
+++ b/xos/templates/admin/wholePage.html
@@ -5,8 +5,8 @@
<link rel="stylesheet" type="text/css" href="{% static 'suit/bootstrap/css/bootstrap.min.css' %}" media="all"/>
<link rel="stylesheet" type="text/css" href="{% static 'suit/css/suit.css' %}" media="all">
<link rel="stylesheet" type="text/css" href="{% static 'xos.css' %}" media="all">
-<script src="{% static 'suit/js/jquery-1.9.1.min.js' %}"></script>
-<script src="http://code.jquery.com/ui/1.10.4/jquery-ui.js"></script>
+<script src="{% static 'suit/js/jquery.min.js' %}"></script>
+<script src="{% static 'suit/js/jquery-ui.min.js' %}"></script>
{% block extrahead %}{% endblock %}
</head>
<body>{% block content %}{% endblock %}</body>
diff --git a/xos/tools/xos-manage b/xos/tools/xos-manage
index 706aeda..e10a47e 100755
--- a/xos/tools/xos-manage
+++ b/xos/tools/xos-manage
@@ -128,7 +128,7 @@
mkdir -p private_keys
echo "Generating keys"
keyczart create --location=private_keys --name="OpenCloud" --purpose=crypt --asymmetric=rsa
- keyczart addkey --location=private_keys --status=primary --size=1024
+ keyczart addkey --location=private_keys --status=primary --size=2048
keyczart pubkey --location=private_keys --destination=public_keys
if [[ ! -f public_keys/1 ]]; then
echo "FAILED to create keys"