creating a stand-alone XOS book
Change-Id: I61c6708626655c99fda7cf7ba71d370bd9a8ad65
diff --git a/docs/Makefile b/docs/Makefile
index 08c6179..7468139 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -1,28 +1,52 @@
-DOCSDIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
+# Makefile for building CORD docs site, guide.opencord.org
+# Building docs requires the following tools:
+# - Gitbook toolchain: https://toolchain.gitbook.com/setup.html
+# - NPM (for Gitbook and Swagger)
+# - Python (for build glossary script)
+# - linkchecker (for test target) http://wummel.github.io/linkchecker/
-swagger: swagger_clean swagger_docs swagger_serve_spec
+default: serve
-venv-xosdocs:
- ./venv-xosdocs.sh
+# use bash for pushd/popd, and to fail if commands within a pipe fail
+SHELL = bash -o pipefail
-swagger_docs: venv-xosdocs
- bash -c 'source $(DOCSDIR)venv-xosdocs/bin/activate && python $(DOCSDIR)scripts/swagger_docs.py'
+GENERATED_DOCS =
-clean: swagger_clean venv_clean
+LINT_STYLE ?= mdl_relaxed.rb
-swagger_clean:
- rm -rf swagger/specs/*
+serve: setup
+ gitbook serve
-venv_clean:
- rm -rf $(DOCSDIR)venv-xosdocs
+build: setup
+ gitbook build
-swagger_serve_spec:
- http-server --cors ./swagger
+setup: xos-gui xos-tosca
+ gitbook init
+ gitbook install
-# HOW TO
-# - load the virtual env (https://guide.opencord.org/xos/dev/local_env.html)
-# - run `make`
-# - open the browser at `http://127.0.0.1:8080/ui`
-# - you'll see the 'core' apis
-# - you can see a list of available apis at `http://127.0.0.1:8080/specs/`
-# - just replace the swagger file (top bar) with another one to see different APIs (to be improved)
+test: linkcheck lint
+
+linkcheck: build
+ linkchecker -a _book/
+
+lint:
+ @echo "markdownlint(mdl) version: `mdl --version`"
+ @echo "style config:"
+ @echo "---"
+ @cat $(LINT_STYLE)
+ @echo "---"
+ mdl -s $(LINT_STYLE) `find -L . ! -path "./_book/*" ! -path "./node_modules/*" -name "*.md"`
+
+# link directories that contain other documentation
+xos-gui:
+ ln -s ../../xos-gui/docs xos-gui
+
+xos-tosca:
+ ln -s ../../xos-tosca/docs xos-tosca
+
+
+clean:
+ rm -rf $(GENERATED_DOCS)
+ rm -rf _book
+ rm -rf node_modules
+ rm -rf xos xos-gui xos-tosca
diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
index 0fbdc90..590be00 100644
--- a/docs/SUMMARY.md
+++ b/docs/SUMMARY.md
@@ -1,12 +1,15 @@
# Summary
* [Introduction](README.md)
- * [Contribute to the documentation](contribute_docs.md)
-* Modules
- * [XOS Config](modules/xosconfig.md)
-* Developer How Tos
- * [Workflow: Mock Config](dev/workflow_mock.md)
- * [Workflow: CORD-in-a-Box](dev/workflow_ciab.md)
- * [Local Dev Environmet](dev/local_env.md)
- * [xproto](dev/xproto.md)
+* [Overview](overview.md)
+* [Installing XOS](install.md)
+* [Using XOS](use.md)
+ * [XOS Modeling Framework](xos/dev/xproto.md)
+ * [XOS Tool Chain (Internals)](xos/dev/xosgenx.md)
+ * [XOS Synchronizer Framework](xos/dev/synchronizers.md)
+ * [Synchronizer Design](xos/dev/sync_arch.md)
+ * [Synchronizer Implementation](xos/dev/sync_impl.md)
+ * [Synchronizer Reference](xos/dev/sync_reference.md)
+ * [Core Models](xos/core_models.md)
+ * [Security Policies](xos/security_policies.md)
diff --git a/docs/book.json b/docs/book.json
new file mode 100644
index 0000000..cf106cf
--- /dev/null
+++ b/docs/book.json
@@ -0,0 +1,16 @@
+{
+ "title": "XOS Guide",
+ "root": ".",
+ "structure": {
+ "summary": "SUMMARY.md"
+ },
+ "variables": {
+ "branch": "master"
+ },
+ "plugins": [
+ "anchorjs",
+ "smart-nav-collapse",
+ "page-toc",
+ "versions-select"
+ ]
+}
diff --git a/docs/install.md b/docs/install.md
new file mode 100644
index 0000000..88f125a
--- /dev/null
+++ b/docs/install.md
@@ -0,0 +1,176 @@
+# Installing XOS
+
+This section described how to deploy XOS. It is a minimal deployment
+that includes the set of micro-services that make up XOS itself, plus
+optionally, an example service (a web server running in a Kubernetes pod).
+For an example of how XOS is integrated into a larger system, see the
+[CORD Installation Guide](https://guide.opencord.org/).
+
+XOS runs on any version of Kubernetes (1.10 or greater), and uses the
+Helm client-side tool. If you are new to Kubernetes, we recommend this [tutorial](https://kubernetes.io/docs/tutorials/) as a good place to start.
+
+Although you are free to set up Kubernetes and Helm in whatever way
+makes sense for your system, the following walks you through an
+example installation sequence on MacOS. It was tested on version 10.12.6.
+
+## Prerequisites
+
+You need to install Docker. Visit `https://docs.docker.com/docker-for-mac/install/` for instructions.
+
+You also need to install VirtualBox. Visit `https://www.virtualbox.org/wiki/Downloads` for instructions.
+
+The following assumes you've installed the Homebrew package manager. Visit
+`https://brew.sh/` for instructions.
+
+## Install Minikube and Kubectl
+
+To install Minikube, run the following command:
+
+```shell
+curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.28.0/minikube-darwin-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
+```
+To install Kubectl, run the following command:
+
+```shell
+brew install kubectl
+```
+
+## Install Helm and Tiller
+
+The following installs both Helm and Tiller.
+
+```shell
+brew install kubernetes-helm
+```
+
+## Bring Up a Kubernetes Cluster
+
+Start a minikube cluster as follows. This automatically runs inside VirtualBox.
+
+```shell
+minikube start
+```
+
+To see that it's running, type
+
+```shell
+kubectl cluster-info
+```
+
+You should see something like the following
+
+```shell
+Kubernetes master is running at https://192.168.99.100:8443
+KubeDNS is running at https://192.168.99.100:8443/api/v1/namespaces/kube-system/services/kube-dns:dn s/proxy
+
+To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
+```
+
+You can also see how the cluster is configured by looking at `~/.kube/config`.
+Other tools described on this page use this configuration file to find your cluster.
+
+If you want, you can see minikube running by looking at the VirtualBox dashboard.
+Or alternatively, you can visit the Minikube dashboard:
+
+```shell
+minikube dashboard
+```
+
+As a final setp, you need to start Tiller on the Kubernetes cluster.
+
+```shell
+helm init
+```
+
+## Download XOS Helm-Charts
+
+The helm charts used to deploy XOS are currently bundled in the CORD
+`helm-chart` repository. The rest of this section assumes all you download
+this repository into a directory named `~/cord`.
+
+```shell
+mkdir ~/cord
+cd ~/cord
+git clone https://gerrit.opencord.org/helm-charts
+cd helm-charts
+```
+
+## Bring Up XOS
+
+To deploy `xos-core` (plus affiliated micro-services) into your
+Kubernetes cluster, execute the following from the `~/cord/helm-charts`
+directory:
+
+```shell
+helm dep update xos-core
+helm install xos-core -n xos-core
+```
+
+Use `kubectl get pods` to verify that all containers that implement XOS
+are successfully running. You should see output that looks something
+like this:
+
+```shell
+NAME READY STATUS RESTARTS AGE
+xos-chameleon-6f49b67f68-pdf6n 1/1 Running 0 2m
+xos-core-57fd788db-8b97d 1/1 Running 0 2m
+xos-db-f9ddc6589-rtrml 1/1 Running 0 2m
+xos-gui-7fcfcd4474-prhfb 1/1 Running 0 2m
+xos-redis-74c5cdc969-ppd7z 1/1 Running 0 2m
+xos-tosca-7c665f97b6-krp5k 1/1 Running 0 2m
+xos-ws-55d676c696-pxsqk 1/1 Running 0 2m
+```
+
+## Bring Up a Service
+
+Optionally, you can bring up a simple service to be managed by XOS.
+This involves deploying two additional helm charts: `base-kubernetes` and `demo-simpleexampleservice`. Again from the `~/cord/helm-charts`
+directory, execute the following:
+
+```shell
+helm dep update xos-profiles/base-kubernetes
+helm install xos-profiles/base-kubernetes -n base-kubernetes
+helm dep update xos-profiles/demo-simpleexampleservice
+helm install xos-profiles/demo-simpleexampleservice -n demo-simpleexampleservice
+```
+
+> **Note:** It will take some time for the various helm charts to
+> deploy and the containers to come online. The `tosca-loader`
+> container may error and retry several times as it waits for
+> services to be dynamically loaded. This is normal, and eventually
+> the `tosca-loader` will enter the completed state.
+
+As before, when all the containers are successfully up and running,
+`kubectl get pod` will return output that looks something like this:
+
+```shell
+NAME READY STATUS RESTARTS AGE
+base-kubernetes-kubernetes-55c55bd897-rn9ln 1/1 Running 0 2m
+base-kubernetes-tosca-loader-vs6pv 1/1 Running 1 2m
+demo-simpleexampleservice-787454b84b-ckpn2 1/1 Running 0 1m
+demo-simpleexampleservice-tosca-loader-4q7zg 1/1 Running 0 1m
+xos-chameleon-6f49b67f68-pdf6n 1/1 Running 0 12m
+xos-core-57fd788db-8b97d 1/1 Running 0 12m
+xos-db-f9ddc6589-rtrml 1/1 Running 0 12m
+xos-gui-7fcfcd4474-prhfb 1/1 Running 0 12m
+xos-redis-74c5cdc969-ppd7z 1/1 Running 0 12m
+xos-tosca-7c665f97b6-krp5k 1/1 Running 0 12m
+xos-ws-55d676c696-pxsqk 1/1 Running 0 12m
+```
+
+## Visit XOS Dashboard
+
+Finally, to view the XOS dashboard, run the following:
+
+```shell
+minikube service xos-gui
+```
+
+This will launch a window in your default browser. Administrator login
+and password are defined in `~/cord/helm-charts/xos-core/values.yaml`.
+
+## Next Steps
+
+This completes the installation process. At this point, you can
+drill down on the internals of `SimpleExampleService` by visiting
+its [CORD Guid tutorial](https://guide.opencord.org/simpleexampleservice/simple-example-service.html).
diff --git a/docs/intro.md b/docs/intro.md
new file mode 100644
index 0000000..69cf65a
--- /dev/null
+++ b/docs/intro.md
@@ -0,0 +1,41 @@
+# Writing Models and Synchronizers
+
+CORD adopts a model-based design, which is to say all aspects
+of operating and managing CORD is mediated by a model-based
+control plane. XOS is a tool that CORD uses to implement this
+control plane. For an overview of XOS, see the following
+white paper:
+[XOS: Modeling-as-a-Service](https://wiki.opencord.org/display/CORD/XOS+and+the+CORD+Controller?preview=/1279317/4981376/XOS%20Modeling-as-a-Service.pdf).
+
+XOS has three inter-related aspects, and this section is
+organized accordingly.
+
+* **Modeling Framework:** XOS defines a
+ [modeling framework](dev/xproto.md), which
+ includes both a modeling language (*xproto*) and a generative
+ toolchain (*xosgenx*). The abstractions that define CORD's
+ behavior are expressed in xproto, with xosgenx then used to
+ generate code for several elements required to control CORD
+ (including an API that serves the set of models that have been
+ loaded into XOS). Service developers typically write one or more
+ models to on-board their service.
+
+* **Synchronizer Framework:** XOS defines a
+ [synchronization framework](dev/synchronizers.md)
+ that actuates the CORD data model. This framework is reponsible for
+ driving the underlying components configured into CORD (for example,
+ services, access devices) towards the desired state. Service developers
+ typically write a synchronizer to on-board their services.
+
+* **Core Models and Policies:** The CORD platform is defined by
+ a [core](core_models.md) set of *xproto* models, plus a set of
+ [security policies](security_policies.md) that govern how
+ various principals can act on those models in a multi-tenant
+ environment. Platform developers typically define and evolve the
+ core models and policies, which effectively establishes the
+ foundation on which all services run and are interconnected into
+ service graphs.
+
+
+
+
diff --git a/docs/mdl_relaxed.rb b/docs/mdl_relaxed.rb
new file mode 100644
index 0000000..bebc671
--- /dev/null
+++ b/docs/mdl_relaxed.rb
@@ -0,0 +1,53 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# markdownlint(mdl) relaxed rules file
+#
+# Rule descriptions:
+# https://github.com/markdownlint/markdownlint/blob/master/docs/RULES.md
+# Instructions on modifying the rules file:
+# https://github.com/markdownlint/markdownlint/blob/master/docs/creating_styles.md
+
+# use all rules
+all
+
+# Exclude rule: Header levels should only increment by one level at a time
+exclude_rule 'MD001'
+
+# Indent lists with 4 spaces
+rule 'MD007', :indent => 4
+
+# Exclude rule: Trailing spaces
+exclude_rule 'MD009'
+
+# Don't enforce line length limitations
+exclude_rule 'MD013'
+
+# Allow ! and ? as trailing punctuation in headers
+rule 'MD026', :punctuation => '.,;:'
+
+# Exclude rule: Multiple spaces after blockquote symbol
+exclude_rule 'MD027'
+
+# Numbered lists should have the correct order
+rule 'MD029', :style => "ordered"
+
+# Exclude rule: Fenced code blocks should be surrounded by blank lines
+exclude_rule 'MD031'
+
+# Exclude rule: Emphasis used instead of a header
+exclude_rule 'MD036'
+
+# Gitbook won't care about multiple blank lines
+exclude_rule 'MD012'
diff --git a/docs/mdl_strict.rb b/docs/mdl_strict.rb
new file mode 100644
index 0000000..2e02e7c
--- /dev/null
+++ b/docs/mdl_strict.rb
@@ -0,0 +1,37 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# markdownlint(mdl) strict rules file
+# To use: `make LINT_STYLE=mdl_strict.rb lint`
+#
+# Rule descriptions:
+# https://github.com/markdownlint/markdownlint/blob/master/docs/RULES.md
+# Instructions on modifying the rules file:
+# https://github.com/markdownlint/markdownlint/blob/master/docs/creating_styles.md
+
+# use all rules
+all
+
+# Indent lists with 4 spaces
+rule 'MD007', :indent => 4
+
+# Don't enforce line length limitations within code blocks and tables
+rule 'MD013', :code_blocks => false, :tables => false
+
+# Allow ! and ? as trailing punctuation in headers
+rule 'MD026', :punctuation => '.,;:'
+
+# Numbered lists should have the correct order
+rule 'MD029', :style => "ordered"
+
diff --git a/docs/overview.md b/docs/overview.md
new file mode 100644
index 0000000..b35ccce
--- /dev/null
+++ b/docs/overview.md
@@ -0,0 +1,42 @@
+# XOS Overview
+
+XOS provides a framework for defining a set of declarative models and
+then using those models to manage a collection of components that have
+been configured into an operational system. XOS is itself deployed as
+a set of micro-services, collectively forming a
+*ControlPlane-as-a-Service* that:
+
+* Serves as a single unifying interface to a collection of backend
+ services, avoiding the management silos that otherwise result from
+ disaggregation. This includes a framework for creating and
+ operating on services across organizational boundaries, across a range
+ of implementations, and across multiple tenants.
+
+* Implements end-to-end service chains across a service mesh, supporting
+ visibility and control at the granularity of individual subscribers
+ or flows. This provides a fine-grain means to correlate diagnostic and
+ monitoring information, allocate resources and isolate performance,
+ and distribute/migrate functionality.
+
+XOS is currently being used in three projects:
+
+* **CORD Controller:** XOS is a central part of CORD, providing a coherent
+ service control plane that runs on on top of a mix of disaggregated
+ access technologies, legacy VNFs running in OpenStack VMs, horizontally
+ scalable micro-services running in Kubernetes, and SDN control
+ applications running on ONOS.
+
+* **Network Edge Mediator (NEM):** XOS is being used to provide a
+ mediation layer for SEBA (Software-Enabled Broadband Access),
+ addressing the challenge of how to integrate an edge site with different
+ (and potentially multiple) global orchestrators and legacy OSS/BSS.
+
+* **End-to-End Service Chains in a Multi-Cloud:** XOS is being used
+ to manage end-to-end service chains that span customer premises,
+ edge sites, and commodity clouds.
+
+For additional white papers describing XOS, see the project
+[wiki page](https://wiki.opencord.org/display/CORD/XOS+and+the+CORD+Controller).
+
+
+
diff --git a/docs/use.md b/docs/use.md
new file mode 100644
index 0000000..a242713
--- /dev/null
+++ b/docs/use.md
@@ -0,0 +1,31 @@
+# Using XOS
+
+XOS has three inter-related aspects, and this section is
+organized accordingly.
+
+* **Modeling Framework:** XOS defines a
+ [modeling framework](dev/xproto.md), which
+ includes both a modeling language (*xproto*) and a generative
+ toolchain (*xosgenx*). The abstractions that define a system's
+ behavior are expressed in xproto, with xosgenx then used to
+ generate code for several elements required to control the system
+ (including an API that serves the set of models that have been
+ loaded into XOS). Service developers typically write one or more
+ models to on-board their service.
+
+* **Synchronizer Framework:** XOS defines a
+ [synchronization framework](dev/synchronizers.md)
+ that actuates the XOS data model. This framework is reponsible for
+ driving the underlying components configured into a system
+ (for example, services, access devices) towards the desired state.
+ Service developers typically write a synchronizer to on-board their
+ services.
+
+* **Core Models and Policies:** A system as a whole is defined by
+ a [core](core_models.md) set of *xproto* models, plus a set of
+ [security policies](security_policies.md) that govern how
+ various principals can act on those models in a multi-tenant
+ environment. Platform developers typically define and evolve the
+ core models and policies, which effectively establishes the
+ foundation on which all services run and are interconnected into
+ service graphs.