updated stand-alone content
Change-Id: I9902768b3ada80e9e9c5db1194fcf848fbfe4bc2
diff --git a/VERSION b/VERSION
index 227cea2..335195d 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-2.0.0
+2.0.1.dev
diff --git a/docs/Makefile b/docs/Makefile
index 83fb922..e82b741 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -20,7 +20,7 @@
build: setup
gitbook build
-setup: xos-gui xos-tosca
+setup: xos-gui xos-tosca kubernetes openstack onos simpleexampleservice exampleservice
gitbook init
gitbook install
@@ -44,6 +44,22 @@
xos-tosca:
ln -s ../../xos-tosca/docs xos-tosca
+kubernetes:
+ ln -s ../../services/kubernetes-service/docs kubernetes
+
+openstack:
+ ln -s ../../services/openstack/docs openstack
+
+onos:
+ ln -s ../../services/onos-service/docs onos
+
+exampleservice:
+ ln -s ../../services/exampleservice/docs exampleservice
+
+simpleexampleservice:
+ ln -s ../../services/simpleexampleservice/docs simpleexampleservice
+
+
# HOW TO
# - load the virtual env (https://guide.opencord.org/xos/dev/local_env.html)
# - run `make`
@@ -75,4 +91,4 @@
rm -rf $(GENERATED_DOCS)
rm -rf _book
rm -rf node_modules
- rm -rf xos xos-gui xos-tosca
+ rm -rf xos xos-gui xos-tosca kubernetes openstack onos simpleexampleservice exampleservice
diff --git a/docs/README.md b/docs/README.md
index 69cf65a..b35ccce 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -1,41 +1,42 @@
-# Writing Models and Synchronizers
+# XOS Overview
-CORD adopts a model-based design, which is to say all aspects
-of operating and managing CORD is mediated by a model-based
-control plane. XOS is a tool that CORD uses to implement this
-control plane. For an overview of XOS, see the following
-white paper:
-[XOS: Modeling-as-a-Service](https://wiki.opencord.org/display/CORD/XOS+and+the+CORD+Controller?preview=/1279317/4981376/XOS%20Modeling-as-a-Service.pdf).
+XOS provides a framework for defining a set of declarative models and
+then using those models to manage a collection of components that have
+been configured into an operational system. XOS is itself deployed as
+a set of micro-services, collectively forming a
+*ControlPlane-as-a-Service* that:
-XOS has three inter-related aspects, and this section is
-organized accordingly.
+* Serves as a single unifying interface to a collection of backend
+ services, avoiding the management silos that otherwise result from
+ disaggregation. This includes a framework for creating and
+ operating on services across organizational boundaries, across a range
+ of implementations, and across multiple tenants.
-* **Modeling Framework:** XOS defines a
- [modeling framework](dev/xproto.md), which
- includes both a modeling language (*xproto*) and a generative
- toolchain (*xosgenx*). The abstractions that define CORD's
- behavior are expressed in xproto, with xosgenx then used to
- generate code for several elements required to control CORD
- (including an API that serves the set of models that have been
- loaded into XOS). Service developers typically write one or more
- models to on-board their service.
+* Implements end-to-end service chains across a service mesh, supporting
+ visibility and control at the granularity of individual subscribers
+ or flows. This provides a fine-grain means to correlate diagnostic and
+ monitoring information, allocate resources and isolate performance,
+ and distribute/migrate functionality.
-* **Synchronizer Framework:** XOS defines a
- [synchronization framework](dev/synchronizers.md)
- that actuates the CORD data model. This framework is reponsible for
- driving the underlying components configured into CORD (for example,
- services, access devices) towards the desired state. Service developers
- typically write a synchronizer to on-board their services.
-
-* **Core Models and Policies:** The CORD platform is defined by
- a [core](core_models.md) set of *xproto* models, plus a set of
- [security policies](security_policies.md) that govern how
- various principals can act on those models in a multi-tenant
- environment. Platform developers typically define and evolve the
- core models and policies, which effectively establishes the
- foundation on which all services run and are interconnected into
- service graphs.
+XOS is currently being used in three projects:
+* **CORD Controller:** XOS is a central part of CORD, providing a coherent
+ service control plane that runs on on top of a mix of disaggregated
+ access technologies, legacy VNFs running in OpenStack VMs, horizontally
+ scalable micro-services running in Kubernetes, and SDN control
+ applications running on ONOS.
+
+* **Network Edge Mediator (NEM):** XOS is being used to provide a
+ mediation layer for SEBA (Software-Enabled Broadband Access),
+ addressing the challenge of how to integrate an edge site with different
+ (and potentially multiple) global orchestrators and legacy OSS/BSS.
+
+* **End-to-End Service Chains in a Multi-Cloud:** XOS is being used
+ to manage end-to-end service chains that span customer premises,
+ edge sites, and commodity clouds.
+
+For additional white papers describing XOS, see the project
+[wiki page](https://wiki.opencord.org/display/CORD/XOS+and+the+CORD+Controller).
diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
index 30f0576..dcd11fd 100644
--- a/docs/SUMMARY.md
+++ b/docs/SUMMARY.md
@@ -1,8 +1,8 @@
# Summary
* [Introduction](README.md)
-* [Overview](overview.md)
* [Installing XOS](install.md)
+ * [Download Source Code](repo.md)
* [Using XOS](use.md)
* [XOS Modeling Framework](dev/xproto.md)
* [XOS Tool Chain (Internals)](dev/xosgenx.md)
@@ -12,4 +12,12 @@
* [Synchronizer Reference](dev/sync_reference.md)
* [Core Models](core_models.md)
* [Security Policies](security_policies.md)
+ * [Tutorial](tutorials/basic_synchronizer.md)
+* [Platform Services](platform.md)
+ * [Kubernetes](kubernetes/kubernetes-service.md)
+ * [OpenStack](openstack/openstack-service.md)
+ * [ONOS](onos/README.md)
+* [Example Services](examples.md)
+ * [Simple Example Service](simpleexampleservice/simple-example-service.md)
+ * [Example Service](exampleservice/example-service.md)
diff --git a/docs/core_models.md b/docs/core_models.md
index f3b2e7a..6351857 100644
--- a/docs/core_models.md
+++ b/docs/core_models.md
@@ -1,20 +1,21 @@
# Core Models
-The XOS modeling framework provides a foundation for building CORD,
-but it is just a tool used to define a set of core models. It is these
-core models that provide a coherent interface for configuring,
-controlling, and applying policy to a CORD POD. This gives operators a
-way to specify and reason about the behavior of CORD, while allowing
-for a wide range of implementation choices for the underlying software
-components.
+The XOS modeling framework provides a foundation for building systems
+like CORD, but it is just a tool used to define a set of core models. It is
+these core models that provide a coherent interface for configuring,
+controlling, and applying policies to an operational system. This
+gives operators a way to specify and reason about the system's
+behavior, while allowing for a wide range of implementation choices
+for the underlying software components.
## Services, Slices, and ServiceInstances
-CORD's core starts with the **Service** model, which represents all
-functionality that can be on-boarded into CORD. This model is designed to meet
-two requirements. The first is to be implementation-agnostic, supporting both
-*server-based* implementations (e.g., legacy VNFs running in VMs and
-micro-services running in containers) and *switch-based* implementations (e.g.,
+The XOS core starts with the **Service** model, which represents all
+functionality that can be on-boarded into an XOS-managed system.
+This model is designed to meet two requirements. The first is to be
+implementation-agnostic, supporting both *server-based*
+implementations (e.g., legacy VNFs running in VMs and micro-services
+running in containers) and *switch-based* implementations (e.g.,
SDN control applications that install flow rules into white-box switches). The
second is to be multi-tenant, supporting isolated and virtualized instances
that can be created on behalf of both trusted and untrusted tenants.
@@ -75,13 +76,13 @@
## Service Graphs and Service Chains
Given a set of Services (and their corresponding Slices and ServiceInstances),
-CORD also defines two core models for interconnecting them:
+XOS also defines two core models for interconnecting them:
**ServiceDependencies** and **ServiceInstanceLinks**. The first defines a
-dependency of one Service on another, thereby forming a CORD-wide *Service
+dependency of one Service on another, thereby forming a system-wide *Service
Graph*. The second defines a dependency between a pair of ServiceInstances,
thereby forming a per-subscriber *Service Chain*.
-> NOTE: Service Graphs and Service Chains are not explicit models in CORD, but
+> NOTE: Service Graphs and Service Chains are not explicit models in XOS, but
> rather, they are defined by a set of vertices (Services, ServiceInstances)
> and edges (ServiceDependency, ServiceInstanceLink).
@@ -119,7 +120,7 @@
## Model Glossary
-CORD's core models are defined by a set of [xproto](dev/xproto.md)
+The XOS core models are defined by a set of [xproto](dev/xproto.md)
specifications. They are defined in their full detail in the source code (see
[core.xproto](https://github.com/opencord/xos/blob/master/xos/core/models/core.xproto)).
The following summarizes these core models—along with the key relationships
@@ -139,16 +140,16 @@
having a "controller" which effectively corresponds to the
`Service` model itself (i.e., its purpose is to generate a "control
interface" for the service). There is no "Controller" model
- bound to a service. (Confusingly, CORD does include a `Controller`
+ bound to a service. (Confusingly, XOS does include a `Controller`
model, but it represents information about OpenStack. There is
- also a `ServiceController` construct in the TOSCA interface for
- CORD, which provides a means to load the `Service` model for
- a given service into CORD.)
+ also a `ServiceController` construct in the TOSCA interface,
+ which provides a means to load the `Service` model for
+ a given service into XOS.)
* **ServiceDependency:** Represents a dependency between a *Subscriber* service
on a *Provider* service. The set of `ServiceDependency` and `Service` models
- defined in CORD collectively represent the edges and verticies of a *Service
- Graph*, but there is no explicit "ServiceGraph" model in CORD. The dependency
+ collectively represent the edges and verticies of a *Service
+ Graph*, but there is no explicit "ServiceGraph" model. The dependency
between a pair of services is parameterized by the `connect_method` by which
the service are interconnected in the data plane.Connect methods include:
@@ -164,15 +165,14 @@
Service might be called a "Volume" or a "Bucket." Confusingly, there are also
instances of a `Service` model that represent different services, but this is
a consequence of standard modeling terminology, whereas `ServiceInstance` is
- a core model in CORD (and yes, there are instances of the `ServiceInstance`
+ a core model (and yes, there are instances of the `ServiceInstance`
model).
* **ServiceInstanceLink:** Represents a logical connection between
`ServiceInstances` of two `Services`. A related model, `ServiceInterface`,
types the `ServiceInstanceLink` between two `ServiceInstances`. A connected
sequence of `ServiceInstances` and `ServiceInstanceLinks` form what is often
- called a *Service Chain*, but there is no explicit "ServiceChain" model in
- CORD.
+ called a *Service Chain*, but there is no explicit "ServiceChain" model.
* **Slice:** Represents a distributed resource container that includes the
compute and network resources that belong to (are used by) some `Service`.
@@ -220,7 +220,7 @@
* **Site:** Represents a logical grouping of `Nodes` that are co-located at the
same geographic location, which also typically corresponds to the nodes'
location in the physical network. The typical use case involves one
- configuration of a CORD POD deployed at a single location, although the
+ configuration of a system deployed at a single location, although the
underlying core includes allows for multi-site deployments.
* Bound to a set of `Nodes` located at the `Site`.
diff --git a/docs/dev/sync_reference.md b/docs/dev/sync_reference.md
index 0cfe959..dba54c9 100644
--- a/docs/dev/sync_reference.md
+++ b/docs/dev/sync_reference.md
@@ -1,79 +1,79 @@
# Synchronizer Reference
-This section is a reference for the commonly used APIs exposed
-by the synchronizer framework.
+This section is a reference for the commonly used APIs exposed
+by the synchronizer framework.
-## Convenience Methods
+## Convenience Methods
-As part of the model definition, it is possible to extend the autogenerated gRPC
-APIs with custom methods, for example, to facilitate access to some kind of data
-from the synchronizers.
+As part of the model definition, it is possible to extend the autogenerated gRPC
+APIs with custom methods, for example, to facilitate access to some kind of data
+from the synchronizers.
-`convenience_methods` can be defined in a folder (`convenience`) that needs to
-be a child of the `models_dir` as per your configuration.
+`convenience_methods` can be defined in a folder (`convenience`) that needs to
+be a child of the `models_dir` as per your configuration.
For example if your configuration contains:
-```yaml
+```yaml
models_dir: "/opt/xos/synchronizers/<synchronizer_name>/models"
```
-then you `convenience methods` needs to be located in
+then you `convenience methods` needs to be located in
`/opt/xos/synchronizers/<synchronizer_name>/models/convenience`
Assuming our model definition looks like:
-```proto
+```proto
message MyModel (XOSBase){
required string first_name = 1 [null = False, blank = False];
required string last_name = 2 [null = False, blank = False];
}
```
-here is an example of a basic convenience methods that will expose a
+here is an example of a basic convenience methods that will expose a
`full_name` property over the APIs used by the synchronizers:
-```python
-from xosapi.orm import ORMWrapper, register_convenience_wrapper
+```python
+from xosapi.orm import ORMWrapper, register_convenience_wrapper
-from xosconfig import Config
-from multistructlog import create_logger
+from xosconfig import Config
+from multistructlog import create_logger
-log = create_logger(Config().get('logging'))
+log = create_logger(Config().get('logging'))
class ORMWrapperMyModel(ORMWrapper):
- @property
+ @property
def full_name(self):
- return "%s %s" % (self.first_name, self.last_name)
+ return "%s %s" % (self.first_name, self.last_name)
-register_convenience_wrapper("MyModel", ORMWrapperMyModel)
+register_convenience_wrapper("MyModel", ORMWrapperMyModel)
```
-> **Note:** The convenience methods will be loaded in all the synchronizer
-> containers so that they can be used in multiple places.
+> **Note:** The convenience methods will be loaded in all the synchronizer
+> containers so that they can be used in multiple places.
-## Model Policies
+## Model Policies
-Model Policies can be seen as `post-save` hooks and they are generally defined
-in the `xos/synchronizer/model_policies` folder of your service.
+Model Policies can be seen as `post-save` hooks and they are generally defined
+in the `xos/synchronizer/model_policies` folder of your service.
-Model policies are generally used to dynamically create a service chain (when a
-ServiceInstance is created it will create a ServiceInstance of its east side
-Service)
+Model policies are generally used to dynamically create a service chain (when a
+ServiceInstance is created it will create a ServiceInstance of its east side
+Service).
-> **Note:** You'll need to add this folder in your synchronizer configuration file
+> **Note:** You'll need to add this folder in your synchronizer configuration file
> as:
>
-> ```yaml
+> ```yaml
> model_policies_dir: "/opt/xos/synchronizers/<synchronizer_name>/model_policies"
> ```
A model policy is a class that inherits from `Policy`:
-```python
-from synchronizers.new_base.modelaccessor import MyServiceInstance, ServiceInstanceLink, model_accessor
-from synchronizers.new_base.policy import Policy
+```python
+from synchronizers.new_base.modelaccessor import MyServiceInstance, ServiceInstanceLink, model_accessor
+from synchronizers.new_base.policy import Policy
class MyServiceInstancePolicy(Policy):
model_name = "MyServiceInstance"
@@ -81,185 +81,185 @@
and overrides one or more of the following methods:
-```python
+```python
def handle_create(self, model):
```
-```python
+```python
def handle_update(self, model):
```
-```python
+```python
def handle_delete(self, model):
```
-Where `model` is the instance of the model that has been created.
+Where `model` is the instance of the model that has been created.
-## Sync Steps
+## Sync Steps
-Sync Steps are the actual piece of code that provide the mapping between your
+Sync Steps are the actual piece of code that provide the mapping between your
models and your backend. You will need to define a sync step for each model.
> **Note:** You'll need to add this folder in your synchronizer configuration file
> as:
>
-> ```yaml
+> ```yaml
> steps_dir: "/opt/xos/synchronizers/<synchronizer_name>/steps"
> ```
A Sync Step is a class that inherits from `SyncStep`:
-```python
+```python
-from synchronizers.new_base.SyncInstanceUsingAnsible import SyncStep
-from synchronizers.new_base.modelaccessor import MyModel
+from synchronizers.new_base.SyncInstanceUsingAnsible import SyncStep
+from synchronizers.new_base.modelaccessor import MyModel
-from xosconfig import Config
-from multistructlog import create_logger
+from xosconfig import Config
+from multistructlog import create_logger
-log = create_logger(Config().get('logging'))
+log = create_logger(Config().get('logging'))
class SyncMyModel(SyncStep):
provides = [MyModel]
- observes = MyModel
+ observes = MyModel
```
and provides these methods:
-```python
+```python
def sync_record(self, o):
- log.info("sync'ing object", object=str(o), **o.tologdict())
+ log.info("sync'ing object", object=str(o), **o.tologdict())
```
-```python
+```python
def delete_record(self, o):
log.info("deleting object", object=str(o), **o.tologdict())
```
-This methods will be invoked anytime there is change in the model passing as
-argument the changed models. After performing the required operations to sync
-the model state with the backend state the synchronizer framework will update
-the models with the operational informations needed.
+This methods will be invoked anytime there is change in the model passing as
+argument the changed models. After performing the required operations to sync
+the model state with the backend state the synchronizer framework will update
+the models with the operational informations needed.
-## Pull Steps
+## Pull Steps
-Pull Steps can be used to observe the surrounding environment and update the
-data-model accordingly.
+Pull Steps can be used to observe the surrounding environment and update the
+data-model accordingly.
-> **Note:** You'll need to add this folder in your synchronizer configuration file
+> **Note:** You'll need to add this folder in your synchronizer configuration file
> as:
>
-> ```yaml
+> ```yaml
> pull_steps_dir: "/opt/xos/synchronizers/<synchronizer_name>/pull_steps"
> ```
A Sync Step is a class that inherits from `PullStep`
-```python
-from synchronizers.new_base.pullstep import PullStep
-from synchronizers.new_base.modelaccessor import OLTDevice
+```python
+from synchronizers.new_base.pullstep import PullStep
+from synchronizers.new_base.modelaccessor import OLTDevice
-from xosconfig import Config
-from multistructlog import create_logger
+from xosconfig import Config
+from multistructlog import create_logger
-log = create_logger(Config().get('logging'))
+log = create_logger(Config().get('logging'))
-from synchronizers.new_base.modelaccessor import MyModel
+from synchronizers.new_base.modelaccessor import MyModel
class MyModelPullStep(PullStep):
def __init__(self):
- super(MyModelPullStep, self).__init__(observed_model=OLTDevice)
+ super(MyModelPullStep, self).__init__(observed_model=OLTDevice)
```
and override the following method:
-```python
+```python
def pull_records(self):
- log.info("pulling MyModels")
- # create an empty model
- o = MyModel()
- # code to fetch information
- # populate the model
+ log.info("pulling MyModels")
+ # create an empty model
+ o = MyModel()
+ # code to fetch information
+ # populate the model
o.first_name = "John"
o.last_name = "Doe"
- o.no_sync = True # this is required to prevent the synchronizer to be invoked and start a loop
- o.save()
+ o.no_sync = True # this is required to prevent the synchronizer to be invoked and start a loop
+ o.save()
```
-## Event Steps
+## Event Steps
-Event Steps are similar to pull steps in that they are often used to implement a flow of information from the environment into the data model. However, rather than using polling, event steps rely on externally generated events delivered via an event bus, such as Kafka.
+Event Steps are similar to pull steps in that they are often used to implement a flow of information from the environment into the data model. However, rather than using polling, event steps rely on externally generated events delivered via an event bus, such as Kafka.
-> **Note:** You'll need to add this folder in your synchronizer configuration file
+> **Note:** You'll need to add this folder in your synchronizer configuration file
> as:
>
-> ```yaml
+> ```yaml
> event_steps_dir: "/opt/xos/synchronizers/<synchronizer_name>/event_steps"
> ```
>
-> You'll also need to make sure the event bus endpoint is specified in the
+> You'll also need to make sure the event bus endpoint is specified in the
> synchronizer config file. For example:
>
-> ```yaml
+> ```yaml
> event_bus:
-> endpoint: cord-kafka-kafka
-> kind: kafka
+> endpoint: cord-kafka
+> kind: kafka
>```
An event step inherits from the `EventStep` class:
-```python
+```python
-import json
-from synchronizers.new_base.eventstep import EventStep
-from synchronizers.new_base.modelaccessor import MyModel
-from xosconfig import Config
-from multistructlog import create_logger
+import json
+from synchronizers.new_base.eventstep import EventStep
+from synchronizers.new_base.modelaccessor import MyModel
+from xosconfig import Config
+from multistructlog import create_logger
-log = create_logger(Config().get('logging'))
+log = create_logger(Config().get('logging'))
class MyModelEventStep(EventStep):
technology = "kafka"
topics = ["MyTopic"]
def __init__(self, *args, **kwargs):
- super(MyEventStep, self).__init__(*args, **kwargs)
+ super(MyEventStep, self).__init__(*args, **kwargs)
```
-Two important class members that are defined in each event step are `technology` and `topics`. `technology` tells what type of event bus to use. There's currently only one bus interface implemented by the synchronizer framework, and that is `kafka`. The `topics` member is a list of topics that will be listened on for events. The precise meaning of `topics` is left to the particular event bus technology that is in use.
+Two important class members that are defined in each event step are `technology` and `topics`. `technology` tells what type of event bus to use. There's currently only one bus interface implemented by the synchronizer framework, and that is `kafka`. The `topics` member is a list of topics that will be listened on for events. The precise meaning of `topics` is left to the particular event bus technology that is in use.
Service-specific logic is implemented by overriding the `process_event()` method:
-```python
+```python
def process_event(self, event):
- value = json.loads(event.value)
+ value = json.loads(event.value)
first_name = value["first_name"]
last_name = value["last_name"]
- # See if the object already exists
- objs = MyModel.filter(first_name=first_name, last_name=last_name)
+ # See if the object already exists
+ objs = MyModel.filter(first_name=first_name, last_name=last_name)
if objs:
- return
+ return
- # Create a new object
- obj = MyModel()
- obj.first_name = first_name
- obj.last_name = last_name
- obj.save(always_update_timestamp = True)
+ # Create a new object
+ obj = MyModel()
+ obj.first_name = first_name
+ obj.last_name = last_name
+ obj.save(always_update_timestamp = True)
```
-In this example we've made the assumption that the value of an event is a json-encoded dictionary containing the keys `first_name` and `last_name`. The event step in this case checks to see if an object with those fields already exists, and if not then it creates the object.
+In this example we've made the assumption that the value of an event is a json-encoded dictionary containing the keys `first_name` and `last_name`. The event step in this case checks to see if an object with those fields already exists, and if not then it creates the object.
-In this example, we've differed from the Pull Step example in that we omitted `no_sync=True` and we added `always_update_timestamp = True` to the `save()` call. This has the effect of causing the synchronizer to excute any sync steps that might exist for `MyModel`. Whether or not you want sync_steps to run is an implementation decision and depends upon the design of your synchronizer.
+In this example, we've differed from the Pull Step example in that we omitted `no_sync=True` and we added `always_update_timestamp = True` to the `save()` call. This has the effect of causing the synchronizer to excute any sync steps that might exist for `MyModel`. Whether or not you want sync_steps to run is an implementation decision and depends upon the design of your synchronizer.
Sending an event to Kafka can be done using a variety of Kafka clients for various languages, Kafka command-line tools, etc. A python example is as follows:
-```python
-import json
-from kafka import KafkaProducer
-producer = KafkaProducer(bootstrap_servers="cord-kafka-kafka")
-producer.send("MyTopic", json.dumps({"first_name": "John", "last_name": "Doe"}))
-producer.flush()
+```python
+import json
+from kafka import KafkaProducer
+producer = KafkaProducer(bootstrap_servers="cord-kafka")
+producer.send("MyTopic", json.dumps({"first_name": "John", "last_name": "Doe"}))
+producer.flush()
```
diff --git a/docs/dev/xosgenx.md b/docs/dev/xosgenx.md
index a6abc1e..58bb3e9 100644
--- a/docs/dev/xosgenx.md
+++ b/docs/dev/xosgenx.md
@@ -4,12 +4,15 @@
passes it to a target, which in turn generates the output code. The target has
access to a library of auxiliary functions implemented in Python. The target
itself is written as a jinja2 template. The following figure depicts the
-processing pipeline. Developers will not need to interact with xosgenx
-directly, except when there is an opportunity to create a new target.
+processing pipeline, using the `slice.xproto` model and `Django`
+target as an example.
+
+> **Note:** Developers will not need to interact with xosgenx
+> directly, except when there is an opportunity to create a new target.
-## Intermediate Representation (IR)
+## Intermediate Representation (IR)
The IR is a representation of a parsed xproto file in the form of nested Python
dictionaries. Here is a description of its structure.
diff --git a/docs/examples.md b/docs/examples.md
new file mode 100644
index 0000000..170be7e
--- /dev/null
+++ b/docs/examples.md
@@ -0,0 +1,7 @@
+# Example Services
+
+This section describes two example services that have been on-boarded
+into XOS.
+
+* [Simple Example Service](simpleexampleservice/simple-example-service.md): Runs a web server in a Kubernetes-managed pod.
+* [Example Service](exampleservice/example-service.md): Runs a web server in an OpenStack-managed VM.
diff --git a/docs/install.md b/docs/install.md
index 88f125a..568f17b 100644
--- a/docs/install.md
+++ b/docs/install.md
@@ -86,19 +86,25 @@
The helm charts used to deploy XOS are currently bundled in the CORD
`helm-chart` repository. The rest of this section assumes all you download
-this repository into a directory named `~/cord`.
+this repository into a directory named `~/xos`.
```shell
-mkdir ~/cord
-cd ~/cord
+mkdir ~/xos
+cd ~/xos
git clone https://gerrit.opencord.org/helm-charts
cd helm-charts
```
+While downloading the simple `helm-charts` repository is sufficient
+for bringing up XOS, you may also want to download the XOS source
+code, for example, so you can walk through the
+[XOS tutorial](tutorial/basic_synchronizer.md). The easiest way to do
+this uses the `repo` tool, as described [here](repo.md).
+
## Bring Up XOS
To deploy `xos-core` (plus affiliated micro-services) into your
-Kubernetes cluster, execute the following from the `~/cord/helm-charts`
+Kubernetes cluster, execute the following from the `~/xos/helm-charts`
directory:
```shell
@@ -117,14 +123,15 @@
xos-db-f9ddc6589-rtrml 1/1 Running 0 2m
xos-gui-7fcfcd4474-prhfb 1/1 Running 0 2m
xos-redis-74c5cdc969-ppd7z 1/1 Running 0 2m
-xos-tosca-7c665f97b6-krp5k 1/1 Running 0 2m
-xos-ws-55d676c696-pxsqk 1/1 Running 0 2m
+xos-tosca-7c665f97b6-krp5k 1/1 Running 0 2m
+xos-ws-55d676c696-pxsqk 1/1 Running 0 2m
```
## Bring Up a Service
Optionally, you can bring up a simple service to be managed by XOS.
-This involves deploying two additional helm charts: `base-kubernetes` and `demo-simpleexampleservice`. Again from the `~/cord/helm-charts`
+This involves deploying two additional helm charts: `base-kubernetes`
+and `demo-simpleexampleservice`. Again from the `~/xos/helm-charts`
directory, execute the following:
```shell
@@ -167,10 +174,11 @@
```
This will launch a window in your default browser. Administrator login
-and password are defined in `~/cord/helm-charts/xos-core/values.yaml`.
+and password are defined in `~/xos/helm-charts/xos-core/values.yaml`.
## Next Steps
This completes the installation process. At this point, you can
-drill down on the internals of `SimpleExampleService` by visiting
-its [CORD Guid tutorial](https://guide.opencord.org/simpleexampleservice/simple-example-service.html).
+drill down on the internals of
+[Simple Example Service](simpleexampleservice/simple-example-service.md).
+
diff --git a/docs/overview.md b/docs/overview.md
deleted file mode 100644
index b35ccce..0000000
--- a/docs/overview.md
+++ /dev/null
@@ -1,42 +0,0 @@
-# XOS Overview
-
-XOS provides a framework for defining a set of declarative models and
-then using those models to manage a collection of components that have
-been configured into an operational system. XOS is itself deployed as
-a set of micro-services, collectively forming a
-*ControlPlane-as-a-Service* that:
-
-* Serves as a single unifying interface to a collection of backend
- services, avoiding the management silos that otherwise result from
- disaggregation. This includes a framework for creating and
- operating on services across organizational boundaries, across a range
- of implementations, and across multiple tenants.
-
-* Implements end-to-end service chains across a service mesh, supporting
- visibility and control at the granularity of individual subscribers
- or flows. This provides a fine-grain means to correlate diagnostic and
- monitoring information, allocate resources and isolate performance,
- and distribute/migrate functionality.
-
-XOS is currently being used in three projects:
-
-* **CORD Controller:** XOS is a central part of CORD, providing a coherent
- service control plane that runs on on top of a mix of disaggregated
- access technologies, legacy VNFs running in OpenStack VMs, horizontally
- scalable micro-services running in Kubernetes, and SDN control
- applications running on ONOS.
-
-* **Network Edge Mediator (NEM):** XOS is being used to provide a
- mediation layer for SEBA (Software-Enabled Broadband Access),
- addressing the challenge of how to integrate an edge site with different
- (and potentially multiple) global orchestrators and legacy OSS/BSS.
-
-* **End-to-End Service Chains in a Multi-Cloud:** XOS is being used
- to manage end-to-end service chains that span customer premises,
- edge sites, and commodity clouds.
-
-For additional white papers describing XOS, see the project
-[wiki page](https://wiki.opencord.org/display/CORD/XOS+and+the+CORD+Controller).
-
-
-
diff --git a/docs/platform.md b/docs/platform.md
new file mode 100644
index 0000000..2c0a42d
--- /dev/null
+++ b/docs/platform.md
@@ -0,0 +1,15 @@
+# Platform Services
+
+XOS adopts an *Everything-as-a-Service* model. This includes
+platform (infrastructure) services like Kubernetes and OpenStack.
+Either or both can be integrated into an XOS-managed system,
+and like any other service, they have models and synchronizers.
+The only difference is that their models are part of the
+[XOS core](core-models.md) (e.g., Slices, Instances, Networks) rather
+than service extensions. This section describes three example platform
+services in more detail:
+
+* [Kubernetes](kubernetes/docs/kubernetes-service.md)
+* [OpenStack](openstack/docs/openstack-service.md)
+* [ONOS](onos/docs/README.md)
+
diff --git a/docs/repo.md b/docs/repo.md
new file mode 100644
index 0000000..c6a0c6d
--- /dev/null
+++ b/docs/repo.md
@@ -0,0 +1,53 @@
+# Download Source Code
+
+The easiest way to download source code for XOS -- including the
+XOS core, the XOS-related interfaces (e.g., GUI, TOSCA), the helm
+charts used to deploy XOS, and the model/synchronizers
+for the services being managed by XOS -- is to use the `repo` tool.
+
+## Install repo
+
+If you don't already have `repo` installed, you may be able to install
+it with your system package manager, or you can follow these
+[instructions from the android source site](https://source.android.com/source/downloading#installing-repo):
+
+```sh
+curl -o /tmp/repo 'https://gerrit.opencord.org/gitweb?p=repo.git;a=blob_plain;f=repo;hb=refs/heads/stable'
+echo '394d93ac7261d59db58afa49bb5f88386fea8518792491ee3db8baab49c3ecda /tmp/repo' | sha256sum -c -
+sudo mv /tmp/repo /usr/local/bin/repo
+sudo chmod a+x /usr/local/bin/repo
+```
+
+> **Note:** You may want to install `repo` using the official
+> repository instead. We forked the original repository and host a copy of the
+> file to make repo downloadable also by organizations that don't have access
+> to Google servers.
+
+## Download Repositories
+
+The XOS repositories are usually checked out to `~/xos` in most of our
+examples:
+
+```shell
+mkdir ~/xos
+cd ~/xos
+repo init -u https://gerrit.opencord.org/xos-manifest -b master
+repo sync
+```
+
+When this is complete, a listing (`ls`) inside this directory should yield
+output similar to:
+
+```shell
+./ helm-charts/ xos-rest-gw/
+../ profiles/ xos-sample-gui-extension/
+.repo/ services/ xos-tosca/
+automation-tools/ xos-core/
+components/ xos-gui/
+```
+
+## Development Notes
+
+You can also use `repo` to download the latest patchsets from Gerrit, and
+to contribute code back into Gerrit (for review). More information on how
+to do both are available on the [CORD Wiki](https://wiki.opencord.org/display/CORD/Working+with+Gerrit).
diff --git a/docs/security_policies.md b/docs/security_policies.md
index b6aa8e3..c028e47 100644
--- a/docs/security_policies.md
+++ b/docs/security_policies.md
@@ -1,8 +1,9 @@
# Security Policies
-CORD security policies are implemented by XOS. These policies answer the
-question: *Who can do what?* The *who* in this case generally refers to a user
-(represented by a *User* model), but it can also refer to an API context. The
+XOS provides hooks for specifying security policies. These policies
+answer the question: *Who can do what?* The *who* in this case
+generally refers to a user (represented by a *User* model), but it can
+also refer to an API context. The
*what* refers to two things: (1) the piece of information being accessed (a
model, an object, or a field within that object), and (2) the access type
(whether it is a read, a write, or a privilege update).
diff --git a/docs/tutorials/basic_synchronizer.md b/docs/tutorials/basic_synchronizer.md
new file mode 100644
index 0000000..778621f
--- /dev/null
+++ b/docs/tutorials/basic_synchronizer.md
@@ -0,0 +1,536 @@
+# Tutorial
+
+This section walks you through the process of writing an XOS
+model and synchronizer. This exercise is not extensive. Its only
+purpose is to guide you through the most common use cases.
+This includes:
+
+- How to define XOS models.
+- How to load the models into the `xos-core`.
+- How to write a TOSCA recipe that creates an instance of the models.
+- How to write a `sync_step` for the models.
+- How to write unit tests for a `sync_step` (TODO).
+
+## Prerequisites
+
+The following assumes you start with a simple deployment of
+XOS on `minikube` as described [here](../install.md), so that
+running `kubectl get pods` returns something similar to:
+
+```shell
+NAME READY STATUS RESTARTS AGE
+xos-chameleon-6fb76d5689-s7vxb 1/1 Running 0 21h
+xos-core-58bcf4f477-79hs7 1/1 Running 0 21h
+xos-db-566dd8c6f9-l24h5 1/1 Running 0 21h
+xos-gui-665c5f85bc-kdmbm 1/1 Running 0 21h
+xos-redis-5cf77fd49f-fcw5h 1/1 Running 0 21h
+xos-tosca-69588f677c-77lll 1/1 Running 0 20h
+xos-ws-748c7f9f75-cnjnh 1/1 Running 0 21h
+```
+
+The tutorial also assumes you have downloaded the XOS source code,
+as described [here](../repo.md).
+
+## Directory Structure
+
+XOS services are located under `~/xos/services`. The first step
+is to create a new directory to store our models and synchronizer
+code. We will use the name `hello-world` for this example:
+
+```shell
+cd ~/xos/services
+mkdir hello-world
+cd hello-world
+```
+
+Although empty when we start, we will end up with a directory
+structure that looks like the following. You can look at the
+corresponding directories of other services in `~/xos/services`
+for examples.
+
+```shell
+hello-world
+├── Dockerfile.synchronizer
+├── VERSION
+├── samples
+│ └── hello-world.yaml
+└── xos
+ ├── synchronizer
+ │ ├── config.yaml
+ │ ├── hello-world-synchronizer.py
+ │ ├── models
+ │ │ └── hello-world.xproto
+ │ ├── steps
+ │ │ ├── sync_hello_world_service.py
+ │ │ ├── sync_hello_world_service_instance.py
+ │ │ ├── test_sync_hello_world_service.py
+ │ │ └── test_sync_hello_world_service_instance.py
+ │ └── test_config.yaml
+ └── unittest.cfg
+```
+
+Walking through the structure, we see the following:
+
+- `Dockerfile.synchronizer` specifies the Docker image we will build
+ to run the synchronizer.
+- `VERSION` specifies the version of our code; it is reported to the core.
+- `xos/synchronizer` contains all the code that will be bundled in the
+ Docker image.
+
+Looking at some of the files, we see:
+
+- `xos/synchronizer/models/hello-world.xproto` contains the model definitions.
+- `samples/hello-world.yaml` is an example of a TOSCA recipe to instantiate those models.
+- `xos/synchronizer/hello-world-synchronizer.py` is the main synchronizer process.
+- `xos/synchronizer/steps/sync_hello_world_service.py` contains the
+ operations needed to synchronize the backend component.
+
+## Create the Synchronizer Entry Point
+
+The synchronizer entry point (main looping process) is responsible for:
+
+- loading the synchronizer configuration
+- loading and running the synchronizer framework
+
+Cut-and-paste the following into `hello-world-synchronizer.py`:
+
+```python
+import importlib
+import os
+import sys
+from xosconfig import Config
+
+config_file = os.path.abspath(os.path.dirname(os.path.realpath(__file__)) + '/config.yaml')
+Config.init(config_file, 'synchronizer-config-schema.yaml')
+
+observer_path = os.path.join(os.path.dirname(os.path.realpath(__file__)),"../../synchronizers/new_base")
+sys.path.append(observer_path)
+mod = importlib.import_module("xos-synchronizer")
+mod.main()
+```
+
+### Define the Synchronizer Configuration
+
+In `xos/synchronizer/config.yaml` add this content:
+
+```yaml
+name: hello-world
+accessor:
+ username: admin@opencord.org
+ password: letmein
+ endpoint: xos-core:50051
+models_dir: "/opt/xos/synchronizers/hello-world/models"
+steps_dir: "/opt/xos/synchronizers/hello-world/steps"
+required_models:
+ - HelloWorldService
+ - HelloWorldServiceInstance
+logging:
+ version: 1
+ handlers:
+ console:
+ class: logging.StreamHandler
+ loggers:
+ 'multistructlog':
+ handlers:
+ - console
+ level: DEBUG
+```
+
+This tells the synchronizer framework in a running system where
+to fine the configuration parameters specific to HelloWorld.
+
+## Define Models
+
+For our example, we are going to define the two most common models for
+an XOS-managed service: `HelloWorldService` and `HelloWorldServiceInstance`.
+These are HelloWorld-specific definitions of the two
+[core models](../core_models.md#model-glossary):
+
+- `Service`: defines service-wide parameters/fields.
+- `ServiceInstance`: defines subscriber-specific parameters/fields.
+
+Open the `hello-world.xproto` file and add the following content:
+
+```text
+option name = "hello-world";
+option app_label = "hello-world";
+
+message HelloWorldService (Service){
+ required string hello_from = 1 [help_text = "The name of who is saying hello", null = False, db_index = False, blank = False];
+}
+
+message HelloWorldServiceInstance (ServiceInstance){
+ option owner_class_name="HelloWorldService";
+ required string hello_to = 1 [help_text = "The name of who is being greeted", null = False, db_index = False, blank = False];
+}
+```
+
+This specifies two models: `HelloWorldService` extends the
+`Service` model, and `HelloWorldServiceInstance` extends the
+`ServiceInstance` model. Both of these models inherit the
+attributes defined in the parent classes, which you can see in
+file `~/xos/xos-core/core/models/core.xproto`.
+
+## Load the Models into the Core
+
+Service models are pushed to the core through a mechanism referred to
+as `dynamic onboarding` or `dynamic loading`. In practice, when a
+synchronizer container runs, the first thing it does is to push its
+models into the core container.
+
+But first we need to build and deploy our synchronizer container in
+the test environment.
+
+### Build the Synchronizer Container
+
+We assume that you are familiar with the Docker concepts of
+*container* and *image*. If not, we encourage you to look here:
+[Docker concepts](https://docs.docker.com/get-started/#docker-concepts)
+
+The first thing we need to do is to define a `Dockerfile`. To
+do that, open `Dockerfile.synchronizer` and add the following
+content:
+
+```text
+FROM xosproject/xos-synchronizer-base:candidate
+
+COPY xos/synchronizer /opt/xos/synchronizers/hello-world
+COPY VERSION /opt/xos/synchronizers/hello-world/
+
+ENTRYPOINT []
+
+WORKDIR "/opt/xos/synchronizers/hello-world"
+
+CMD bash -c "python hello-world-synchronizer.py"
+```
+
+This file is used to build our synchronizer container image. As you
+might have noticed, the container image we're defining inherits
+`FROM xosproject/xos-synchronizer-base:candidate`, so we'll need
+to obtain that image.
+
+We can use the following commands to do this:
+
+```shell
+eval $(minikube docker-env) # this will point our shell on the minikube docker daemon
+docker pull xosproject/xos-synchronizer-base:master
+docker tag xosproject/xos-synchronizer-base:master xosproject/xos-synchronizer-base:candidate
+```
+
+Now we can build our synchronizer image by executing the following
+from the `~/xos/service/hello-world` directory:
+
+```shell
+eval $(minikube docker-env)
+docker build -t xosproject/hello-world-synchronizer:candidate -f Dockerfile.synchronizer .
+```
+
+### Run Your Synchronizer Container
+
+You can create a simple Kubernetes resource in a file called `kb8s-hello-world.yaml`:
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+ name: hello-world-synchronizer
+spec:
+ containers:
+ - name: hello-world-synchronizer
+ image: xosproject/hello-world-synchronizer:candidate
+ volumeMounts:
+ - name: certchain-volume
+ mountPath: /usr/local/share/ca-certificates/local_certs.crt
+ subPath: config/ca_cert_chain.pem
+ volumes:
+ - name: certchain-volume
+ configMap:
+ name: ca-certificates
+ items:
+ - key: chain
+ path: config/ca_cert_chain.pem
+ restartPolicy: Never
+```
+
+and then run it using `kubectl create -f kb8s-hello-world.yaml`
+
+Check the logs of your synchronizer using:
+
+```shell
+kubetcl logs -f hello-world-synchronizer
+```
+
+You should see output similar to the following:
+
+```text
+Service version is 1.0.0.dev
+required_models, found: models=HelloWorldService, HelloWorldServiceInstance
+Loading sync steps step_dir=/opt/xos/synchronizers/hello-world/steps synchronizer_name=hello-world
+Loaded sync steps steps=[] synchronizer_name=hello-world
+Skipping event engine due to no event_steps dir. synchronizer_name=hello-world
+Skipping model policies thread due to no model_policies dir. synchronizer_name=hello-world
+No sync steps, no policies, and no event steps. Synchronizer exiting. synchronizer_name=hello-world
+```
+
+Check that your models are on-boarded in the XOS GUI by opening the GUI:
+
+```shell
+minikube service xos-gui
+```
+
+Use the default credentials `admin@opencord.org/letmein` to login.
+
+### Create TOSCA Recipes to Instantiate Your Models
+
+The models you defined earlier in this tutorial are, more precisely,
+a model schema. Once your model schema has been loaded into the
+XOS core, you can create one or more *instances* of those models.
+We typically use a TOSCA recipe to do this.
+
+The XOS TOSCA engine automatically understands workflows for any
+models that have been loaded into the core. You can consult them
+at any time connecting to the TOSCA endpoint from a browser:
+
+```text
+http://<minikube-ip>:30007
+```
+
+> **Note:** You can find the minikube ip by executing this command on your
+> system:`minikube ip`.
+
+In this page you should find a list of all the available resources. Just search for
+`helloworldservice` and visit the corresponding page at:
+
+```text
+http://<minikube-ip>:30007/custom_type/helloworldservice
+```
+
+You will see the TOSCA definition for the `HelloWorldService` model.
+
+You can use that (and the `HelloWorldServiceInstance` model definition
+too) to create an instance of both models. For your convenience, save
+the following content to a file called `hello-world-tosca.yaml`
+```yaml
+tosca_definitions_version: tosca_simple_yaml_1_0
+imports:
+ - custom_types/helloworldservice.yaml
+ - custom_types/helloworldserviceinstance.yaml
+ - custom_types/servicegraphconstraint.yaml
+
+description: Create an instance of HelloWorldService and one of HelloWorldServiceInstance
+
+topology_template:
+ node_templates:
+
+ service:
+ type: tosca.nodes.HelloWorldService
+ properties:
+ name: HelloWorld
+ hello_from: Jhon Snow
+
+ serviceinstance:
+ type: tosca.nodes.HelloWorldServiceInstance
+ properties:
+ name: HelloWorld Service Instance
+ hello_to: Daenerys Targaryen
+
+ constraints:
+ type: tosca.nodes.ServiceGraphConstraint
+ properties:
+ constraints: '["HelloWorld"]'
+```
+
+This TOSCA will create an instance of your service and
+an instance of your service instance.
+
+You can then submit this TOSCA using this command:
+
+```shell
+curl -H "xos-username: admin@opencord.org" -H "xos-password: letmein" -X POST --data-binary @hello-world-tosca.yaml http://<minikube-ip>:30007/run
+Created models: ['service', 'serviceinstance', 'serviceinstance']
+```
+
+Once this command has been executed, connect to the GUI at:
+
+```shell
+http://<minikube-ip>:30001
+```
+
+and see that your models have been instantiated.
+
+> **Note:** In the home page press the `Service Instances` button to display
+> `ServiceInstance` models, and the navigate to the `Hello World`
+> sub-menu to the left.
+
+## Create your First Synchronizer Steps
+
+Everything up to this point (with the exception of defining the models
+themselves) is the boilerplate needed to run a synchronizer. It is the
+`sync_step` that is actually responsible for mapping changes in the
+XOS data model into some action on the backend component XOS is
+managing.
+
+To keep the tutorial simple we are not going to operate on a real
+component, but we can demonstrate the basic idea of how a `sync_step`
+interacts with the models.
+
+### Successful `sync_step`
+
+Before continuing, let's remove the container we just deployed. Do
+this by running:
+
+```shell
+kubectl delete pod hello-world-synchronizer
+```
+
+To write the `sync_step` we need to create two files in
+`xos/synchronizer/sync_step`. The first one synchronizes the
+`HelloWorldService` model and it is called
+`sync_hello_world_service.py`.
+
+Every `sync_step` extends the `SyncStep` base class, and overrides two
+methods:
+
+- `sync_record`
+- `delete_record`
+
+See the [synchronizer reference](../dev/sync_reference.md#sync-steps)
+for more details.
+
+Here is an example of `sync_step` that simply logs changes on
+the `HelloWorldService` model:
+
+```python
+from synchronizers.new_base.SyncInstanceUsingAnsible import SyncStep
+from synchronizers.new_base.modelaccessor import HelloWorldService
+
+from xosconfig import Config
+from multistructlog import create_logger
+
+log = create_logger(Config().get('logging'))
+
+class SyncHelloWorldService(SyncStep):
+ provides = [HelloWorldService]
+
+ observes = HelloWorldService
+
+ def sync_record(self, o):
+ log.info("HelloWorldService has been updated!", object=str(o), hello_from=o.hello_from)
+
+ def delete_record(self, o):
+ log.info("HelloWorldService has been deleted!", object=str(o), hello_from=o.hello_from)
+```
+
+Let's deploy this first step and see what happens. The first thing you
+need to do is rebuild the synchronizer container:
+
+```shell
+eval $(minikube docker-env)
+docker build -t xosproject/hello-world-synchronizer:candidate -f Dockerfile.synchronizer .
+```
+
+Once done, restart it as follows:
+
+```shell
+kubectl create -f kb8s-hello-world.yaml
+```
+
+At his point, running `kubectl logs -f hello-world-synchronizer`
+should show that your synchronizer is no longer exiting, but is now
+looping while waiting for changes in the models.
+
+Every time you make a change to the model, you will see:
+
+- The event is logged in the synchronizer log (`kubectl logs -f hello-world-synchronizer`)
+- The `backend_code` and backend status of the model are updated
+- The model is not picked up by the synchronizer until you make some changes to it
+
+When you make changes to the models (you can do this via the GUI or by
+updating the TOSCA you created before), you will see a message similar
+to this one in the logs:
+
+```shell
+Syncing object model_name=HelloWorldService pk=1 synchronizer_name=hello-world thread_id=140152420452096
+HelloWorldService has been updated! hello_from=u'Jhon Snow' object=HelloWorld
+Synced object model_name=HelloWorldService pk=1 synchronizer_name=hello-world thread_id=140152420452096
+```
+
+> **Note:** The `sync_record` method is triggered also when a model is created,
+> so as soon as you start the synchronizer you will see the above message.
+
+If you delete the model, you'll see the `delete_record` method being invoked.
+
+### Handling Errors in a `sync_step`
+
+We are now going to trigger an error, to demonstrate how the synchronizer
+framework is going to help us in dealing with them.
+
+Let's start creating the `sync_step` for `HelloWorldServiceInstance`
+in a file named `sync_hello_world_service.py`.
+
+```python
+from synchronizers.new_base.SyncInstanceUsingAnsible import SyncStep, DeferredException
+from synchronizers.new_base.modelaccessor import HelloWorldServiceInstance
+
+from xosconfig import Config
+from multistructlog import create_logger
+
+log = create_logger(Config().get('logging'))
+
+class SyncHelloWorldServiceInstance(SyncStep):
+ provides = [HelloWorldServiceInstance]
+
+ observes = HelloWorldServiceInstance
+
+ def sync_record(self, o):
+ log.debug("HelloWorldServiceInstance has been updated!", object=str(o), hello_to=o.hello_to)
+
+ if o.hello_to == "Tyrion Lannister":
+ raise DeferredException("Maybe later")
+
+ if o.hello_to == "Joffrey Baratheon":
+ raise Exception("Maybe not")
+
+ log.info("%s is saying hello to %s" % (o.owner.leaf_model.hello_from, o.hello_to))
+
+ def delete_record(self, o):
+ log.debug("HelloWorldServiceInstance has been deleted!", object=str(o), hello_to=o.hello_to)
+```
+
+To run this code you will need to:
+
+- delete the running container
+- rebuild the image
+- run the container again
+
+In this case we are emulating an error in our `sync_step`. In reality,
+this can be caused by a connection error, malformed data, or any
+of a number of reasons.
+
+Go to the GUI and start playing a little bit with the models!
+
+If you set the `HelloWorldServiceInstance.hello_to` property to `Tyrion Lannister`
+you will see this keep popping up:
+
+```shell
+HelloWorldServiceInstance has been updated! hello_to=u'Tyrion Lannister' object=HelloWorld Service Instance
+sync step failed! e=DeferredException('Maybe later',) model_name=HelloWorldServiceInstance pk=1 synchronizer_name=hello-world
+Traceback (most recent call last):
+ File "/opt/xos/synchronizers/new_base/event_loop.py", line 357, in sync_cohort
+ self.sync_record(o, log)
+ File "/opt/xos/synchronizers/new_base/event_loop.py", line 227, in sync_record
+ step.sync_record(o)
+ File "/opt/xos/synchronizers/hello-world/steps/sync_hello_world_service_instance.py", line 18, in sync_record
+ raise DeferredException("Maybe later")
+DeferredException: Maybe later
+```
+
+Here is what happens when an error occurs. The synchronizer framework will:
+
+- Log the exception
+- Set the `backend_code` of that instance to `2`
+- Set the `Exception` error in the `backend_status`
+- Keep retrying
+
+> **Note:** To see `backend_code` and `backend_status` in the GUI you
+> can press `d` to open the debug tab while looking at a model detail view.
diff --git a/docs/use.md b/docs/use.md
index a242713..c719bc6 100644
--- a/docs/use.md
+++ b/docs/use.md
@@ -21,9 +21,9 @@
Service developers typically write a synchronizer to on-board their
services.
-* **Core Models and Policies:** A system as a whole is defined by
- a [core](core_models.md) set of *xproto* models, plus a set of
- [security policies](security_policies.md) that govern how
+* **Core Models and Policies:** An XOS-managed system is
+ defined by a [core](core_models.md) set of *xproto* models, plus
+ a set of [security policies](security_policies.md) that govern how
various principals can act on those models in a multi-tenant
environment. Platform developers typically define and evolve the
core models and policies, which effectively establishes the