Config/management model framework initial version

This is the initial commit of the internal config
model tree for Voltha.

A system that borrows key concepts from Docker, repo,
and git for multi-revision configuration handling with
transaction (commit/abort) logic, tagging, and the
ability to diff between config versions.

Key features:
* Stored model is defined using *annotated* Google protobuf
  (*.proto) files
* Model is kept in memory as Python protobuf message objects
* The whole configuration is arranged in a nested (tree)
  structure, made up of ConfigNode objects. Each
  ConfigNode holds its config data as (possibly nested)
  protobuf message object, as well as lists of "pointers"
  to its logically nested children nodes. What message
  fields are considered part of the node-local config vs.
  what is stored as a child node is controlled by "child_node"
  annotations in the *.proto files.
* Each ConifgNode stores its config in immutable
  ConfigRevision obects, each revision being identified
  by a unique hash value, calculated from a unique
  hash value of its local configuration as well as
  the list of hash values of all its children node.
* Collection type children nodes can be indexed (keyed)
  so that they can be addressed with "path" notation
  friendly to REST and other interfaces/APIs. Which
  field is used as key is defined in the parent message
  using "key" attribute of the "child_node" annotation.
  Currently string and integer type fields can be used
  as keys.
* Changes to the model create new revisions in all
  affected nodes, which are rolled up as new revisions
  to the root node.
* Root revisions can be tagged, tags can be moved
* The root node by default keeps a rev, but provides
  a mechanism to purge untagged revs.
* All branch and leaf nodes auto-purge interim revs
  not needed. A rev is not needed if no one refers
  to it.
* Diffing between revs is supported, it yields RFC6902
  jsonpatch objects. Diffing can be done between any
  revs.
* The main operations are: CRUD (we call them .add,
  .get, .update, .remove)
* Get can be recursive to an optionally limited depth.
* There is support for Read-Only attributes (fields)
* All CRUD operation support "path" based addressing.
* There is support for an branch/leaf node perspective
  implemented by ConfigProxy. This proxy, when attached
  to an arbitrary node in the tree, provides all the
  CRUD operations in that context, that is, path args
  are used relative to that node.
* Transaction support: All mutations made in a transaction
  are invisible to others until the transaction is committed.
  The commit is atomic (either all changes are applied
  to the tree or none). Conflicts between transactions
  are detected at the per-node level and a conflict
  results in rejecting the conflicting transaction (first
  one wins).
* Registered callbacks: via the proxy objects an
  observer can register for pre- and post- operation
  callbacks. Also, there is a post-get callback which
  can be used to augment stored data with real-time
  data.

I started hooking up the new config infrastructure to
Voltha's CORE, but this is still in progress, as not
all existing APIs have bee moved over yet.

Note: I also lumped in some experimental files working
with "Any" types in protobufs

Change-Id: Ic547b36e9b893d54e6d9ce67bdfcb32a6e8acd4c
diff --git a/experiments/extensions/Makefile b/experiments/extensions/Makefile
new file mode 100644
index 0000000..67a06c9
--- /dev/null
+++ b/experiments/extensions/Makefile
@@ -0,0 +1,22 @@
+default: test
+
+build:
+	protoc -I . --python_out=. ext1.proto
+	protoc -I . --python_out=. ext2.proto
+
+test: build
+	./write_generic.py | ./read_generic.py
+	./write_generic.py | ./read_ext1.py
+	./write_generic.py | ./read_ext2.py
+	./write_generic.py | ./read_both.py
+
+	./write_ext1.py | ./read_generic.py
+	./write_ext1.py | ./read_ext1.py
+	./write_ext1.py | ./read_ext2.py
+	./write_ext1.py | ./read_both.py
+
+	./write_ext2.py | ./read_generic.py
+	./write_ext2.py | ./read_ext1.py
+	./write_ext2.py | ./read_ext2.py
+	./write_ext2.py | ./read_both.py
+
diff --git a/experiments/extensions/README.md b/experiments/extensions/README.md
new file mode 100644
index 0000000..058aae3
--- /dev/null
+++ b/experiments/extensions/README.md
@@ -0,0 +1,6 @@
+This is an experiment to work with Any type fields in protobuf and
+in Python. Run 'make' to see how a set of writers emit data with
+no or one of two extensions, while four readers try to decode the
+data. Depending on which of them pre-loads the extension protobufs,
+they can decode the extension data or leave it in the packed value
+format.
diff --git a/experiments/extensions/__init__.py b/experiments/extensions/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/experiments/extensions/__init__.py
diff --git a/experiments/extensions/ext1.proto b/experiments/extensions/ext1.proto
new file mode 100644
index 0000000..d8fd448
--- /dev/null
+++ b/experiments/extensions/ext1.proto
@@ -0,0 +1,17 @@
+syntax = "proto3";
+
+package vendor.x1;
+
+// Vendor specific adapter descriptor data
+message AdapterDescription {
+    string internal_name = 1;
+    string internal_code = 2;
+    uint32 price = 3;
+}
+
+// Vendor specific adapter configuration data
+message AdapterConfig {
+    uint32 volume = 1;
+    uint32 bass = 2;
+    uint32 treble = 3;
+}
diff --git a/experiments/extensions/ext2.proto b/experiments/extensions/ext2.proto
new file mode 100644
index 0000000..ac9f1ec
--- /dev/null
+++ b/experiments/extensions/ext2.proto
@@ -0,0 +1,24 @@
+syntax = "proto3";
+
+package vendor.x2;
+
+// Vendor specific adapter descriptor data
+message AdapterDescription {
+    string foo = 1;
+    uint32 arg1 = 2;
+    uint32 arg2 = 3;
+    uint32 arg3 = 4;
+    uint32 arg4 = 5;
+    uint32 arg5 = 6;
+    uint32 arg6 = 7;
+}
+
+// Vendor specific adapter configuration data
+message AdapterConfig {
+    uint64 conf1 = 1;
+    uint64 conf2 = 2;
+    uint64 conf3 = 3;
+    uint64 conf4 = 4;
+    uint64 conf5 = 5;
+    repeated string things = 6;
+}
diff --git a/experiments/extensions/read_both.py b/experiments/extensions/read_both.py
new file mode 100755
index 0000000..682e51a
--- /dev/null
+++ b/experiments/extensions/read_both.py
@@ -0,0 +1,18 @@
+#!/usr/bin/env python
+
+"""
+Read adapter data while decoding both known custom fields
+"""
+
+import sys
+from json import dumps
+
+from common.utils.json_format import MessageToDict
+from voltha.protos import adapter_pb2
+
+adapter = adapter_pb2.Adapter()
+binary = sys.stdin.read()
+adapter.ParseFromString(binary)
+
+print dumps(MessageToDict(adapter, strict_any_handling=False))
+
diff --git a/experiments/extensions/read_ext1.py b/experiments/extensions/read_ext1.py
new file mode 100755
index 0000000..8032615
--- /dev/null
+++ b/experiments/extensions/read_ext1.py
@@ -0,0 +1,16 @@
+#!/usr/bin/env python
+"""
+Read adapter data while decoding ext1 custom fields.
+"""
+import sys
+from json import dumps
+
+from common.utils.json_format import MessageToDict
+from voltha.protos import adapter_pb2
+
+adapter = adapter_pb2.Adapter()
+binary = sys.stdin.read()
+adapter.ParseFromString(binary)
+
+print dumps(MessageToDict(adapter, strict_any_handling=False))
+
diff --git a/experiments/extensions/read_ext2.py b/experiments/extensions/read_ext2.py
new file mode 100755
index 0000000..a187137
--- /dev/null
+++ b/experiments/extensions/read_ext2.py
@@ -0,0 +1,16 @@
+#!/usr/bin/env python
+"""
+Read adapter data while decoding ext2 custom fields
+"""
+import sys
+from json import dumps
+
+from common.utils.json_format import MessageToDict
+from voltha.protos import adapter_pb2
+
+adapter = adapter_pb2.Adapter()
+binary = sys.stdin.read()
+adapter.ParseFromString(binary)
+
+print dumps(MessageToDict(adapter, strict_any_handling=False))
+
diff --git a/experiments/extensions/read_generic.py b/experiments/extensions/read_generic.py
new file mode 100755
index 0000000..e910188
--- /dev/null
+++ b/experiments/extensions/read_generic.py
@@ -0,0 +1,15 @@
+#!/usr/bin/env python
+"""
+Read adapter data while ignoring custom fields
+"""
+import sys
+from json import dumps
+
+from common.utils.json_format import MessageToDict
+from voltha.protos import adapter_pb2
+
+adapter = adapter_pb2.Adapter()
+binary = sys.stdin.read()
+adapter.ParseFromString(binary)
+print dumps(MessageToDict(adapter, strict_any_handling=False))
+
diff --git a/experiments/extensions/write_ext1.py b/experiments/extensions/write_ext1.py
new file mode 100755
index 0000000..57131ef
--- /dev/null
+++ b/experiments/extensions/write_ext1.py
@@ -0,0 +1,40 @@
+#!/usr/bin/env python
+"""
+Write adapter data without any custom fields
+"""
+import sys
+
+import ext1_pb2
+from voltha.protos import adapter_pb2
+from google.protobuf import any_pb2
+
+def custom_config():
+    any = any_pb2.Any()
+    any.Pack(ext1_pb2.AdapterConfig(
+        volume=20,
+        bass=50,
+        treble=50
+    ))
+    return any
+
+
+def custom_description():
+    any = any_pb2.Any()
+    any.Pack(ext1_pb2.AdapterDescription(
+        internal_name='hulu',
+        internal_code='foo',
+        price=42
+    ))
+    return any
+
+
+adapter = adapter_pb2.Adapter(
+    id='42',
+    config=adapter_pb2.AdapterConfig(
+        additional_config=custom_config()
+    ),
+    additional_description=custom_description()
+)
+
+sys.stdout.write(adapter.SerializeToString())
+
diff --git a/experiments/extensions/write_ext2.py b/experiments/extensions/write_ext2.py
new file mode 100755
index 0000000..b9316a1
--- /dev/null
+++ b/experiments/extensions/write_ext2.py
@@ -0,0 +1,47 @@
+#!/usr/bin/env python
+"""
+Write adapter data without any custom fields
+"""
+import sys
+
+import ext2_pb2
+from voltha.protos import adapter_pb2
+from google.protobuf import any_pb2
+
+
+def custom_config():
+    any = any_pb2.Any()
+    any.Pack(ext2_pb2.AdapterConfig(
+        conf1=1,
+        conf2=42,
+        conf3=0,
+        conf4=11111111111,
+        conf5=11231231,
+        things = ['foo', 'bar', 'baz', 'zoo']
+    ))
+    return any
+
+
+def custom_description():
+    any = any_pb2.Any()
+    any.Pack(ext2_pb2.AdapterDescription(
+        foo='hulu',
+        arg1=42,
+        arg2=42,
+        arg3=42,
+        arg4=42,
+        arg5=42
+    ))
+    return any
+
+
+adapter = adapter_pb2.Adapter(
+    id='42',
+    config=adapter_pb2.AdapterConfig(
+        additional_config=custom_config()
+    ),
+    additional_description=custom_description()
+)
+
+sys.stdout.write(adapter.SerializeToString())
+
diff --git a/experiments/extensions/write_generic.py b/experiments/extensions/write_generic.py
new file mode 100755
index 0000000..1cfca6c
--- /dev/null
+++ b/experiments/extensions/write_generic.py
@@ -0,0 +1,15 @@
+#!/usr/bin/env python
+"""
+Write adapter data without any custom fields
+"""
+import sys
+
+from voltha.protos import adapter_pb2
+
+adapter = adapter_pb2.Adapter(
+    id='42',
+    config=adapter_pb2.AdapterConfig()
+)
+
+sys.stdout.write(adapter.SerializeToString())
+