Async/streaming gRPC client/server proto

This experiment was to fine-tune how we can implement
async gRPC client and server code inside a Twisted
python app.

Change-Id: I945014e27f4b9d6ed624666e0284cc298548adb3

Major cleanup of openflow_13.proto

Change-Id: I4e54eaf87b682124ec518a0ade1a6050a6ec6da8

Relocated openflow_13.proto to voltha

Change-Id: I66ae45a9142d180c2c6651e75c7a1ee08aef7ef8

Removed forced utest from make build

Change-Id: If0da58e9d135ebde6ca68c3316688a03a7b10f2f

twisted openflow agent first pass

Change-Id: Ibe5b4727ccfe92e6fd464ccd3baf6275569ef5d3

store openflow derived files

Change-Id: Ib3e1384bb2ca2a9c0872767f7b793f96b0a154e2

Minor cleanup

Change-Id: I1280ed3acb606121b616a0efd573f5f59d010dca

Factored out common utils

Change-Id: Icd86fcd50f60d0900924674cbcd65e13e47782a1

Refactored twisted agent

Change-Id: I71f26ce5357a4f98477df60b8c5ddc068cf75d43

Relocated openflow agent to ofagent

... and preserved obsolete working (non-twisted) agent under
~/obsolete, so we can still run the olt-oftest and pass tests,
unit the new twisted based agent reaches that maturity point.

Change-Id: I727f8d7144b1291a40276dad2966b7643bd7bc4b

olt-oftest in fake mode works with new agent

Change-Id: I43b4f5812e8dfaa9f45e4a77fdcf6c30ac520f8d

Initial ofagent/voltha operation

Change-Id: Ia8104f1285a6b1c51635d36d7d78fc113f800e79

Additional callouts to Voltha

Change-Id: If8f483d5140d3c9d45f22b480b8d33249a29cd4e

More gRPC calls

Change-Id: I7d24fadf9425217fb26ffe18f25359d072ef38fa

Flow add/list now works

Change-Id: Ie3e3e73108645b47891cef798fc61372a022fd93

Missed some files

Change-Id: I29e81238ff1a26c095c0c73e521579edf7092e21
diff --git a/common/utils/message_queue.py b/common/utils/message_queue.py
new file mode 100644
index 0000000..74b763f
--- /dev/null
+++ b/common/utils/message_queue.py
@@ -0,0 +1,89 @@
+#
+# Copyright 2016 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+from twisted.internet.defer import Deferred
+from twisted.internet.defer import succeed
+
+
+class MessageQueue(object):
+    """
+    An event driven queue, similar to twisted.internet.defer.DeferredQueue
+    but which allows selective dequeing based on a predicate function.
+    Unlike DeferredQueue, there is no limit on backlog, and there is no queue
+    limit.
+    """
+
+    def __init__(self):
+        self.waiting = []  # tuples of (d, predicate)
+        self.queue = []  # messages piling up here if no one is waiting
+
+    def reset(self):
+        """
+        Purge all content as well as waiters (by errback-ing their entries).
+        :return: None
+        """
+        for d, _ in self.waiting:
+            d.errback(Exception('mesage queue reset() was called'))
+        self.waiting = []
+        self.queue = []
+
+    def _cancelGet(self, d):
+        """
+        Remove a deferred from our waiting list.
+        :param d: The deferred that was been canceled.
+        :return: None
+        """
+        for i in range(len(self.waiting)):
+            if self.waiting[i][0] is d:
+                self.waiting.pop(i)
+
+    def put(self, obj):
+        """
+        Add an object to this queue
+        :param obj: arbitrary object that will be added to the queue
+        :return:
+        """
+
+        # if someone is waiting for this, return right away
+        for i in range(len(self.waiting)):
+            d, predicate = self.waiting[i]
+            if predicate is None or predicate(obj):
+                self.waiting.pop(i)
+                d.callback(obj)
+                return
+
+        # otherwise...
+        self.queue.append(obj)
+
+    def get(self, predicate=None):
+        """
+        Attempt to retrieve and remove an object from the queue that
+        matches the optional predicate.
+        :return: Deferred which fires with the next object available.
+        If predicate was provided, only objects for which
+        predicate(obj) is True will be considered.
+        """
+        for i in range(len(self.queue)):
+            msg = self.queue[i]
+            if predicate is None or predicate(msg):
+                self.queue.pop(i)
+                return succeed(msg)
+
+        # there were no matching entries if we got here, so we wait
+        d = Deferred(canceller=self._cancelGet)
+        self.waiting.append((d, predicate))
+        return d
+
+