VOL-1108: Initial checking of common PKI/PM manager & reporting
Cleanup of License headers and renamed file extension for markdown file
Added additional ONU Historical Intervals and improved README.md files
Change-Id: I704c4363b857749bca4fce5ee184d18edc3cebce
diff --git a/voltha/extensions/pki/README.md b/voltha/extensions/pki/README.md
new file mode 100644
index 0000000..903c457
--- /dev/null
+++ b/voltha/extensions/pki/README.md
@@ -0,0 +1,227 @@
+# VOLTHA Performance Monitoring/PKI Library
+
+This directory provides a common library for the creation of Performance Monitoring groups
+within VOLTHA and should be used to insure that PKI information from different adapters use
+the same format
+
+## PKI Manager Creation
+
+Currently, each device adapter is required to follow the following steps to create and
+register PM Metric manager. This is typically performed in the device handler's
+'activate' method (called in response to the device handler first being enabled)
+
+1. Create an instance of a **AdapterPmMetrics** manager object. This is typically an
+ **OltPmMetrics** object for an _OLT_ adapter, or an **OnuPmMetrics** adapter for an
+ _ONU_ adapter. If you have additional device specific metrics to report, you can
+ derive your own manager object from one of these two derived classes.
+
+ This call takes a number of device adapter specific arguments and these are detailed
+ in the pydoc headers for the managers _\_\_init___() method.
+
+2. Create the ProtoBuf message for your metrics by calling the newly created _manager's_
+ **_make_proto_**() method.
+
+3. Register the ProtoBuf message configuration with the adapter agent via the
+ _update_device_pm_config_() method with the optional init parameter set to **True**.
+
+4. Request the manager to schedule the first PM collection interval by calling the
+ manager's _start_collector_() method. You may wish to do this after a short pause
+ depending on how your adapter is designed.
+
+The next two subsections provides examples of these steps for both an OLT and an ONU
+device adapter
+
+### OLT Device Adapters PM Manager setup
+
+```python
+ # Create the OLT PM Manager object
+ kwargs = {
+ 'nni-ports': self.northbound_ports.values(),
+ 'pon-ports': self.southbound_ports.values()
+ }
+ self.pm_metrics = OltPmMetrics(self.adapter_agent, self.device_id,
+ grouped=True, freq_override=False,
+ **kwargs)
+
+ # Create the protobuf message configuration
+ pm_config = self.pm_metrics.make_proto()
+ self.log.debug("initial-pm-config", pm_config=pm_config)
+
+ # Create the PM information in the adapter agent
+ self.adapter_agent.update_device_pm_config(pm_config, init=True)
+
+ # Start collecting stats from the device after a brief pause
+ reactor.callLater(10, self.pm_metrics.start_collector)
+```
+
+### ONU Device Adapters PM Manager setup
+
+For ONU devices, if you wish to include OpenOMCI 15-minute historical interval
+intervals, you will need to register the PM Metrics OpenOMCI Interval PM class
+with OpenOMCI
+
+```python
+
+ # Create the OLT PM Manager object
+ kwargs = {
+ 'heartbeat': self.heartbeat,
+ 'omci-cc': self.openomci.omci_cc
+ }
+ self.pm_metrics = OnuPmMetrics(self.adapter_agent, self.device_id,
+ grouped=True, freq_override=False,
+ **kwargs)
+
+ # Create the protobuf message configuration
+ pm_config = self.pm_metrics.make_proto()
+
+ # Register the OMCI history intervals with OpenOMCI
+ self.openomci.set_pm_config(self.pm_metrics.omci_pm.openomci_interval_pm)
+
+ # Create the PM information in the adapter agent
+ self.adapter_agent.update_device_pm_config(pm_config, init=True)
+
+ # Start collecting stats from the device after a brief pause
+ reactor.callLater(30, self.pm_metrics.start_collector)
+```
+
+# Basic PKI Format
+
+**TODO**: This needs to be defined by the community with assistance from the _SEBA_
+developers.
+
+The PKI information is published on the kafka bus under the _voltha.kpi_ topic. For
+VOLTHA PM information, the kafka key is empty and the value is a JSON message composed
+of the following key-value pairs.
+
+| key | value | Notes |
+| :-: | :----- | :---- |
+| type | string | "slice" or "ts". A "slice" is a set of path/metric data for the same time-stamp. A "ts" is a time-series: array of data for same metric |
+| ts | float | UTC time-stamp of data in slice mode (seconds since the epoch of January 1, 1970) |
+| prefixes | map | One or more prefix_name - value pairs as described below |
+
+**NOTE**: The timestamp is currently retrieved as a whole value. It is also possible to easily get
+the floating timestamp which contains the fractional seconds since epoch. **Is this of use**?
+
+For group PM information, the key composed of a string with the following format:
+```
+ voltha.<device-adapter>.<device-id>.<group>[.<group-id>]
+```
+Here is an JSON **example** of a current PKI published on the kafka bus under the
+_voltha.pki_ topic. In this case, the _device-adapter_ is the **adtran_olt**, the _device-id_ is
+the value **0001c4397d43bc51**, the _group_ is **nni** port statistics, and the _group-id_ is the
+port number is **1**.
+
+```json
+{
+ "type": "slice",
+ "ts": 1532379520.0,
+ "prefixes": {
+ "voltha.adtran_olt.0001c4397d43bc51.nni.1": {
+ "metrics": {
+ "tx_dropped": 0.0,
+ "rx_packets": 0.0,
+ "rx_bytes": 0.0,
+ "rx_mcast": 0.0,
+ "tx_mcast": 16.0,
+ "rx_bcast": 0.0,
+ "oper_status": 4.0,
+ "admin_state": 3.0,
+ "tx_bcast": 5639.0,
+ "tx_bytes": 1997642.0,
+ "rx_dropped": 0.0,
+ "tx_packets": 5655.0,
+ "port_no": 1.0,
+ "rx_errors": 0.0
+ }
+ },
+ "voltha.adtran_olt.0001c4397d43bc51.pon.0.onu.0": {
+ "metrics": {
+ "fiber_length": 29.0,
+ "onu_id": 0.0,
+ "pon_id": 0.0,
+ "equalization_delay": 621376.0,
+ "rssi": -167.0
+ }
+ },
+ "voltha.adtran_olt.0001c4397d43bc51.pon.0.onu.1": {
+ "metrics": {
+ "fiber_length": 29.0,
+ "onu_id": 1.0,
+ "pon_id": 0.0,
+ "equalization_delay": 621392.0,
+ "rssi": -164.0
+ },
+ ...
+
+ "voltha.adtran_olt.0001c4397d43bc51.pon.0.onu.0.gem.2176": {
+ "metrics": {
+ "rx_packets": 0.0,
+ "rx_bytes": 0.0,
+ "alloc_id": 1024.0,
+ "gem_id": 2176.0,
+ "pon_id": 0.0,
+ "tx_bytes": 0.0,
+ "onu_id": 0.0,
+ "tx_packets": 0.0
+ }
+ },
+ ...
+ }
+}
+
+```
+
+For OpenOMCI historical intervals, the name is derived from the Managed Entity class:
+
+```json
+{
+ "type": "slice",
+ "ts": 1532372864.0,
+ "prefixes": {
+ "voltha.adtran_onu.0001b8c505090b5b.EthernetFrameExtendedPerformanceMonitoring": {
+ "metrics": {
+ "entity_id": 2.0,
+ "class_id": 334.0,
+ "packets": 0.0,
+ "octets": 0.0,
+ "interval_end_time": 0.0,
+ "crc_errored_packets": 0.0,
+ "broadcast_packets": 0.0,
+ "64_octets": 0.0,
+ "65_to_127_octets": 0.0,
+ "128_to_255_octets": 0.0,
+ "256_to_511_octets": 0.0,
+ "undersize_packets": 0.0,
+ "drop_events": 0.0,
+ "multicast_packets": 0.0,
+ "oversize_packets": 0.0
+ }
+ }
+ }
+}
+```
+More information on the OpenOMCI ONU Historical Intervals is detailed in the _IntervalMetrics.md_
+file in the _onu/_ subdirectory.
+
+# Remaining Work Items
+
+This initial code is only a preliminary sample. The following tasks need to be
+added to the VOLTHA JIRA or performed in the SEBA group:
+
+- Get a list from SEBA/VOLTHA on required metrics.
+
+- Provide example JSON output and verify that it meets SEBA's requirements
+
+- Get feedback from other OLT/ONU developers on any needed changes
+
+- Test PM group enable/disable
+
+- Allow PM groups to have different collection times
+
+- Solicit VOLTHA/SEBA if support for disabling of individual items in a PM group would be useful
+
+- Support calling a 'get-data' method before collect the metrics. Currently metrics are collected
+ in a device adapter independent way and the PM just updates what the attributes happen to have.
+
+- TODO: Probably a few more. Look through code for more 'TODO' Notes
+
diff --git a/voltha/extensions/pki/__init__.py b/voltha/extensions/pki/__init__.py
new file mode 100644
index 0000000..b0fb0b2
--- /dev/null
+++ b/voltha/extensions/pki/__init__.py
@@ -0,0 +1,13 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
diff --git a/voltha/extensions/pki/adapter_pm_metrics.py b/voltha/extensions/pki/adapter_pm_metrics.py
new file mode 100644
index 0000000..e5b2c96
--- /dev/null
+++ b/voltha/extensions/pki/adapter_pm_metrics.py
@@ -0,0 +1,137 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import structlog
+from twisted.internet.task import LoopingCall
+
+
+class AdapterPmMetrics(object):
+ """
+ Base class for Device Adapter PM Metrics Manager
+
+ Device specific (OLT, ONU, OpenOMCI, ...) will derive groups of PM information
+ and this base class is primarily used to provide a consistent interface to configure,
+ start, and stop statistics collection.
+ """
+ def __init__(self, adapter_agent, device_id,
+ grouped=False, freq_override=False, **kwargs):
+ """
+ Initializer for shared Device Adapter PM metrics manager
+
+ :param adapter_agent: (AdapterAgent) Adapter agent for the device
+ :param device_id: (str) Device ID
+ :param grouped: (bool) Flag indicating if statistics are managed as a group
+ :param freq_override: (bool) Flag indicating if frequency collection can be specified
+ on a per group basis
+ :param kwargs: (dict) Device Adapter specific values
+ """
+ self.log = structlog.get_logger(device_id=device_id)
+ self.device_id = device_id
+ self.adapter_agent = adapter_agent
+ self.name = adapter_agent.adapter_name
+ self.default_freq = 150
+ self.grouped = grouped
+ self.freq_override = grouped and freq_override
+ self.lc = None
+ self.prefix = 'voltha.{}.{}'.format(self.name, self.device_id)
+
+ def update(self, pm_config):
+ # TODO: Move any common steps into base class
+ raise NotImplementedError('Your derived class should override this method')
+
+ def make_proto(self, pm_config=None):
+ raise NotImplementedError('Your derived class should override this method')
+
+ def start_collector(self, callback=None):
+ """
+ Start the collection loop for an adapter if the frequency > 0
+
+ :param callback: (callable) Function to call to collect PM data
+ """
+ self.log.info("starting-pm-collection", device_name=self.name)
+ if callback is None:
+ callback = self.collect_and_publish_metrics
+
+ if self.lc is None:
+ self.lc = LoopingCall(callback)
+
+ if self.default_freq > 0:
+ self.lc.start(interval=self.default_freq / 10)
+
+ def stop_collector(self):
+ """ Stop the collection loop"""
+ if self.lc is not None and self.default_freq > 0:
+ self.lc.stop()
+
+ def collect_metrics(self, group, names, config):
+ """
+ Collect the metrics for a specific PM group.
+
+ This common collection method expects that the group object provide as the first
+ parameter supports an attribute or property with the name of the value to
+ retrieve.
+
+ :param group: (object) The object to query for the value of various attributes (PM names)
+ :param names: (set) A collection of PM names that, if implemented as a property in the object,
+ will return a value to store in the returned PM dictionary
+ :param config: (PMConfig) PM Configuration settings. The enabled flag is examined to determine
+ if the data associated with a PM Name will be collected.
+
+ :return: (dict) collected metrics
+ """
+ metrics = dict()
+
+ for (metric, t) in names:
+ if config[metric].enabled and hasattr(group, metric):
+ metrics[metric] = getattr(group, metric)
+ return metrics
+
+ def collect_group_metrics(self, metrics=None):
+ raise NotImplementedError('Your derived class should override this method')
+
+ def collect_and_publish_metrics(self):
+ try:
+ metrics = self.collect_group_metrics()
+ self.publish_metrics(metrics)
+
+ except Exception as e:
+ self.log.exception('failed-to-collect-kpis', e=e)
+
+ def publish_metrics(self, metrics):
+ """
+ Publish the metrics during a collection
+
+ :param metrics: (dict) Metrics to publish
+ """
+ import arrow
+ from voltha.protos.events_pb2 import KpiEvent, KpiEventType, MetricValuePairs
+
+ try:
+ ts = arrow.utcnow().timestamp
+ kpi_event = KpiEvent(
+ type=KpiEventType.slice,
+ ts=ts,
+ prefixes={
+ self.prefix + '.{}'.format(k): MetricValuePairs(metrics=metrics[k])
+ for k in metrics.keys()}
+ )
+ self.adapter_agent.submit_kpis(kpi_event)
+
+ except Exception as e:
+ self.log.exception('failed-to-submit-kpis', e=e)
+
+ # TODO: Need to support on-demand counter update if provided by the PM 'group'.
+ # Currently we expect PM data to be periodically polled by a separate
+ # mechanism. The on-demand counter update should be optional in case the
+ # particular device adapter group of data is polled anyway for other reasons.
diff --git a/voltha/extensions/pki/olt/__init__.py b/voltha/extensions/pki/olt/__init__.py
new file mode 100644
index 0000000..b0fb0b2
--- /dev/null
+++ b/voltha/extensions/pki/olt/__init__.py
@@ -0,0 +1,13 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
diff --git a/voltha/extensions/pki/olt/olt_pm_metrics.py b/voltha/extensions/pki/olt/olt_pm_metrics.py
new file mode 100644
index 0000000..04e403f
--- /dev/null
+++ b/voltha/extensions/pki/olt/olt_pm_metrics.py
@@ -0,0 +1,284 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from voltha.protos.device_pb2 import PmConfig, PmConfigs, PmGroupConfig
+from voltha.extensions.pki.adapter_pm_metrics import AdapterPmMetrics
+
+
+class OltPmMetrics(AdapterPmMetrics):
+ """
+ Shared OL Device Adapter PM Metrics Manager
+
+ This class specifically addresses ONU genernal PM (health, ...) area
+ specific PM (OMCI, PON, UNI) is supported in encapsulated classes accessible
+ from this object
+ """
+ def __init__(self, adapter_agent, device_id, grouped=False, freq_override=False,
+ **kwargs):
+ """
+ Initializer for shared ONU Device Adapter PM metrics
+
+ :param adapter_agent: (AdapterAgent) Adapter agent for the device
+ :param device_id: (str) Device ID
+ :param grouped: (bool) Flag indicating if statistics are managed as a group
+ :param freq_override: (bool) Flag indicating if frequency collection can be specified
+ on a per group basis
+ :param kwargs: (dict) Device Adapter specific values. For an ONU Device adapter, the
+ expected key-value pairs are listed below. If not provided, the
+ associated PM statistics are not gathered:
+
+ 'nni-ports': List of objects that provide NNI (northbound) port statistics
+ 'pon-ports': List of objects that provide PON port statistics
+ """
+ super(OltPmMetrics, self).__init__(adapter_agent, device_id,
+ grouped=grouped, freq_override=freq_override,
+ **kwargs)
+
+ # PM Config Types are COUNTER, GUAGE, and STATE # GAUGE is misspelled device.proto
+ self.nni_pm_names = {
+ ('admin_state', PmConfig.STATE),
+ ('oper_status', PmConfig.STATE),
+ ('port_no', PmConfig.GUAGE), # Device and logical_device port numbers same
+ ('rx_packets', PmConfig.COUNTER),
+ ('rx_bytes', PmConfig.COUNTER),
+ ('rx_dropped', PmConfig.COUNTER),
+ ('rx_errors', PmConfig.COUNTER),
+ ('rx_bcast', PmConfig.COUNTER),
+ ('rx_mcast', PmConfig.COUNTER),
+ ('tx_packets', PmConfig.COUNTER),
+ ('tx_bytes', PmConfig.COUNTER),
+ ('tx_dropped', PmConfig.COUNTER),
+ ('tx_bcast', PmConfig.COUNTER),
+ ('tx_mcast', PmConfig.COUNTER),
+ #
+ # Commented out are from spec. May not be supported or implemented yet
+ # ('rx_64', PmConfig.COUNTER),
+ # ('rx_65_127', PmConfig.COUNTER),
+ # ('rx_128_255', PmConfig.COUNTER),
+ # ('rx_256_511', PmConfig.COUNTER),
+ # ('rx_512_1023', PmConfig.COUNTER),
+ # ('rx_1024_1518', PmConfig.COUNTER),
+ # ('rx_frame_err', PmConfig.COUNTER),
+ # ('rx_over_err', PmConfig.COUNTER),
+ # ('rx_crc_err', PmConfig.COUNTER),
+ # ('rx_64', PmConfig.COUNTER),
+ # ('tx_65_127', PmConfig.COUNTER),
+ # ('tx_128_255', PmConfig.COUNTER),
+ # ('tx_256_511', PmConfig.COUNTER),
+ # ('tx_512_1023', PmConfig.COUNTER),
+ # ('tx_1024_1518', PmConfig.COUNTER),
+ # ('collisions', PmConfig.COUNTER),
+ }
+ self.pon_pm_names = {
+ ('admin_state', PmConfig.STATE),
+ ('oper_status', PmConfig.STATE),
+ ('port_no', PmConfig.GUAGE), # Physical device port number
+ ('pon_id', PmConfig.GUAGE),
+ ('rx_packets', PmConfig.COUNTER),
+ ('rx_bytes', PmConfig.COUNTER),
+ ('tx_packets', PmConfig.COUNTER),
+ ('tx_bytes', PmConfig.COUNTER),
+ ('tx_bip_errors', PmConfig.COUNTER),
+ ('in_service_onus', PmConfig.GUAGE),
+ ('closest_onu_distance', PmConfig.GUAGE)
+ }
+ self.onu_pm_names = {
+ ('pon_id', PmConfig.GUAGE),
+ ('onu_id', PmConfig.GUAGE),
+ ('fiber_length', PmConfig.GUAGE),
+ ('equalization_delay', PmConfig.GUAGE),
+ ('rssi', PmConfig.GUAGE), #
+ }
+ self.gem_pm_names = {
+ ('pon_id', PmConfig.GUAGE),
+ ('onu_id', PmConfig.GUAGE),
+ ('gem_id', PmConfig.GUAGE),
+ ('alloc_id', PmConfig.GUAGE),
+ ('rx_packets', PmConfig.COUNTER),
+ ('rx_bytes', PmConfig.COUNTER),
+ ('tx_packets', PmConfig.COUNTER),
+ ('tx_bytes', PmConfig.COUNTER),
+ }
+ self.nni_metrics_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in self.nni_pm_names}
+ self.pon_metrics_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in self.pon_pm_names}
+ self.onu_metrics_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in self.onu_pm_names}
+ self.gem_metrics_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in self.gem_pm_names}
+
+ self._nni_ports = kwargs.pop('nni-ports', None)
+ self._pon_ports = kwargs.pop('pon-ports', None)
+
+ def update(self, pm_config):
+ # TODO: Test both 'group' and 'non-group' functionality
+ # TODO: Test frequency override capability for a particular group
+ if self.default_freq != pm_config.default_freq:
+ # Update the callback to the new frequency.
+ self.default_freq = pm_config.default_freq
+ self.lc.stop()
+ self.lc.start(interval=self.default_freq / 10)
+
+ if pm_config.grouped:
+ for m in pm_config.groups:
+ # TODO: Need to support individual group enable/disable
+ pass
+ # self.pm_group_metrics[m.group_name].config.enabled = m.enabled
+ # if m.enabled is True:
+ # self.enable_pm_collection(m.group_name, remote)
+ # else:
+ # self.disable_pm_collection(m.group_name, remote)
+ else:
+ for m in pm_config.metrics:
+ self.nni_metrics_config[m.name].enabled = m.enabled
+ self.pon_metrics_config[m.name].enabled = m.enabled
+ self.onu_metrics_config[m.name].enabled = m.enabled
+ self.gem_metrics_config[m.name].enabled = m.enabled
+
+ def make_proto(self, pm_config=None):
+ if pm_config is None:
+ pm_config = PmConfigs(id=self.device_id, default_freq=self.default_freq,
+ grouped=self.grouped,
+ freq_override=self.freq_override)
+ metrics = set()
+ have_nni = self._nni_ports is not None and len(self._nni_ports) > 0
+ have_pon = self._pon_ports is not None and len(self._pon_ports) > 0
+
+ if self.grouped:
+ if have_nni:
+ pm_ether_stats = PmGroupConfig(group_name='Ethernet',
+ group_freq=self.default_freq,
+ enabled=True)
+ else:
+ pm_ether_stats = None
+
+ if have_pon:
+ pm_pon_stats = PmGroupConfig(group_name='PON',
+ group_freq=self.default_freq,
+ enabled=True)
+
+ pm_ont_stats = PmGroupConfig(group_name='ONT',
+ group_freq=self.default_freq,
+ enabled=True)
+
+ pm_gem_stats = PmGroupConfig(group_name='GEM',
+ group_freq=self.default_freq,
+ enabled=True)
+ else:
+ pm_pon_stats = None
+ pm_ont_stats = None
+ pm_gem_stats = None
+
+ else:
+ pm_ether_stats = pm_config if have_nni else None
+ pm_pon_stats = pm_config if have_pon else None
+ pm_ont_stats = pm_config if have_pon else None
+ pm_gem_stats = pm_config if have_pon else None
+
+ if have_nni:
+ for m in sorted(self.nni_metrics_config):
+ pm = self.nni_metrics_config[m]
+ if not self.grouped:
+ if pm.name in metrics:
+ continue
+ metrics.add(pm.name)
+ pm_ether_stats.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+ if have_pon:
+ for m in sorted(self.pon_metrics_config):
+ pm = self.pon_metrics_config[m]
+ if not self.grouped:
+ if pm.name in metrics:
+ continue
+ metrics.add(pm.name)
+ pm_pon_stats.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+
+ for m in sorted(self.onu_metrics_config):
+ pm = self.onu_metrics_config[m]
+ if not self.grouped:
+ if pm.name in metrics:
+ continue
+ metrics.add(pm.name)
+ pm_ont_stats.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+
+ for m in sorted(self.gem_metrics_config):
+ pm = self.gem_metrics_config[m]
+ if not self.grouped:
+ if pm.name in metrics:
+ continue
+ metrics.add(pm.name)
+ pm_gem_stats.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+ if self.grouped:
+ pm_groups = [stats for stats in (pm_ether_stats,
+ pm_pon_stats,
+ pm_ont_stats,
+ pm_gem_stats) if stats is not None]
+ pm_config.groups.extend(pm_groups)
+
+ return pm_config
+
+ def collect_group_metrics(self, metrics=None):
+ # TODO: Currently PM collection is done for all metrics/groups on a single timer
+ if metrics is None:
+ metrics = dict()
+
+ for port in self._nni_ports:
+ metrics['nni.{}'.format(port.port_no)] = self.collect_nni_metrics(port)
+
+ for port in self._pon_ports:
+ metrics['pon.{}'.format(port.pon_id)] = self.collect_pon_metrics(port)
+
+ for onu_id in port.onu_ids:
+ onu = port.onu(onu_id)
+ if onu is not None:
+ metrics['pon.{}.onu.{}'.format(port.pon_id, onu.onu_id)] = \
+ self.collect_onu_metrics(onu)
+ for gem in onu.gem_ports:
+ if gem.multicast:
+ continue
+
+ metrics['pon.{}.onu.{}.gem.{}'.format(port.pon_id,
+ onu.onu_id,
+ gem.gem_id)] = \
+ self.collect_gem_metrics(gem)
+ # TODO: Do any multicast GEM PORT metrics here...
+ return metrics
+
+ def collect_nni_metrics(self, nni_port):
+ stats = {metric: getattr(nni_port, metric) for (metric, t) in self.nni_pm_names}
+ return {metric: value for metric, value in stats.iteritems()
+ if self.nni_metrics_config[metric].enabled}
+
+ def collect_pon_metrics(self, pon_port):
+ stats = {metric: getattr(pon_port, metric) for (metric, t) in self.pon_pm_names}
+ return {metric: value for metric, value in stats.iteritems()
+ if self.pon_metrics_config[metric].enabled}
+
+ def collect_onu_metrics(self, onu):
+ stats = {metric: getattr(onu, metric) for (metric, t) in self.onu_pm_names}
+ return {metric: value for metric, value in stats.iteritems()
+ if self.onu_metrics_config[metric].enabled}
+
+ def collect_gem_metrics(self, gem):
+ stats = {metric: getattr(gem, metric) for (metric, t) in self.gem_pm_names}
+ return {metric: value for metric, value in stats.iteritems()
+ if self.gem_metrics_config[metric].enabled}
diff --git a/voltha/extensions/pki/onu/IntervalMetrics.md b/voltha/extensions/pki/onu/IntervalMetrics.md
new file mode 100644
index 0000000..1f5bc53
--- /dev/null
+++ b/voltha/extensions/pki/onu/IntervalMetrics.md
@@ -0,0 +1,264 @@
+# ONU OMCI Historical Interval PM Groups
+
+This document outlines the 15-minute interval groups currently supported by the
+**OnuPmIntervalMetrics** _onu_pm_interval_metrics.py_ file. These groups
+cover a 15-minute interval.
+
+## Performance Interval State Machine
+
+At OpenOMCI startup within an ONU Device Adapter, as soon as the OpenOMCI ME database has
+been declared to be in-sync ONU's ME Database, the Performance Interval State Machine is
+started for the ONU. The first task it performs is to synchronize the ONU's (hardware) time with
+the ONU Device Handler's (Linux container) so that a 15-minute interval is established.
+
+The OpenOMCI PM interval state machine then examines managed elements created by the
+ONU autonomously or created by OpenOMCI in response to a OMCI request from an ONU
+adapter to determine if an appropriate 15-Minute historical PM ME needs to be attached. The
+state machine also registers for notification of any create/delete requests at that
+point so that it can add/remove 15-minute historical PM MEs as services are applied or
+removed.
+
+Before adding a 15-minute historical PM ME, the ME capabilities of the ONU is
+examined to insure that it can support that particular ME. This is important as the
+Ethernet Frame historical intervals are actually supported by up to 4 different MEs
+reporting the basically the same data. This is detailed below in the _Ethernet Frame
+Performance Monitoring MEs_ section.
+
+## Timezone
+
+The ONU will be synchronized to the Linux Container running the ONU Device handler's
+time in UTC upon startup. Not all ONUs have the capability to set their calendar
+date (month, day, year) to that of the ONU's Device Handler, but it will set the
+clock to that date. For reporting of 15-minute intervals, only an accurate 15-minute
+boundary is really of any great importance.
+
+## Interval Reporting
+
+After the ONU time synchronization request is made, the first reported interval is
+schedule to occur in the next 15-minute boundry. For example, if the OpenOMCI
+state machine synchronizes the ONU's time at
+
+## Common Elements for All Reported MEs
+
+In addition to counter elements (attributes) reported in each ME, every reported
+historical interval report the following Elements. All widths are reported below
+in bytes.
+
+| Label | Width | Description |
+| ----------------: | :---: | :---------- |
+| class_id | 2 | The ME Class ID of the PM Interval ME |
+| entity_id | 2 | The OMCI Entity Instance of the particular PM Interval ME |
+| interval_end_time | 2 | Identifies the most recently finished 15 minute. This attribute is set to zero when a synchronize time request is performed by OpenOMCI. This counter rolls over from 255 to 0 upon saturation.
+
+
+## Ethernet Frame Performance Monitoring MEs
+
+The OMCI Ethernet PM supported by OpenOMCI includes 4 possible MEs. These MEs are attached to
+the MAC Bridge Port Configuration MEs for the ONU. For downstream data, the ME is placed on the
+MAC Bridge Port Configuration ME closest to the ANI Port. For upstream data, the ME is placed
+on the MAC Bridge Port Configuration ME closest to the associated UNI.
+
+The OpenOMCI will first attempt to use the Extended Performance Monitoring MEs if they are
+supported by the ONU. First the 64-bit counter version will be attempted and then the 32-bit
+counters as a fallback. If of the Extended Performance Monitoring MEs are supported, the
+appropriate Upstream or DownStream Monitoring ME will be used.
+
+### ME Information
+
+The table below describes the four Ethernet Frame Performance Monitoring MEs and provides their
+counter width (in bytes) and ME Class ID.
+
+| ME Name | Class ID | Width |
+| ----------------------------------------------------------: | :------: | :---: |
+| Ethernet Frame Extended Performance Monitoring64Bit | 426 | 8 |
+| Ethernet Frame Extended Performance Monitoring | 334 | 8 |
+| Ethernet Frame Upstream Performance MonitoringHistoryData | 322 | 8 |
+| Ethernet Frame Downstream Performance MonitoringHistoryData | 321 | 8 |
+
+### Counter Information
+
+Each of the Ethernet Frame PM MEs contain the following counters
+
+| Attribute Name | Description |
+| ------------------: | :-----------|
+| drop_events | The total number of events in which packets were dropped due to a lack of resources. This is not necessarily the number of packets dropped; it is the number of times this event was detected. |
+| octets | The total number of upstream octets received, including those in bad packets, excluding framing bits, but including FCS. |
+| packets | The total number of upstream packets received, including bad packets, broadcast packets and multicast packets. |
+| broadcast_packets | The total number of upstream good packets received that were directed to the broadcast address. This does not include multicast packets. |
+| multicast_packets | The total number of upstream good packets received that were directed to a multicast address. This does not include broadcast packets. |
+| crc_errored_packets | The total number of upstream packets received that had a length (excluding framing bits, but including FCS octets) of between 64 octets and 1518 octets, inclusive, but had either a bad FCS with an integral number of octets (FCS error) or a bad FCS with a non-integral number of octets (alignment error). |
+| undersize_packets | The total number of upstream packets received that were less than 64 octets long, but were otherwise well formed (excluding framing bits, but including FCS). |
+| oversize_packets | The total number of upstream packets received that were longer than 1518 octets (excluding framing bits, but including FCS) and were otherwise well formed. NOTE 2 – If 2 000 byte Ethernet frames are supported, counts in this performance parameter are not necessarily errors. |
+| 64_octets | The total number of upstream received packets (including bad packets) that were 64 octets long, excluding framing bits but including FCS. |
+| 65_to_127_octets | The total number of upstream received packets (including bad packets) that were 65..127 octets long, excluding framing bits but including FCS. |
+| 128_to_255_octets | The total number of upstream packets (including bad packets) received that were 128..255 octets long, excluding framing bits but including FCS. |
+| 256_to_511_octets | The total number of upstream packets (including bad packets) received that were 256..511 octets long, excluding framing bits but including FCS. |
+| 512_to_1023_octets | The total number of upstream packets (including bad packets) received that were 512..1 023 octets long, excluding framing bits but including FCS. |
+| 1024_to_1518_octets | The total number of upstream packets (including bad packets) received that were 1024..1518 octets long, excluding framing bits, but including FCS. |
+
+## Ethernet PM Monitoring History Data (Class ID 24)
+
+This managed entity collects some of the performance monitoring data for a physical
+Ethernet interface. Instances of this managed entity are created and deleted by the OLT.
+
+### Application
+
+For performance monitoring of Ethernet UNI.
+
+### Relationships
+
+An instance of this managed entity is associated with an instance of the physical path
+termination point Ethernet UNI.
+
+### Attributes
+All counters are 2 bytes wide.
+
+| Attribute Name | Description |
+| ------------------: | :-----------|
+| fcs_errors | This attribute counts frames received on a particular interface that were an integral number of octets in length but failed the frame check sequence (FCS) check. The count is incremented when the MAC service returns the frameCheckError status to the link layer control (LLC) or other MAC user. Received frames for which multiple error conditions are obtained are counted according to the error status presented to the LLC. |
+| excessive_collision_counter | This attribute counts frames whose transmission failed due to excessive collisions. |
+| late_collision_counter | This attribute counts the number of times that a collision was detected later than 512 bit times into the transmission of a packet. |
+| frames_too_long | This attribute counts received frames that exceeded the maximum permitted frame size. The count is incremented when the MAC service returns the frameTooLong status to the LLC. |
+| buffer_overflows_on_rx | This attribute counts the number of times that the receive buffer overflowed. |
+| buffer_overflows_on_tx | This attribute counts the number of times that the transmit buffer overflowed. |
+| single_collision_frame_counter | This attribute counts successfully transmitted frames whose transmission was delayed by exactly one collision. |
+| multiple_collisions_frame_counter | This attribute counts successfully transmitted frames whose transmission was delayed by more than one collision. |
+| sqe_counter | This attribute counts the number of times that the SQE test error message was generated by the PLS sublayer. |
+| deferred_tx_counter | This attribute counts frames whose first transmission attempt was delayed because the medium was busy. The count does not include frames involved in collisions. |
+| internal_mac_tx_error_counter | This attribute counts frames whose transmission failed due to an internal MAC sublayer transmit error. |
+| carrier_sense_error_counter | This attribute counts the number of times that carrier sense was lost or never asserted when attempting to transmit a frame. |
+| alignment_error_counter | This attribute counts received frames that were not an integral number of octets in length and did not pass the FCS check. |
+| internal_mac_rx_error_counter | This attribute counts frames whose reception failed due to an internal MAC sublayer receive error. |
+
+## FEC Performance Monitoring History Data (Class ID 312)
+
+This managed entity collects performance monitoring data associated with PON downstream FEC
+counters. Instances of this managed entity are created and deleted by the OLT.
+
+### Application
+This managed entity collects performance monitoring data associated with PON downstream FEC
+counters.
+
+### Relationships
+An instance of this managed entity is associated with an instance of the ANI-G managed entity.
+
+### Attributes
+
+| Attribute Name | Width | Description |
+| -----------------------: | :---: | :-----------|
+| corrected_bytes | 4 | This attribute counts the number of bytes that were corrected by the FEC function. |
+| corrected_code_words | 4 | This attribute counts the code words that were corrected by the FEC function. |
+| uncorrectable_code_words | 4 | This attribute counts errored code words that could not be corrected by the FEC function. |
+| total_code_words | 4 | This attribute counts the total received code words. |
+| fec_seconds | 2 | This attribute counts seconds during which there was a forward error correction anomaly. |
+
+
+## GEM Port Network CTP Monitoring History Data (Class ID 341)
+
+This managed entity collects GEM frame performance monitoring data associated with a GEM port
+network CTP. Instances of this managed entity are created and deleted by the OLT.
+
+Note 1: One might expect to find some form of impaired or discarded frame count associated with
+a GEM port. However, the only impairment that might be detected at the GEM frame level would be
+a corrupted GEM frame header. In this case, no part of the header could be considered reliable
+including the port ID. For this reason, there is no impaired or discarded frame count in this ME.
+
+Note 2: This managed entity replaces the GEM port performance history data managed entity and
+is preferred for new implementations.
+
+### Relationships
+
+An instance of this managed entity is associated with an instance of the GEM port network CTP
+managed entity.
+
+### Attributes
+
+| Attribute Name | Width | Description |
+| ------------------------: | :---: | :-----------|
+| transmitted_gem_frames | 4 | This attribute counts GEM frames transmitted on the monitored GEM port. |
+| received_gem_frames | 4 | This attribute counts GEM frames received correctly on the monitored GEM port. A correctly received GEM frame is one that does not contain uncorrectable errors and has a valid HEC. |
+| received_payload_bytes | 8 | This attribute counts user payload bytes received on the monitored GEM port. |
+| transmitted_payload_bytes | 8 | This attribute counts user payload bytes transmitted on the monitored GEM port. |
+| encryption_key_errors | 4 | This attribute is defined in ITU-T G.987 systems only. It counts GEM frames with erroneous encryption key indexes. If the GEM port is not encrypted, this attribute counts any frame with a key index not equal to 0. If the GEM port is encrypted, this attribute counts any frame whose key index specifies a key that is not known to the ONU. |
+
+Note 3: GEM PM ignores idle GEM frames.
+
+Note 4: GEM PM counts each non-idle GEM frame, whether it contains an entire user frame or only
+a fragment of a user frame.
+
+## XgPon TC Performance Monitoring History Data (Class ID 344)
+
+This managed entity collects performance monitoring data associated with the XG-PON
+transmission convergence layer, as defined in ITU-T G.987.3.
+
+### Relationships
+An instance of this managed entity is associated with an ANI-G.
+
+### Attributes
+
+All counters are 2 bytes wide.
+
+| Attribute Name | Description |
+| ------------------: | :-----------|
+| psbd_hec_error_count | This attribute counts HEC errors in any of the fields of the downstream physical sync block. |
+| xgtc_hec_error_count | This attribute counts HEC errors detected in the XGTC header. |
+| unknown_profile_count | This attribute counts the number of grants received whose specified profile was not known to the ONU. |
+| transmitted_xgem_frames | This attribute counts the number of non-idle XGEM frames transmitted. If an SDU is fragmented, each fragment is an XGEM frame and is counted as such. |
+| fragment_xgem_frames | This attribute counts the number of XGEM frames that represent fragmented SDUs, as indicated by the LF bit = 0. |
+| xgem_hec_lost_words_count | This attribute counts the number of four-byte words lost because of an XGEM frame HEC error. In general, all XGTC payload following the error is lost, until the next PSBd event. |
+| xgem_key_errors | This attribute counts the number of downstream XGEM frames received with an invalid key specification. The key may be invalid for several reasons. |
+| xgem_hec_error_count | This attribute counts the number of instances of an XGEM frame HEC error. |
+
+## XgPon Downstream Performance Monitoring History Data (Class ID 345)
+
+This managed entity collects performance monitoring data associated with the XG-PON
+transmission convergence layer, as defined in ITU-T G.987.3. It collects counters associated with
+downstream PLOAM and OMCI messages.
+
+### Relationships
+
+An instance of this managed entity is associated with an ANI-G.
+
+### Attributes
+
+All counters are 2 bytes wide.
+
+| Attribute Name | Description |
+| ------------------: | :-----------|
+| ploam_mic_error_count | This attribute counts MIC errors detected in downstream PLOAM messages, either directed to this ONU or broadcast to all ONUs. |
+| downstream_ploam_messages_count | This attribute counts PLOAM messages received, either directed to this ONU or broadcast to all ONUs. |
+| profile_messages_received | This attribute counts the number of profile messages received, either directed to this ONU or broadcast to all ONUs. |
+| ranging_time_messages_received | This attribute counts the number of ranging_time messages received, either directed to this ONU or broadcast to all ONUs. |
+| deactivate_onu_id_messages_received | This attribute counts the number of deactivate_ONU-ID messages received, either directed to this ONU or broadcast to all ONUs. Deactivate_ONU-ID messages do not reset this counter. |
+| disable_serial_number_messages_received | This attribute counts the number of disable_serial_number messages received, whose serial number specified this ONU. |
+| request_registration_messages_received | This attribute counts the number request_registration messages received. |
+| assign_alloc_id_messages_received | This attribute counts the number of assign_alloc-ID messages received. |
+| key_control_messages_received | This attribute counts the number of key_control messages received, either directed to this ONU or broadcast to all ONUs. |
+| sleep_allow_messages_received | This attribute counts the number of sleep_allow messages received, either directed to this ONU or broadcast to all ONUs. |
+| baseline_omci_messages_received_count | This attribute counts the number of OMCI messages received in the baseline message format. |
+| extended_omci_messages_received_count | This attribute counts the number of OMCI messages received in the extended message format. |
+| assign_onu_id_messages_received | This attribute counts the number of assign_ONU-ID messages received since the last re-boot. |
+| omci_mic_error_count | This attribute counts MIC errors detected in OMCI messages directed to this ONU. |
+
+## XgPon Upstream Performance Monitoring History Data (Class ID 346)
+
+This managed entity collects performance monitoring data associated with the XG-PON
+transmission convergence layer, as defined in ITU-T G.987.3. It counts upstream PLOAM
+messages transmitted by the ONU.
+
+###Relationships
+
+An instance of this managed entity is associated with an ANI-G.
+
+### Attributes
+
+All counters are 2 bytes wide.
+
+| Attribute Name | Description |
+| ------------------: | :-----------|
+| upstream_ploam_message_count | This attribute counts PLOAM messages transmitted upstream, excluding acknowledge messages. |
+| serial_number_onu_message_count | This attribute counts Serial_number_ONU PLOAM messages transmitted. |
+| registration_message_count | This attribute counts registration PLOAM messages transmitted. |
+| key_report_message_count | This attribute counts key_report PLOAM messages transmitted. |
+| acknowledge_message_count | This attribute counts acknowledge PLOAM messages transmitted. It includes all forms of acknowledgement, including those transmitted in response to a PLOAM grant when the ONU has nothing to send. |
+| sleep_request_message_count | This attribute counts sleep_request PLOAM messages transmitted. |
diff --git a/voltha/extensions/pki/onu/__init__.py b/voltha/extensions/pki/onu/__init__.py
new file mode 100644
index 0000000..b0fb0b2
--- /dev/null
+++ b/voltha/extensions/pki/onu/__init__.py
@@ -0,0 +1,13 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
diff --git a/voltha/extensions/pki/onu/onu_omci_pm.py b/voltha/extensions/pki/onu/onu_omci_pm.py
new file mode 100644
index 0000000..90a68a0
--- /dev/null
+++ b/voltha/extensions/pki/onu/onu_omci_pm.py
@@ -0,0 +1,129 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from voltha.protos.device_pb2 import PmConfig, PmConfigs, PmGroupConfig
+from voltha.extensions.pki.adapter_pm_metrics import AdapterPmMetrics
+from voltha.extensions.pki.onu.onu_pm_interval_metrics import OnuPmIntervalMetrics
+
+
+class OnuOmciPmMetrics(AdapterPmMetrics):
+ def __init__(self, adapter_agent, device_id,
+ grouped=False, freq_override=False, **kwargs):
+ """
+ Initializer for shared ONU Device Adapter OMCI CC PM metrics
+
+ :param adapter_agent: (AdapterAgent) Adapter agent for the device
+ :param device_id: (str) Device ID
+ :param grouped: (bool) Flag indicating if statistics are managed as a group
+ :param freq_override: (bool) Flag indicating if frequency collection can be specified
+ on a per group basis
+ :param kwargs: (dict) Device Adapter specific values. For an ONU Device adapter, the
+ expected key-value pairs are listed below. If not provided, the
+ associated PM statistics are not gathered:
+
+ 'omci-cc': Reference to the OMCI_CC object for retrieval of OpenOMCI
+ Communications channel statistics. Available from the ONU's
+ OpenOMCI OnuDeviceEntry object.
+ """
+ super(OnuOmciPmMetrics, self).__init__(adapter_agent, device_id,
+ grouped=grouped, freq_override=freq_override,
+ **kwargs)
+ self._omci_cc = kwargs.pop('omci-cc', None)
+
+ # PM Config Types are COUNTER, GUAGE, and STATE # Note: GAUGE is misspelled in device.proto
+ self.omci_pm_names = {
+ ('enabled', PmConfig.STATE),
+ ('tx_frames', PmConfig.COUNTER),
+ ('tx_errors', PmConfig.COUNTER),
+ ('rx_frames', PmConfig.COUNTER),
+ ('rx_unknown_tid', PmConfig.COUNTER),
+ ('rx_onu_frames', PmConfig.COUNTER), # Rx ONU autonomous messages
+ ('rx_alarm_overflow', PmConfig.COUNTER), # Autonomous ONU generated alarm message overflows
+ ('rx_avc_overflow', PmConfig.COUNTER), # Autonomous ONU generated AVC message overflows
+ ('rx_onu_discards', PmConfig.COUNTER), # Autonomous ONU message unknown type discards
+ ('rx_unknown_me', PmConfig.COUNTER), # Managed Entities without a decode definition
+ ('rx_timeouts', PmConfig.COUNTER),
+ ('consecutive_errors', PmConfig.COUNTER),
+ ('reply_min', PmConfig.GUAGE), # Milliseconds
+ ('reply_max', PmConfig.GUAGE), # Milliseconds
+ ('reply_average', PmConfig.GUAGE), # Milliseconds
+ }
+ self.omci_metrics_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in self.omci_pm_names}
+
+ self.openomci_interval_pm = OnuPmIntervalMetrics(adapter_agent, device_id)
+
+ def update(self, pm_config):
+ # TODO: Test both 'group' and 'non-group' functionality
+ # TODO: Test frequency override capability for a particular group
+ if self.default_freq != pm_config.default_freq:
+ # Update the callback to the new frequency.
+ self.default_freq = pm_config.default_freq
+ self.lc.stop()
+ self.lc.start(interval=self.default_freq / 10)
+
+ if pm_config.grouped:
+ for m in pm_config.groups:
+ # TODO: Need to support individual group enable/disable
+ pass
+ # self.pm_group_metrics[m.group_name].config.enabled = m.enabled
+ # if m.enabled is True:,
+ # self.enable_pm_collection(m.group_name, remote)
+ # else:
+ # self.disable_pm_collection(m.group_name, remote)
+ else:
+ for m in pm_config.metrics:
+ self.omci_metrics_config[m.name].enabled = m.enabled
+
+ self.openomci_interval_pm.update(pm_config)
+
+ def make_proto(self, pm_config=None):
+ assert pm_config is not None
+
+ if self._omci_cc is not None:
+ if self.grouped:
+ pm_omci_stats = PmGroupConfig(group_name='OMCI',
+ group_freq=self.default_freq,
+ enabled=True)
+ else:
+ pm_omci_stats = pm_config
+
+ metrics = set()
+ for m in sorted(self.omci_metrics_config):
+ pm = self.omci_metrics_config[m]
+ if not self.grouped:
+ if pm.name in metrics:
+ continue
+ metrics.add(pm.name)
+
+ pm_omci_stats.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+ if self.grouped:
+ pm_config.groups.extend([pm_omci_stats])
+
+ return self.openomci_interval_pm.make_proto(pm_config)
+
+ def collect_device_metrics(self, metrics=None):
+ # TODO: Currently PM collection is done for all metrics/groups on a single timer
+ if metrics is None:
+ metrics = dict()
+
+ # Note: Interval PM is collection done autonomously, not through this method
+
+ if self._omci_cc is not None:
+ metrics['omci-cc'] = self.collect_metrics(self._omci_cc,
+ self.omci_pm_names,
+ self.omci_metrics_config)
+ return metrics
diff --git a/voltha/extensions/pki/onu/onu_pm_interval_metrics.py b/voltha/extensions/pki/onu/onu_pm_interval_metrics.py
new file mode 100644
index 0000000..c34c6e3
--- /dev/null
+++ b/voltha/extensions/pki/onu/onu_pm_interval_metrics.py
@@ -0,0 +1,334 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from voltha.protos.device_pb2 import PmConfig, PmGroupConfig
+from voltha.extensions.pki.adapter_pm_metrics import AdapterPmMetrics
+from voltha.extensions.omci.omci_entities import \
+ EthernetFrameUpstreamPerformanceMonitoringHistoryData, \
+ EthernetFrameDownstreamPerformanceMonitoringHistoryData, \
+ EthernetFrameExtendedPerformanceMonitoring, \
+ EthernetFrameExtendedPerformanceMonitoring64Bit, \
+ EthernetPMMonitoringHistoryData, FecPerformanceMonitoringHistoryData, \
+ GemPortNetworkCtpMonitoringHistoryData, XgPonTcPerformanceMonitoringHistoryData, \
+ XgPonDownstreamPerformanceMonitoringHistoryData, \
+ XgPonUpstreamPerformanceMonitoringHistoryData
+
+
+class OnuPmIntervalMetrics(AdapterPmMetrics):
+ """
+ ONU OMCI PM Interval metrics
+
+ These differ from other PM Metrics as they are collected and generated as a
+ result of receipt of OMCI get responses on various PM History MEs. They are
+ also always managed as a group with a fixed frequency of 15 minutes.
+ """
+ def __init__(self, adapter_agent, device_id, **kwargs):
+ super(OnuPmIntervalMetrics, self).__init__(adapter_agent, device_id,
+ grouped=True, freq_override=False,
+ **kwargs)
+ ethernet_bridge_history = {
+ ('enabled', PmConfig.STATE),
+ ('class_id', PmConfig.GUAGE),
+ ('entity_id', PmConfig.GUAGE),
+ ("interval_end_time", PmConfig.GUAGE),
+ ("drop_events", PmConfig.COUNTER),
+ ("octets", PmConfig.COUNTER),
+ ("packets", PmConfig.COUNTER),
+ ("broadcast_packets", PmConfig.COUNTER),
+ ("multicast_packets", PmConfig.COUNTER),
+ ("crc_errored_packets", PmConfig.COUNTER),
+ ("undersize_packets", PmConfig.COUNTER),
+ ("oversize_packets", PmConfig.COUNTER),
+ ("64_octets", PmConfig.COUNTER),
+ ("65_to_127_octets", PmConfig.COUNTER),
+ ("128_to_255_octets", PmConfig.COUNTER),
+ ("256_to_511_octets", PmConfig.COUNTER),
+ ("512_to_1023_octets", PmConfig.COUNTER),
+ ("1024_to_1518_octets", PmConfig.COUNTER)
+ }
+ self._ethernet_bridge_history_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in ethernet_bridge_history}
+
+ ethernet_uni_history = { # Ethernet History Data (Class ID 24)
+ ('enabled', PmConfig.STATE),
+ ('class_id', PmConfig.GUAGE),
+ ('entity_id', PmConfig.GUAGE),
+ ("interval_end_time", PmConfig.GUAGE),
+ ("fcs_errors", PmConfig.COUNTER),
+ ("excessive_collision_counter", PmConfig.COUNTER),
+ ("late_collision_counter", PmConfig.COUNTER),
+ ("frames_too_long", PmConfig.COUNTER),
+ ("buffer_overflows_on_rx", PmConfig.COUNTER),
+ ("buffer_overflows_on_tx", PmConfig.COUNTER),
+ ("single_collision_frame_counter", PmConfig.COUNTER),
+ ("multiple_collisions_frame_counter", PmConfig.COUNTER),
+ ("sqe_counter", PmConfig.COUNTER),
+ ("deferred_tx_counter", PmConfig.COUNTER),
+ ("internal_mac_tx_error_counter", PmConfig.COUNTER),
+ ("carrier_sense_error_counter", PmConfig.COUNTER),
+ ("alignment_error_counter", PmConfig.COUNTER),
+ ("internal_mac_rx_error_counter", PmConfig.COUNTER),
+ }
+ self._ethernet_uni_history_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in ethernet_uni_history}
+
+ fec_history = { # FEC History Data (Class ID 312)
+ ('enabled', PmConfig.STATE),
+ ('class_id', PmConfig.GUAGE),
+ ('entity_id', PmConfig.GUAGE),
+ ("interval_end_time", PmConfig.GUAGE),
+ ("corrected_bytes", PmConfig.COUNTER),
+ ("corrected_code_words", PmConfig.COUNTER),
+ ("uncorrectable_code_words", PmConfig.COUNTER),
+ ("total_code_words", PmConfig.COUNTER),
+ ("fec_seconds", PmConfig.COUNTER),
+ }
+ self._fec_history_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in fec_history}
+
+ gem_port_history = { # GEM Port Network CTP History Data (Class ID 341)
+ ('enabled', PmConfig.STATE),
+ ('class_id', PmConfig.GUAGE),
+ ('entity_id', PmConfig.GUAGE),
+ ("interval_end_time", PmConfig.GUAGE),
+ ("transmitted_gem_frames", PmConfig.COUNTER),
+ ("received_gem_frames", PmConfig.COUNTER),
+ ("received_payload_bytes", PmConfig.COUNTER),
+ ("transmitted_payload_bytes", PmConfig.COUNTER),
+ ("encryption_key_errors", PmConfig.COUNTER),
+ }
+ self._gem_port_history_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in gem_port_history}
+
+ xgpon_tc_history = { # XgPon TC History Data (Class ID 344)
+ ('enabled', PmConfig.STATE),
+ ('class_id', PmConfig.GUAGE),
+ ('entity_id', PmConfig.GUAGE),
+ ("interval_end_time", PmConfig.GUAGE),
+ ("psbd_hec_error_count", PmConfig.COUNTER),
+ ("xgtc_hec_error_count", PmConfig.COUNTER),
+ ("unknown_profile_count", PmConfig.COUNTER),
+ ("transmitted_xgem_frames", PmConfig.COUNTER),
+ ("fragment_xgem_frames", PmConfig.COUNTER),
+ ("xgem_hec_lost_words_count", PmConfig.COUNTER),
+ ("xgem_key_errors", PmConfig.COUNTER),
+ ("xgem_hec_error_count", PmConfig.COUNTER),
+ }
+ self._xgpon_tc_history_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in xgpon_tc_history}
+
+ xgpon_downstream_history = { # XgPon Downstream History Data (Class ID 345)
+ ('enabled', PmConfig.STATE),
+ ('class_id', PmConfig.GUAGE),
+ ('entity_id', PmConfig.GUAGE),
+ ("interval_end_time", PmConfig.GUAGE),
+ ("ploam_mic_error_count", PmConfig.COUNTER),
+ ("downstream_ploam_messages_count", PmConfig.COUNTER),
+ ("profile_messages_received", PmConfig.COUNTER),
+ ("ranging_time_messages_received", PmConfig.COUNTER),
+ ("deactivate_onu_id_messages_received", PmConfig.COUNTER),
+ ("disable_serial_number_messages_received", PmConfig.COUNTER),
+ ("request_registration_messages_received", PmConfig.COUNTER),
+ ("assign_alloc_id_messages_received", PmConfig.COUNTER),
+ ("key_control_messages_received", PmConfig.COUNTER),
+ ("sleep_allow_messages_received", PmConfig.COUNTER),
+ ("baseline_omci_messages_received_count", PmConfig.COUNTER),
+ ("extended_omci_messages_received_count", PmConfig.COUNTER),
+ ("assign_onu_id_messages_received", PmConfig.COUNTER),
+ ("omci_mic_error_count", PmConfig.COUNTER),
+ }
+ self._xgpon_downstream_history_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in xgpon_downstream_history}
+
+ xgpon_upstream_history = { # XgPon Upstream History Data (Class ID 346)
+ ('enabled', PmConfig.STATE),
+ ('class_id', PmConfig.GUAGE),
+ ('entity_id', PmConfig.GUAGE),
+ ("interval_end_time", PmConfig.GUAGE),
+ ("upstream_ploam_message_count", PmConfig.COUNTER),
+ ("serial_number_onu_message_count", PmConfig.COUNTER),
+ ("registration_message_count", PmConfig.COUNTER),
+ ("key_report_message_count", PmConfig.COUNTER),
+ ("acknowledge_message_count", PmConfig.COUNTER),
+ ("sleep_request_message_count", PmConfig.COUNTER),
+ }
+ self._xgpon_upstream_history_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in xgpon_upstream_history}
+ self._configs = {
+ EthernetFrameUpstreamPerformanceMonitoringHistoryData.class_id: self._ethernet_bridge_history_config,
+ EthernetFrameDownstreamPerformanceMonitoringHistoryData.class_id: self._ethernet_bridge_history_config,
+ EthernetFrameExtendedPerformanceMonitoring.class_id: self._ethernet_bridge_history_config,
+ EthernetFrameExtendedPerformanceMonitoring64Bit.class_id: self._ethernet_bridge_history_config,
+ EthernetPMMonitoringHistoryData.class_id: self._ethernet_uni_history_config,
+ FecPerformanceMonitoringHistoryData.class_id: self._fec_history_config,
+ GemPortNetworkCtpMonitoringHistoryData.class_id: self._gem_port_history_config,
+ XgPonTcPerformanceMonitoringHistoryData.class_id: self._xgpon_tc_history_config,
+ XgPonDownstreamPerformanceMonitoringHistoryData.class_id: self._xgpon_downstream_history_config,
+ XgPonUpstreamPerformanceMonitoringHistoryData.class_id: self._xgpon_upstream_history_config
+ }
+
+ def update(self, pm_config):
+ """
+ Update the PM Configuration.
+
+ For historical PM Intervals, the frequency always zero since the actual collection
+ and publishing is provided by the OpenOMCI library
+
+ :param pm_config:
+ """
+ for m in pm_config.groups:
+ # TODO: Need to support individual group enable/disable
+ pass
+ # self.pm_group_metrics[m.group_name].config.enabled = m.enabled
+ # if m.enabled is True:,
+ # self.enable_pm_collection(m.group_name, remote)
+ # else:
+ # self.disable_pm_collection(m.group_name, remote)
+
+ def make_proto(self, pm_config=None):
+ """
+ From the PM Configurations defined in this class's initializer, create
+ the PMConfigs protobuf message that defines our PM configuation and
+ data.
+
+ All ONU PM Interval metrics are grouped metrics that are generated autonmouslly
+ from the OpenOMCI Performace Intervals state machine.
+
+ :param pm_config (PMConfigs) PM Configuration message to add OpenOMCI config items too
+ :return: (PmConfigs) PM Configuration Protobuf message
+ """
+ assert pm_config is not None
+
+ pm_ethernet_bridge_history = PmGroupConfig(group_name='Ethernet Bridge Port History',
+ group_freq=0,
+ enabled=True)
+
+ for m in sorted(self._ethernet_bridge_history_config):
+ pm = self._ethernet_bridge_history_config[m]
+ pm_ethernet_bridge_history.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+
+ pm_ethernet_uni_history = PmGroupConfig(group_name='Ethernet UNI History',
+ group_freq=0,
+ enabled=True)
+
+ for m in sorted(self._ethernet_uni_history_config):
+ pm = self._ethernet_uni_history_config[m]
+ pm_ethernet_uni_history.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+
+ pm_fec_history = PmGroupConfig(group_name='Upstream Ethernet History',
+ group_freq=0,
+ enabled=True)
+
+ for m in sorted(self._fec_history_config):
+ pm = self._fec_history_config[m]
+ pm_fec_history.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+
+ pm_gem_port_history = PmGroupConfig(group_name='GEM Port History',
+ group_freq=0,
+ enabled=True)
+
+ for m in sorted(self._gem_port_history_config):
+ pm = self._gem_port_history_config[m]
+ pm_gem_port_history.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+
+ pm_xgpon_tc_history = PmGroupConfig(group_name='xgPON TC History',
+ group_freq=0,
+ enabled=True)
+
+ for m in sorted(self._xgpon_tc_history_config):
+ pm = self._xgpon_tc_history_config[m]
+ pm_xgpon_tc_history.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+
+ pm_xgpon_downstream_history = PmGroupConfig(group_name='xgPON Downstream History',
+ group_freq=0,
+ enabled=True)
+
+ for m in sorted(self._xgpon_downstream_history_config):
+ pm = self._xgpon_downstream_history_config[m]
+ pm_xgpon_downstream_history.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+
+ pm_xgpon_upstream_history = PmGroupConfig(group_name='xgPON Upstream History',
+ group_freq=0,
+ enabled=True)
+
+ for m in sorted(self._xgpon_upstream_history_config):
+ pm = self._xgpon_upstream_history_config[m]
+ pm_xgpon_upstream_history.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+
+ pm_config.groups.extend([pm_ethernet_bridge_history,
+ pm_ethernet_uni_history,
+ pm_fec_history,
+ pm_gem_port_history,
+ pm_xgpon_tc_history,
+ pm_xgpon_downstream_history,
+ pm_xgpon_upstream_history
+ ])
+ return pm_config
+
+ def publish_metrics(self, interval_data):
+ """
+ Collect the metrics for this ONU PM Interval
+
+ :param interval_data: (dict) PM interval dictionary with structure of
+ {
+ 'class_id': self._class_id,
+ 'entity_id': self._entity_id,
+ 'me_name': self._entity.__name__, # Mostly for debugging...
+ 'interval_utc_time': None,
+ # Counters added here as they are retrieved
+ }
+
+ :return: (dict) Key/Value of metric data
+ """
+ try:
+ import arrow
+ from voltha.protos.events_pb2 import KpiEvent, KpiEventType, MetricValuePairs
+ # Locate config
+
+ config = self._configs.get(interval_data['class_id'])
+
+ if config is not None and config['enabled'].enabled:
+ # Extract only the metrics we need to publish
+ config_keys = config.keys()
+ metrics = {
+ interval_data['me_name']: {k: v
+ for k, v in interval_data.items()
+ if k in config_keys and v is not None}
+ }
+ # Prepare the KpiEvent for submission
+ kpi_event = KpiEvent(
+ type=KpiEventType.slice,
+ ts=arrow.get(interval_data['interval_utc_time']).timestamp,
+ prefixes={
+ self.prefix + '.{}'.format(k): MetricValuePairs(metrics=metrics[k])
+ for k in metrics.keys()}
+ )
+ self.adapter_agent.submit_kpis(kpi_event)
+
+ except Exception as e:
+ self.log.exception('failed-to-submit-kpis', e=e)
diff --git a/voltha/extensions/pki/onu/onu_pm_metrics.py b/voltha/extensions/pki/onu/onu_pm_metrics.py
new file mode 100644
index 0000000..f1f08f1
--- /dev/null
+++ b/voltha/extensions/pki/onu/onu_pm_metrics.py
@@ -0,0 +1,144 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from voltha.protos.device_pb2 import PmConfig, PmConfigs, PmGroupConfig
+from voltha.extensions.pki.adapter_pm_metrics import AdapterPmMetrics
+from voltha.extensions.pki.onu.onu_omci_pm import OnuOmciPmMetrics
+
+
+class OnuPmMetrics(AdapterPmMetrics):
+ """
+ Shared ONU Device Adapter PM Metrics Manager
+
+ This class specifically addresses ONU genernal PM (health, ...) area
+ specific PM (OMCI, PON, UNI) is supported in encapsulated classes accessible
+ from this object
+ """
+ def __init__(self, adapter_agent, device_id, grouped=False, freq_override=False, **kwargs):
+ """
+ Initializer for shared ONU Device Adapter PM metrics
+
+ :param adapter_agent: (AdapterAgent) Adapter agent for the device
+ :param device_id: (str) Device ID
+ :param grouped: (bool) Flag indicating if statistics are managed as a group
+ :param freq_override: (bool) Flag indicating if frequency collection can be specified
+ on a per group basis
+ :param kwargs: (dict) Device Adapter specific values. For an ONU Device adapter, the
+ expected key-value pairs are listed below. If not provided, the
+ associated PM statistics are not gathered:
+
+ 'heartbeat': Reference to the a class that provides an ONU heartbeat
+ statistics. TODO: This needs to be standardized
+ """
+ super(OnuPmMetrics, self).__init__(adapter_agent, device_id,
+ grouped=grouped, freq_override=freq_override, **kwargs)
+
+ #
+ # The following HeartBeat PM is only an example. We may want to have a common heartbeat
+ # object for OLT and ONU DAs that work the same. If so, it could also provide PM information
+ #
+ # TODO: In the actual 'collection' of PM data, I have the heartbeat stats disabled since
+ # there is not yet a common 'heartbeat' object
+ #
+ self.health_pm_names = {
+ ('enabled', PmConfig.STATE),
+ ('alarm_active', PmConfig.STATE),
+ ('heartbeat_count', PmConfig.COUNTER),
+ ('heartbeat_miss', PmConfig.COUNTER),
+ ('alarms_raised_count', PmConfig.COUNTER),
+ ('heartbeat_failed_limit', PmConfig.COUNTER),
+ ('heartbeat_interval', PmConfig.COUNTER),
+ }
+ # TODO Add PON Port pollable PM as a separate class and include like OMCI
+ # TODO Add UNI Port pollable PM as a separate class and include like OMCI
+ self._heartbeat = kwargs.pop('heartbeat', None)
+ self.health_metrics_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in self.health_pm_names}
+
+ self.omci_pm = OnuOmciPmMetrics(adapter_agent, device_id, grouped=grouped,
+ freq_override=freq_override, **kwargs)
+
+ def update(self, pm_config):
+ # TODO: Test both 'group' and 'non-group' functionality
+ # TODO: Test frequency override capability for a particular group
+ if self.default_freq != pm_config.default_freq:
+ # Update the callback to the new frequency.
+ self.default_freq = pm_config.default_freq
+ self.lc.stop()
+ self.lc.start(interval=self.default_freq / 10)
+
+ if pm_config.grouped:
+ for m in pm_config.groups:
+ # TODO: Need to support individual group enable/disable
+ pass
+ # self.pm_group_metrics[m.group_name].config.enabled = m.enabled
+ # if m.enabled is True:,
+ # self.enable_pm_collection(m.group_name, remote)
+ # else:
+ # self.disable_pm_collection(m.group_name, remote)
+ else:
+ for m in pm_config.metrics:
+ self.health_metrics_config[m.name].enabled = m.enabled
+
+ self.omci_pm.update(pm_config)
+
+ def make_proto(self, pm_config=None):
+ if pm_config is None:
+ pm_config = PmConfigs(id=self.device_id,
+ default_freq=self.default_freq,
+ grouped=self.grouped,
+ freq_override=self.freq_override)
+ metrics = set()
+
+ if self._heartbeat is not None:
+ if self.grouped:
+ pm_health_stats = PmGroupConfig(group_name='Heartbeat',
+ group_freq=self.default_freq,
+ enabled=True)
+ else:
+ pm_health_stats = pm_config
+
+ # Add metrics to the PM Group (or as individual metrics_
+ for m in sorted(self.health_metrics_config):
+ pm = self.health_metrics_config[m]
+ if not self.grouped:
+ if pm.name in metrics:
+ continue
+ metrics.add(pm.name)
+
+ pm_health_stats.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+ if self.grouped:
+ pm_config.groups.extend([pm_health_stats])
+
+ # TODO Add PON Port PM
+ # TODO Add UNI Port PM
+ pm_config = self.omci_pm.make_proto(pm_config)
+ return pm_config
+
+ def collect_group_metrics(self, metrics=None):
+ # TODO: Currently PM collection is done for all metrics/groups on a single timer
+ if metrics is None:
+ metrics = dict()
+
+ # TODO: Heartbeat stats disabled since it is not a common item on all ONUs (or OLTs)
+ # if self._heartbeat is not None:
+ # metrics['heartbeat'] = self.collect_metrics(self._heartbeat,
+ # self.health_pm_names,
+ # self.health_metrics_config)
+ self.omci_pm.collect_device_metrics(metrics=metrics)
+ # TODO Add PON Port PM
+ # TODO Add UNI Port PM
+ return metrics