Initial commit moving openolt adapter from voltha-go to the new repo.
This version works with ponsim rather than openolt, this is temporary.
It is currently being fixed to work with openolt.
Change-Id: I34a800c98f050140b367e2d474b7aa8b79f34b9a
Signed-off-by: William Kurkian <wkurkian@cisco.com>
diff --git a/python/adapters/extensions/kpi/olt/README.md b/python/adapters/extensions/kpi/olt/README.md
new file mode 100644
index 0000000..e0012b8
--- /dev/null
+++ b/python/adapters/extensions/kpi/olt/README.md
@@ -0,0 +1,179 @@
+# OLT PM Metrics
+
+
+**THESE ARE PRELIMINARY METRIC GROUPS**, Work is needed by the VOLTHA community to reach a consensus on the
+actual metrics that will be provided. **Also**, please read the **Remaining Work Item** sections of each
+README file.
+
+
+
+This document outlines the metrics reported by VOLTHA OLTs. These are currently collected
+from OLT Device Adapter which is responsible for polling the hardware for information. A future
+version of the Performance Monitoring Library will allow for collection on-demand.
+
+## Format on the Kafka bus
+
+The format of the OLT KPI Events is detailed in the [Basic KPI Format (**KpiEvent2**)](../README.md)
+section of this documents parent directory for wire format on the bus. This document primarily provides
+the group metric information for OLT PKIs and associated metadata context information.
+
+**All** metric values reported by the library are reported as *float*s. The context and metric tables
+listed in the sections below report the type as initially collected by the OLT Device Adapters.
+
+#OLT PM Metric Groups
+
+The following sections outline the KPI metrics gathered by most OLT Device adapters. If an OLT does not
+support a specific metric in a group, it will not report that metric. This is preferred to reporting a
+metric and it always having a value of 0.0 (which could be misleading).
+
+## Admin and Oper State/status
+
+Various interfaces will provide a numeric (integer) value for the current Admin State and Operation
+Status of the interface. These map to the following states:
+
+**Admin State**
+
+| State | Value | Notes |
+| ----------------: | :---: | :---- |
+| UNKNOWN | 0 | The administrative state of the device is unknown |
+| DISABLED | 2 | The device is disabled and shall not perform its intended forwarding functions other than being available for re-activation. |
+| PREPROVISIONED | 1 | The device is pre-provisioned into Voltha, but not contacted by it |
+| ENABLED | 3 | The device is enabled for activation and operation |
+| DOWNLOADING_IMAGE | 4 | The device is in the state of image download |
+
+**Operational Status**
+
+| State | Value | Notes |
+| ---------: | :---: | :---- |
+| UNKNOWN | 0 | The status of the device is unknown at this point |
+| DISCOVERED | 1 | The device has been discovered, but not yet activated |
+| ACTIVATING | 2 | The device is being activated (booted, rebooted, upgraded, etc.) |
+| TESTING | 3 | Service impacting tests are being conducted |
+| ACTIVE | 4 | The device is up and active |
+| FAILED | 5 | The device has failed and cannot fulfill its intended role |
+
+## NNI KPI Metrics
+
+This metric provides metrics for a specific NNI Port of an OLT
+
+**Metadata Context items**
+
+| key | value | Notes |
+| :---------: | :------ | :---- |
+| intf_id | integer | Physical device interface port number for this NNI port |
+
+**Metrics**
+
+| key | type / size | Notes |
+| :--------------: | :----------- | :---- |
+| admin_state | state | See _Admin State_ section above |
+| oper_status | state | See _Operational Status_ section above |
+| rx_bytes | int, 64-bits | TODO: add definition here... |
+| rx_packets | int, 64-bits | TODO: add definition here... |
+| rx_ucast_packets | int, 64-bits | TODO: add definition here... |
+| rx_mcast_packets | int, 64-bits | TODO: add definition here... |
+| rx_bcast_packets | int, 64-bits | TODO: add definition here... |
+| rx_error_packets | int, 64-bits | TODO: add definition here... |
+| tx_bytes | int, 64-bits | TODO: add definition here... |
+| tx_packets | int, 64-bits | TODO: add definition here... |
+| tx_ucast_packets | int, 64-bits | TODO: add definition here... |
+| tx_mcast_packets | int, 64-bits | TODO: add definition here... |
+| tx_bcast_packets | int, 64-bits | TODO: add definition here... |
+| tx_error_packets | int, 64-bits | TODO: add definition here... |
+| rx_crc_errors | int, 64-bits | TODO: add definition here... |
+| bip_errors | int, 64-bits | TODO: add definition here... |
+
+## PON KPI Metrics
+
+The OLT PON Port metrics
+
+**Metadata Context items**
+
+| key | value | Notes |
+| :---------: | :------ | :---- |
+| intf_id | integer | Physical device interface port number for this NNI port |
+| pon_id | integer | PON ID (0..n) |
+
+**Metrics**
+
+| key | type / size | Notes |
+| :------------------: | :----------- | :---- |
+| admin_state | state | See _Admin State_ section above |
+| oper_status | state | See _Operational Status_ section above |
+| rx_packets | int, 64-bits | Sum of all the RX Packets of GEM ports that are not base TCONT's |
+| rx_bytes | int, 64-bits | Sum of all the RX Octets of GEM ports that are not base TCONT's |
+| tx_packets | int, 64-bits | Sum of all the TX Packets of GEM ports that are not base TCONT's |
+| tx_bytes | int, 64-bits | Sum of all the TX Octets of GEM ports that are not base TCONT's |
+| tx_bip_errors | int, 32-bits | Sum of all the TX ONU bip errors to get TX BIP's per PON |
+| in_service_onus | int | The number of activated ONUs on this pon |
+| closest_onu_distance | float | Distance to the closest ONU, units=kM w/granularity in the thousandths |
+
+## ONU KPI Metrics
+
+The OLT metrics for each activated ONUs
+
+**Metadata Context items**
+
+| key | value | Notes |
+| :---------: | :------ | :---- |
+| intf_id | integer | Physical device interface port number for this NNI port |
+| pon_id | integer | PON ID (0..n) |
+| onu_id | integer | ONU ID |
+
+**Metrics**
+
+| key | type / size | Notes |
+| :----------------: | :----------- | :---- |
+| fiber_length | float | Distance to ONU, units=kM w/granularity in the thousandths |
+| equalization_delay | int, 32-bits | Equalization delay |
+| rssi | int, 32-bits | The received signal strength indication of the ONU. |
+
+**TODO**: How about the following as well?
+ - rx_packets - int, 32-bits - Rx packets received on all GEM ports
+ - rx_bytes - int, 64-bits - Rx octets received on all GEM ports
+ - tx_packets - int, 32-bits - Tx packets transmitted on all GEM ports
+ - tx_bytes - int, 64-bits - Rx packets transmitted on all GEM ports
+ - tx_bip_errors - int, 32-bits - Sum of all the TX ONU bip errors to get TX BIP's on all GEM ports
+
+## GEM Port KPI Metrics
+
+The GEM Port metrics for each activated ONUs
+
+**Metadata Context items**
+
+| key | value | Notes |
+| :---------: | :------ | :---- |
+| intf_id | integer | Physical device interface port number for this NNI port |
+| pon_id | integer | PON ID (0..n) |
+| onu_id | integer | ONU ID |
+| gem_id | integer | GEM Port ID |
+
+**Metrics**
+
+| key | type / size | Notes |
+| :---------: | :----------- | :---- |
+| alloc_id | int, 16-bits | TODO: add definition here... |
+| rx_packets | int, 32-bits | Rx packets received |
+| rx_bytes | int, 64-bits | Rx octets received |
+| tx_packets | int, 32-bits | Tx packets transmitted |
+| tx_bytes | int, 64-bits | Rx packets transmitted |
+
+# Remaining Work Items
+
+This initial code is only a preliminary work. See the [Remaining Work Items](../README.md)
+section of this document's parent directory for a list of remaining tasks.
+
+- [VOL-932](https://jira.opencord.org/browse/VOL-932) PM Interval collection on the OLT. Need
+ to consult OLT device adapter vendors and operators for which KPIs would best fit in the
+ interval groups. Intervals differ from other metric groups as they are defined to collect on
+ a specific interval (15-minutes most common) and at the start of the interval, the counters
+ should be set to zero so that the accumulation during the interval is what is reported. See
+ also [VOL-933](https://jira.opencord.org/browse/VOL-932),
+ [VOL-934](https://jira.opencord.org/browse/VOL-934),
+ [VOL-935](https://jira.opencord.org/browse/VOL-935),
+ [VOL-938](https://jira.opencord.org/browse/VOL-938),
+ [VOL-939](https://jira.opencord.org/browse/VOL-939),
+ [VOL-940](https://jira.opencord.org/browse/VOL-940).
+ **NOTE**: A couple of the ones above are for the ONU
+
+TODO: For each group, list if the default is enabled/disabled
\ No newline at end of file
diff --git a/python/adapters/extensions/kpi/olt/__init__.py b/python/adapters/extensions/kpi/olt/__init__.py
new file mode 100644
index 0000000..b0fb0b2
--- /dev/null
+++ b/python/adapters/extensions/kpi/olt/__init__.py
@@ -0,0 +1,13 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
diff --git a/python/adapters/extensions/kpi/olt/olt_pm_metrics.py b/python/adapters/extensions/kpi/olt/olt_pm_metrics.py
new file mode 100644
index 0000000..ea2e0c8
--- /dev/null
+++ b/python/adapters/extensions/kpi/olt/olt_pm_metrics.py
@@ -0,0 +1,300 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from voltha.protos.device_pb2 import PmConfig, PmConfigs, PmGroupConfig
+from voltha.extensions.kpi.adapter_pm_metrics import AdapterPmMetrics
+
+
+class OltPmMetrics(AdapterPmMetrics):
+ """
+ Shared OL Device Adapter PM Metrics Manager
+
+ This class specifically addresses ONU general PM (health, ...) area
+ specific PM (OMCI, PON, UNI) is supported in encapsulated classes accessible
+ from this object
+ """
+ def __init__(self, adapter_agent, device_id, logical_device_id,
+ grouped=False, freq_override=False, **kwargs):
+ """
+ Initializer for shared ONU Device Adapter PM metrics
+
+ :param adapter_agent: (AdapterAgent) Adapter agent for the device
+ :param device_id: (str) Device ID
+ :param logical_device_id: (str) VOLTHA Logical Device ID
+ :param grouped: (bool) Flag indicating if statistics are managed as a group
+ :param freq_override: (bool) Flag indicating if frequency collection can be specified
+ on a per group basis
+ :param kwargs: (dict) Device Adapter specific values. For an ONU Device adapter, the
+ expected key-value pairs are listed below. If not provided, the
+ associated PM statistics are not gathered:
+
+ 'nni-ports': List of objects that provide NNI (northbound) port statistics
+ 'pon-ports': List of objects that provide PON port statistics
+ """
+ super(OltPmMetrics, self).__init__(adapter_agent, device_id, logical_device_id,
+ grouped=grouped, freq_override=freq_override,
+ **kwargs)
+
+ # PM Config Types are COUNTER, GAUGE, and STATE
+ self.nni_pm_names = {
+ ('intf_id', PmConfig.CONTEXT), # Physical device interface ID/Port number
+
+ ('admin_state', PmConfig.STATE),
+ ('oper_status', PmConfig.STATE),
+
+ ('rx_bytes', PmConfig.COUNTER),
+ ('rx_packets', PmConfig.COUNTER),
+ ('rx_ucast_packets', PmConfig.COUNTER),
+ ('rx_mcast_packets', PmConfig.COUNTER),
+ ('rx_bcast_packets', PmConfig.COUNTER),
+ ('rx_error_packets', PmConfig.COUNTER),
+
+ ('tx_bytes', PmConfig.COUNTER),
+ ('tx_packets', PmConfig.COUNTER),
+ ('tx_ucast_packets', PmConfig.COUNTER),
+ ('tx_mcast_packets', PmConfig.COUNTER),
+ ('tx_bcast_packets', PmConfig.COUNTER),
+ ('tx_error_packets', PmConfig.COUNTER),
+ ('rx_crc_errors', PmConfig.COUNTER),
+ ('bip_errors', PmConfig.COUNTER),
+ }
+ self.pon_pm_names = {
+ ('intf_id', PmConfig.CONTEXT), # Physical device port number (PON)
+ ('pon_id', PmConfig.CONTEXT), # PON ID (0..n)
+
+ ('admin_state', PmConfig.STATE),
+ ('oper_status', PmConfig.STATE),
+ ('rx_packets', PmConfig.COUNTER),
+ ('rx_bytes', PmConfig.COUNTER),
+ ('tx_packets', PmConfig.COUNTER),
+ ('tx_bytes', PmConfig.COUNTER),
+ ('tx_bip_errors', PmConfig.COUNTER),
+ ('in_service_onus', PmConfig.GAUGE),
+ ('closest_onu_distance', PmConfig.GAUGE)
+ }
+ self.onu_pm_names = {
+ ('intf_id', PmConfig.CONTEXT), # Physical device port number (PON)
+ ('pon_id', PmConfig.CONTEXT),
+ ('onu_id', PmConfig.CONTEXT),
+
+ ('fiber_length', PmConfig.GAUGE),
+ ('equalization_delay', PmConfig.GAUGE),
+ ('rssi', PmConfig.GAUGE),
+ }
+ self.gem_pm_names = {
+ ('intf_id', PmConfig.CONTEXT), # Physical device port number (PON)
+ ('pon_id', PmConfig.CONTEXT),
+ ('onu_id', PmConfig.CONTEXT),
+ ('gem_id', PmConfig.CONTEXT),
+
+ ('alloc_id', PmConfig.GAUGE),
+ ('rx_packets', PmConfig.COUNTER),
+ ('rx_bytes', PmConfig.COUNTER),
+ ('tx_packets', PmConfig.COUNTER),
+ ('tx_bytes', PmConfig.COUNTER),
+ }
+ self.nni_metrics_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in self.nni_pm_names}
+ self.pon_metrics_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in self.pon_pm_names}
+ self.onu_metrics_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in self.onu_pm_names}
+ self.gem_metrics_config = {m: PmConfig(name=m, type=t, enabled=True)
+ for (m, t) in self.gem_pm_names}
+
+ self._nni_ports = kwargs.pop('nni-ports', None)
+ self._pon_ports = kwargs.pop('pon-ports', None)
+
+ def update(self, pm_config):
+ try:
+ # TODO: Test frequency override capability for a particular group
+ if self.default_freq != pm_config.default_freq:
+ # Update the callback to the new frequency.
+ self.default_freq = pm_config.default_freq
+ self.lc.stop()
+ self.lc.start(interval=self.default_freq / 10)
+
+ if pm_config.grouped:
+ for group in pm_config.groups:
+ group_config = self.pm_group_metrics.get(group.group_name)
+ if group_config is not None:
+ group_config.enabled = group.enabled
+ else:
+ msg = 'There are no independent OLT metrics, only group metrics at this time'
+ raise NotImplemented(msg)
+
+ except Exception as e:
+ self.log.exception('update-failure', e=e)
+ raise
+
+ def make_proto(self, pm_config=None):
+ if pm_config is None:
+ pm_config = PmConfigs(id=self.device_id, default_freq=self.default_freq,
+ grouped=self.grouped,
+ freq_override=self.freq_override)
+ metrics = set()
+ have_nni = self._nni_ports is not None and len(self._nni_ports) > 0
+ have_pon = self._pon_ports is not None and len(self._pon_ports) > 0
+
+ if self.grouped:
+ if have_nni:
+ pm_ether_stats = PmGroupConfig(group_name='Ethernet',
+ group_freq=self.default_freq,
+ enabled=True)
+ self.pm_group_metrics[pm_ether_stats.group_name] = pm_ether_stats
+
+ else:
+ pm_ether_stats = None
+
+ if have_pon:
+ pm_pon_stats = PmGroupConfig(group_name='PON',
+ group_freq=self.default_freq,
+ enabled=True)
+
+ pm_onu_stats = PmGroupConfig(group_name='ONU',
+ group_freq=self.default_freq,
+ enabled=True)
+
+ pm_gem_stats = PmGroupConfig(group_name='GEM',
+ group_freq=self.default_freq,
+ enabled=True)
+
+ self.pm_group_metrics[pm_pon_stats.group_name] = pm_pon_stats
+ self.pm_group_metrics[pm_onu_stats.group_name] = pm_onu_stats
+ self.pm_group_metrics[pm_gem_stats.group_name] = pm_gem_stats
+ else:
+ pm_pon_stats = None
+ pm_onu_stats = None
+ pm_gem_stats = None
+
+ else:
+ pm_ether_stats = pm_config if have_nni else None
+ pm_pon_stats = pm_config if have_pon else None
+ pm_onu_stats = pm_config if have_pon else None
+ pm_gem_stats = pm_config if have_pon else None
+
+ if have_nni:
+ for m in sorted(self.nni_metrics_config):
+ pm = self.nni_metrics_config[m]
+ if not self.grouped:
+ if pm.name in metrics:
+ continue
+ metrics.add(pm.name)
+ pm_ether_stats.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+ if have_pon:
+ for m in sorted(self.pon_metrics_config):
+ pm = self.pon_metrics_config[m]
+ if not self.grouped:
+ if pm.name in metrics:
+ continue
+ metrics.add(pm.name)
+ pm_pon_stats.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+
+ for m in sorted(self.onu_metrics_config):
+ pm = self.onu_metrics_config[m]
+ if not self.grouped:
+ if pm.name in metrics:
+ continue
+ metrics.add(pm.name)
+ pm_onu_stats.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+
+ for m in sorted(self.gem_metrics_config):
+ pm = self.gem_metrics_config[m]
+ if not self.grouped:
+ if pm.name in metrics:
+ continue
+ metrics.add(pm.name)
+ pm_gem_stats.metrics.extend([PmConfig(name=pm.name,
+ type=pm.type,
+ enabled=pm.enabled)])
+ if self.grouped:
+ pm_config.groups.extend([stats for stats in
+ self.pm_group_metrics.itervalues()])
+
+ return pm_config
+
+ def collect_metrics(self, data=None):
+ """
+ Collect metrics for this adapter.
+
+ The data collected (or passed in) is a list of pairs/tuples. Each
+ pair is composed of a MetricMetaData metadata-portion and list of MetricValuePairs
+ that contains a single individual metric or list of metrics if this is a
+ group metric.
+
+ This method is called for each adapter at a fixed frequency.
+ TODO: Currently all group metrics are collected on a single timer tick.
+ This needs to be fixed as independent group or instance collection is
+ desirable.
+
+ :param data: (list) Existing list of collected metrics (MetricInformation).
+ This is provided to allow derived classes to call into
+ further encapsulated classes.
+
+ :return: (list) metadata and metrics pairs - see description above
+ """
+ if data is None:
+ data = list()
+
+ group_name = 'Ethernet'
+ if self.pm_group_metrics[group_name].enabled:
+ for port in self._nni_ports:
+ group_data = self.collect_group_metrics(group_name,
+ port,
+ self.nni_pm_names,
+ self.nni_metrics_config)
+ if group_data is not None:
+ data.append(group_data)
+
+ for port in self._pon_ports:
+ group_name = 'PON'
+ if self.pm_group_metrics[group_name].enabled:
+ group_data = self.collect_group_metrics(group_name,
+ port,
+ self.pon_pm_names,
+ self.pon_metrics_config)
+ if group_data is not None:
+ data.append(group_data)
+
+ for onu_id in port.onu_ids:
+ onu = port.onu(onu_id)
+ if onu is not None:
+ group_name = 'ONU'
+ if self.pm_group_metrics[group_name].enabled:
+ group_data = self.collect_group_metrics(group_name,
+ onu,
+ self.onu_pm_names,
+ self.onu_metrics_config)
+ if group_data is not None:
+ data.append(group_data)
+
+ group_name = 'GEM'
+ if self.pm_group_metrics[group_name].enabled:
+ for gem in onu.gem_ports:
+ if not gem.multicast:
+ group_data = self.collect_group_metrics(group_name,
+ onu,
+ self.gem_pm_names,
+ self.gem_metrics_config)
+ if group_data is not None:
+ data.append(group_data)
+
+ # TODO: Do any multicast GEM PORT metrics here...
+ return data