[VOL-3346] As an Operator , I should be able to push open-olt agent logs to Centralize logging system of VOLTHA
Change-Id: I14e9a181331ccee0663d41f551e82e2055d1109d
diff --git a/README.md b/README.md
index 69492e2..acaf7cc 100644
--- a/README.md
+++ b/README.md
@@ -306,7 +306,7 @@
Note that the required INBAND ONL version `4.14` is built as part of the above
build procedure and is available at path
`build/onl/OpenNetworkLinux/RELEASE/jessie/amd64/ONL-onl-4.14_ONL-OS8_2020-04-22.
-2206-b4af32e_AMD64_INSTALLED_INSTALLER\.
+2206-b4af32e_AMD64_INSTALLED_INSTALLER\.`
This ONL Installer should be used to flash the OS on the OLT.
NOTE: To compile for ASGvOLT 64 port GPON OLT, set `OPENOLTDEVICE` to
@@ -316,6 +316,63 @@
make OPENOLTDEVICE=asgvolt64
```
+### Log Collection for whitebox OLT Device
+
+To collect logs from openolt, dev_mgmt_daemon and syslog processes install td-agent(fluentd variant) directly on OLT device which will capture and transmits the logs to elasticsearch pod running in voltha cluster.
+
+Prerequisite:
+
+OLT should have ntp installed to ensure it has correct time(which is used by td-agent for generated events)
+
+```shell
+apt-get install ntp
+```
+
+Installation of td-agent deb package:
+
+* Download the deb package for td-agent
+
+```shell
+wget http://packages.treasuredata.com.s3.amazonaws.com/3/ubuntu/xenial/pool/contrib/t/td-agent/td-agent_3.8.0-0_amd64.deb
+```
+
+* Install td-agent on device
+
+```shell
+dpkg -i td-agent_3.8.0-0_amd64.deb
+```
+
+Post Installation:
+
+We have created custom td-agent configuration file to handle format of involved log files using right input plugins and elasticsearch output plugin.
+
+* Copy the custom config file
+
+```shell
+cd logConf
+cp td-agent.conf /etc/td-agent.conf
+```
+
+* Set elasticsearch host and port in /etc/td-agent.conf
+* Restart the td-agent service
+
+```shell
+service td-agent restart
+```
+
+Need to redirect syslog to default port of fluentd syslog plugin.
+
+* Add “*.* @127.0.0.1:42185” to /etc/rsyslog.conf
+* Restart syslog using
+
+```shell
+/etc/init.d/rsyslog restart
+```
+
+**Note**:
+
+To enable TLS encryption features with td-agent [reffer:](https://docs.google.com/document/d/1KF1HhE-PN-VY4JN2bqKmQBrZghFC5HQM_s0mC0slapA/edit)
+
### Cleanup
To cleanup the repository and start the build procedure again, run:
diff --git a/logConf/td-agent.conf b/logConf/td-agent.conf
new file mode 100644
index 0000000..01da4f8
--- /dev/null
+++ b/logConf/td-agent.conf
@@ -0,0 +1,113 @@
+# Copyright (c) 2020 Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+# http:#www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# input plugin to collect openolt logs
+# formating the input logs using regex and creating a feilds such as level, instanceId, caller and msg.
+<source>
+ @id openolt.log
+ @type tail
+ path /var/log/openolt.log
+ pos_file /var/log/td-agent/openolt.log.pos
+ read_from_head true
+ tag openolt
+ <parse>
+ @type multiline
+ format_firstline /\[(\d+)(.\s+)(?<level>\w+)(?<instanceId>[\s]\w+)(\s+)\](?<caller>.\w+...\s\d+)(.\s)/
+ format1 /\[(\d+)(.\s+)(?<level>\w+)(?<instanceId>[\s]\w+)(\s+)\](?<caller>.\w+...\s\d+)(.\s)(?<msg>.*)/
+ </parse>
+</source>
+
+# Formating the `instanceId` field by concating with the device `Ip Address`.
+# For example: If the extracted value for the field 'instanceId' is `OPENOLT` and `Ip` of the device is `192.x.x.x` after concating the
+# `instanceId` will be "OPENOLT-192.x.x.x"
+# The log levels which are coming in the logs are like 'I', 'E' etc so replacing the log level 'I' to 'info', 'W' to 'warn' and 'E' to 'error'
+<filter openolt.**>
+ @type record_transformer
+ enable_ruby true
+ <record>
+ instanceId ${record["instanceId"]}-${"#{(Socket.ip_address_list.detect do |intf| intf.ipv4_private? end).ip_address}"}
+ level ${if record['level'] == 'I' then 'info' elsif record['level'] == 'W' then 'warn' else 'error'; end}
+ </record>
+</filter>
+
+# input plugin to collect dev_mgmt_daemon logs
+# formating the input logs using regex and creating a feilds such as level, instanceId, caller and msg.
+<source>
+ @id dev_mgmt_daemon.log
+ @type tail
+ path /var/log/dev_mgmt_daemon.log
+ pos_file /var/log/td-agent/dev_mgmt_daemon.log.pos
+ tag dev_mgmt
+ read_from_head true
+ <parse>
+ @type multiline
+ format_firstline /\[(\d+)(.\s+)(?<level>\w+)(?<instanceId>[\s]\w+)(\s+)\](?<caller>.\w+...\s\d+)(.\s)/
+ format1 /\[(\d+)(.\s+)(?<level>\w+)(?<instanceId>[\s]\w+)(\s+)\](?<caller>.\w+...\s\d+)(.\s)(?<msg>.*)/
+ </parse>
+</source>
+
+# Formating the `instanceId` field by concating with the device `Ip Address`.
+# For example: If the extracted value for the field 'instanceId' is `SW_UTIL` and `Ip` of the device is `192.x.x.x` after concating the
+# `instanceId` will be "SW_UTIL-192.x.x.x"
+# The log levels which are coming in the logs are like 'I', 'E' etc so replacing the log level 'I' to 'info', 'W' to 'warn' and 'E' to 'error'
+<filter dev_mgmt.**>
+ @type record_transformer
+ enable_ruby true
+ <record>
+ instanceId ${record["instanceId"]}-${"#{(Socket.ip_address_list.detect do |intf| intf.ipv4_private? end).ip_address}"}
+ level ${if record['level'] == 'I' then 'info' elsif record['level'] == 'W' then 'warn' else 'error'; end}
+ </record>
+</filter>
+
+# input plugin to collect system logs
+# formating the input logs using regex and creating a feilds such as host, caller and msg.
+<source>
+ @type syslog
+ port 42185
+ tag system
+ format /(.*\S \d{1,2}:\d{1,2}:\d{1,2}.)(?<host>[^\s]\w+) (?<caller>[^\s][^:]+)(.)(?<msg>.*)/
+</source>
+
+<filter **>
+ @type stdout
+</filter>
+
+# Output elasticsearch plugin
+# provide host and port of elasticsearch
+<match **>
+ @id elasticsearch
+ @type elasticsearch
+ @log_level info
+ include_tag_key true
+ host localhost
+ port 9200
+ scheme http
+ ssl_verify false
+ logstash_format true
+ reconnect_on_error true
+ reload_on_failure false
+ reload_connections false
+ <buffer>
+ @type file
+ path /var/log/td-agent/buffer/elasticsearch
+ flush_mode interval
+ retry_type exponential_backoff
+ flush_thread_count 8
+ flush_interval 5s
+ retry_max_interval 30
+ chunk_limit_size 32MB
+ queue_limit_length 8
+ overflow_action block
+ </buffer>
+</match>