Use kafka producer properly
Calling producer.poll() is required or producer will fail with:
BufferError: Local: Queue full, instance_id: ...
Add 'extra_config' parameter to extend configuration of the kafka producer
Update tests/docs
Change-Id: I8f17437c0c24d61a6f3be5de3d4004c21fcad75a
diff --git a/README.rst b/README.rst
index d5470bd..c76886b 100644
--- a/README.rst
+++ b/README.rst
@@ -35,12 +35,23 @@
**Parameters that can be provided to KafkaLogHandler:**
*bootstrap_servers*
- List of Kafka bootstrap servers to connect to. See confluent_kafka docs.
+ List of Kafka bootstrap servers to connect to.
**default:** ``["localhost:9092"]``
+*extra_config*
+ Dictionary of extra `producer configuration
+ <https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md>`_
+ passed to librdkafka.
+
+ NOTE: The ``bootstrap_servers`` parameter will overwrite
+ ``bootstrap.servers``.
+
+ **default:** ``{}``
+
+
*timeout*
- Timeout in seconds for flushing producer queue. See confluent_kafka docs.
+ Timeout in seconds for flushing producer queue. See librdkafka docs.
**default:** ``10.0``
@@ -58,15 +69,17 @@
**default:** ``"klh"``
*flatten*
- Flattens nested dictionary keys passed as structured logging into the parent
+ Flattens nested dictionaries and lists passed as structured logging into the parent
dictionary layer, up to a certain depth.
This is useful when logging to external systems that don't have good support
for hierarchical data.
- Example: ``{'a': {'b': 'c'}}`` would be flattened to ``{'a.b': 'c'}``
+ Example dictionary: ``{'a': {'b': 'c'}}`` would be flattened to ``{'a.b': 'c'}``
- If the depth is exceeded, any remaining deeper dict will be added to the
+ Example list: ``{'a': ['b', 'c']}`` would be flattened to ``{'a.0': 'b', 'a.1': 'c'}``
+
+ If the depth is exceeded, any remaining deeper items will be added to the
output under the flattened key.
Set to ``0`` to turn off flattening.
@@ -74,7 +87,7 @@
**default:** ``5``
*separator*
- Separator used between keys when flattening.
+ Separator used between items when flattening.
**default:** ``.``
@@ -84,9 +97,7 @@
**default:** ``["_logger", "_name"]``
-Tests
-=====
+Testing
+=======
-Unit tests can be run with:
-
- nose2 --verbose --coverage-report term
+Unit tests can be run with ``tox``