src/test/results/voltha/ponsim/test_two_subscribers_with_voltha_for_igmp_multiple_times_toggling_olt_verifying_traffic

ubuntu@cord-rs-s8:~/cord-tester/src/test/setup$ sudo ./cord-test.py run -m manifest-ponsim.json -t voltha:voltha_exchange.test_two_subscribers_with_voltha_for_igmp_multiple_times_toggling_olt_verifying_traffic
# Host [172.17.0.2]:8101 found: line 3 type RSA
onoscli: eof exception found
onoscli:     logout

Connection to 172.17.0.2 closed.

result.ok, result.status_code are False and 409
ONOS app cord-config, version 2.0-SNAPSHOT installed
result.ok, result.status_code are False and 409
ONOS app aaa, version 2.0-SNAPSHOT installed
result.ok, result.status_code are False and 409
ONOS app igmp, version 2.0-SNAPSHOT installed
Controller IP [u'172.17.0.2'], Test type voltha:voltha_exchange.test_two_subscribers_with_voltha_for_igmp_multiple_times_toggling_olt_verifying_traffic
Installing cord tester ONOS app /home/ubuntu/cord-tester/src/test/apps/ciena-cordigmp-multitable-3.0-SNAPSHOT.oar
result.ok, result.status_code are False and 409
WARNING: The DOCKER_HOST_IP variable is not set. Defaulting to a blank string.
compose_consul_1 is up-to-date
compose_zookeeper_1 is up-to-date
compose_fluentd_1 is up-to-date
compose_registrator_1 is up-to-date
compose_kafka_1 is up-to-date
IP 172.18.0.2 for service consul
IP 172.18.0.5 for service kafka
IP 172.18.0.4 for service zookeeper
IP 172.18.0.6 for service registrator
IP 172.18.0.3 for service fluentd
Chameleon voltha sevice is already running. Skipped start
VOLTHA core is already running. Skipped start
VOLTHA ofagent is already running. Skipped start
PONSIM already running. Skipped start
Radius server running with IP 172.17.0.3
Running ['voltha:voltha_exchange.test_two_subscribers_with_voltha_for_igmp_multiple_times_toggling_olt_verifying_traffic'] tests across 1 containers in parallel
Starting test container cord-tester1, image cordtest/nose, tag candidate
Provisioning the ports for the test container

Running PIPEWORK cmd: pipework pon1_128 -i veth0 -l l1 cord-tester1 192.168.100.1/24
Running PIPEWORK cmd: pipework ponmgmt -i veth2 -l l2 cord-tester1 192.168.100.2/24
Running PIPEWORK cmd: pipework pon1_129 -i veth4 -l l3 cord-tester1 192.168.100.3/24
Running PIPEWORK cmd: pipework pon1_130 -i veth6 -l l4 cord-tester1 192.168.100.4/24
Running PIPEWORK cmd: pipework pon1_131 -i veth8 -l l5 cord-tester1 192.168.100.5/24
Running PIPEWORK cmd: pipework pon1_132 -i veth10 -l l6 cord-tester1 192.168.100.6/24
Running PIPEWORK cmd: pipework pon1_133 -i veth12 -l l7 cord-tester1 192.168.100.7/24
Running PIPEWORK cmd: pipework pon1_134 -i veth14 -l l8 cord-tester1 192.168.100.8/24
Running PIPEWORK cmd: pipework pon1_135 -i veth16 -l l9 cord-tester1 192.168.100.9/24
Running PIPEWORK cmd: pipework pon1_136 -i veth18 -l l10 cord-tester1 192.168.100.10/24
Running PIPEWORK cmd: pipework pon1_137 -i veth20 -l l11 cord-tester1 192.168.100.11/24
Running PIPEWORK cmd: pipework pon1_128 -i veth22 -l l12 cord-tester1 192.168.100.12/24
Running PIPEWORK cmd: pipework pon1_129 -i veth24 -l l13 cord-tester1 192.168.100.13/24
Running PIPEWORK cmd: pipework pon1_130 -i veth26 -l l14 cord-tester1 192.168.100.14/24
Running PIPEWORK cmd: pipework pon1_131 -i veth28 -l l15 cord-tester1 192.168.100.15/24
Running PIPEWORK cmd: pipework pon1_132 -i veth30 -l l16 cord-tester1 192.168.100.16/24
Running PIPEWORK cmd: pipework pon1_133 -i veth32 -l l17 cord-tester1 192.168.100.17/24
Running PIPEWORK cmd: pipework pon1_134 -i veth34 -l l18 cord-tester1 192.168.100.18/24
Running PIPEWORK cmd: pipework pon1_135 -i veth36 -l l19 cord-tester1 192.168.100.19/24
Running PIPEWORK cmd: pipework pon1_136 -i veth38 -l l20 cord-tester1 192.168.100.20/24
Running PIPEWORK cmd: pipework pon1_137 -i veth40 -l l21 cord-tester1 192.168.100.21/24
Running PIPEWORK cmd: pipework pon1_138 -i veth42 -l l22 cord-tester1 192.168.100.22/24
Modifying scapy tool files before running a test: ['voltha:voltha_exchange.test_two_subscribers_with_voltha_for_igmp_multiple_times_toggling_olt_verifying_traffic']
Running tests: ['voltha:voltha_exchange.test_two_subscribers_with_voltha_for_igmp_multiple_times_toggling_olt_verifying_traffic']
WARNING: No route found for IPv6 destination :: (no default route?)
Connecting to controller at 172.17.0.2
onoscli: Trying to connect to 172.17.0.2
Spawning pexpect for ip 172.17.0.2
ssh connection asked for password, gave password
Command 'summary -j' sent to onoscli.
onoscli: eof exception found
onoscli:     logout

Connection to 172.17.0.2 closed.

Installing the multi table app /root/test/src/test/voltha/../apps/ciena-cordigmp-multitable-3.0-SNAPSHOT.oar for subscriber test
Test Method: ... Enabling ponsim olt
Pre-provisioning ponsim_olt with address 172.17.0.1:50060
Enabling device 9d3fca92ec26
Checking operational status for device 9d3fca92ec26
Installing OLT app
Adding subscribers through OLT app
172.17.0.4 - - [01/Aug/2017 10:39:57] "POST /RPC2 HTTP/1.1" 200 -
Adding group 225.0.0.1
Adding group 225.0.0.1
All subscribers have joined the channel
Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
All subscribers have joined the channel
Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
Subscriber on port veth4 timed out
Subscriber on port veth0 timed out
Joining channel 0 for subscriber port veth4
Restarting olt or onu device 9d3fca92ec26
All subscribers have joined the channel
Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
Joining channel 0 for subscriber port veth0
All subscribers have joined the channel
Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
Subscriber on port veth4 timed out
Restarting olt or onu device 9d3fca92ec26
Subscriber on port veth0 timed out
Joining channel 0 for subscriber port veth4
All subscribers have joined the channel
Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
Joining channel 0 for subscriber port veth0
Restarting olt or onu device 9d3fca92ec26
All subscribers have joined the channel
Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
Subscriber on port veth4 timed out
Subscriber on port veth0 timed out
Joining channel 0 for subscriber port veth4
Restarting olt or onu device 9d3fca92ec26
All subscribers have joined the channel
Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
Joining channel 0 for subscriber port veth0
All subscribers have joined the channel
Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
Subscriber on port veth4 timed out
Restarting olt or onu device 9d3fca92ec26
Subscriber on port veth0 timed out
Joining channel 0 for subscriber port veth4
All subscribers have joined the channel
Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
Joining channel 0 for subscriber port veth0
Restarting olt or onu device 9d3fca92ec26
All subscribers have joined the channel
Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
Subscriber on port veth4 timed out
Subscriber on port veth0 timed out
Joining channel 0 for subscriber port veth4
Restarting olt or onu device 9d3fca92ec26
All subscribers have joined the channel
Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
Joining channel 0 for subscriber port veth0
All subscribers have joined the channel
Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
Subscriber on port veth4 timed out
Restarting olt or onu device 9d3fca92ec26
Subscriber on port veth0 timed out
Joining channel 0 for subscriber port veth4
All subscribers have joined the channel
Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
Joining channel 0 for subscriber port veth0
Restarting olt or onu device 9d3fca92ec26
All subscribers have joined the channel
Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
Subscriber on port veth4 timed out
Subscriber on port veth0 timed out
Joining channel 0 for subscriber port veth4
Restarting olt or onu device 9d3fca92ec26
All subscribers have joined the channel
Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
Joining channel 0 for subscriber port veth0
All subscribers have joined the channel
Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
Subscriber on port veth4 timed out
Subscriber on port veth0 timed out
Joining channel 0 for subscriber port veth4
All subscribers have joined the channel
Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
Joining channel 0 for subscriber port veth0
All subscribers have joined the channel
Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
Subscriber on port veth4 timed out
Subscriber on port veth0 timed out
Joining channel 0 for subscriber port veth4
All subscribers have joined the channel
Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
Joining channel 0 for subscriber port veth0
All subscribers have joined the channel
Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
Subscriber on port veth4 timed out
Subscriber on port veth0 timed out
Joining channel 0 for subscriber port veth4
All subscribers have joined the channel
Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
Joining channel 0 for subscriber port veth0
All subscribers have joined the channel
Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
Subscriber on port veth4 timed out
Subscriber on port veth0 timed out
Joining channel 0 for subscriber port veth4
This service is failed and other services will not run for this subscriber
This Subscriber is tested for multiple service eligibility
Joining channel 0 for subscriber port veth0
This service is failed and other services will not run for this subscriber
This Subscriber is tested for multiple service eligibility
Deleted subscriber for device of:0000000000000001 on port  128
Deleted subscriber for device of:0000000000000001 on port  129
Deleted subscriber for device of:0000000000000001 on port  130
Deleted subscriber for device of:0000000000000001 on port  131
Deleted subscriber for device of:0000000000000001 on port  132
Deleted subscriber for device of:0000000000000001 on port  133
Deleted subscriber for device of:0000000000000001 on port  134
Deleted subscriber for device of:0000000000000001 on port  135
Deleted subscriber for device of:0000000000000001 on port  136
Deleted subscriber for device of:0000000000000001 on port  137
Deleted subscriber for device of:0000000000000001 on port  138
Disabling device 9d3fca92ec26
Deleting device 9d3fca92ec26
Uninstalling OLT app
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/root/test/src/test/voltha/volthaTest.py", line 1444, in voltha_subscribers
    assert_equal(test_status, True)
  File "/usr/lib/python2.7/unittest/case.py", line 515, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python2.7/unittest/case.py", line 508, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: False != True

Unhandled Error
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/local/lib/python2.7/dist-packages/nose-1.3.7-py2.7.egg/nose/twistedtools.py", line 57, in <lambda>
    installSignalHandlers=False))
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1192, in run
    self.mainLoop()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1201, in mainLoop
    self.runUntilCurrent()
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 824, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/root/test/src/test/voltha/volthaTest.py", line 5305, in igmp_flow_check_operating_olt_admin_restart
    assert_equal(self.success, True)
  File "/usr/lib/python2.7/unittest/case.py", line 515, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python2.7/unittest/case.py", line 508, in _baseAssertEqual
    raise self.failureException(msg)
exceptions.AssertionError: False != True
FAIL
Installing back the cord igmp app /root/test/src/test/voltha/../apps/ciena-cordigmp-3.0-SNAPSHOT.oar for subscriber test on exit

======================================================================
FAIL: Test Method:
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/nose-1.3.7-py2.7.egg/nose/twistedtools.py", line 165, in wrapper
    % timeout)
TimeExpired: timeout expired before end of test (300.000000 s.)
-------------------- >> begin captured stdout << ---------------------
result.ok, result.status_code are True and 200

--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
scapy.runtime: WARNING: No route found for IPv6 destination :: (no default route?)
docker.auth.auth: DEBUG: Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
docker.auth.auth: DEBUG: No config file found
docker.auth.auth: DEBUG: Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
docker.auth.auth: DEBUG: No config file found
cordTester: INFO: Connecting to controller at 172.17.0.2
cordTester: INFO: onoscli: Trying to connect to 172.17.0.2
onoscli: INFO:
cordTester: INFO: Spawning pexpect for ip 172.17.0.2
cordTester: INFO: ssh connection asked for password, gave password
cordTester: INFO: Command 'summary -j' sent to onoscli.
cordTester: ERROR: onoscli: eof exception found
cordTester: ERROR: onoscli:     logout

Connection to 172.17.0.2 closed.

cordTester: INFO: Installing the multi table app /root/test/src/test/voltha/../apps/ciena-cordigmp-multitable-3.0-SNAPSHOT.oar for subscriber test
cordTester: INFO: Enabling ponsim olt
cordTester: INFO: Pre-provisioning ponsim_olt with address 172.17.0.1:50060
cordTester: INFO: Enabling device 9d3fca92ec26
cordTester: INFO: Checking operational status for device 9d3fca92ec26
cordTester: INFO: Installing OLT app
cordTester: INFO: Adding subscribers through OLT app
cordTester: DEBUG: Adding group 225.0.0.1
cordTester: DEBUG: Adding group 225.0.0.1
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Subscriber on port veth4 timed out
cordTester: INFO: Subscriber on port veth0 timed out
cordTester: INFO: Joining channel 0 for subscriber port veth4
cordTester: INFO: Restarting olt or onu device 9d3fca92ec26
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Joining channel 0 for subscriber port veth0
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Subscriber on port veth4 timed out
cordTester: INFO: Restarting olt or onu device 9d3fca92ec26
cordTester: INFO: Subscriber on port veth0 timed out
cordTester: INFO: Joining channel 0 for subscriber port veth4
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Joining channel 0 for subscriber port veth0
cordTester: INFO: Restarting olt or onu device 9d3fca92ec26
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Subscriber on port veth4 timed out
cordTester: INFO: Subscriber on port veth0 timed out
cordTester: INFO: Joining channel 0 for subscriber port veth4
cordTester: INFO: Restarting olt or onu device 9d3fca92ec26
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Joining channel 0 for subscriber port veth0
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Subscriber on port veth4 timed out
cordTester: INFO: Restarting olt or onu device 9d3fca92ec26
cordTester: INFO: Subscriber on port veth0 timed out
cordTester: INFO: Joining channel 0 for subscriber port veth4
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Joining channel 0 for subscriber port veth0
cordTester: INFO: Restarting olt or onu device 9d3fca92ec26
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Subscriber on port veth4 timed out
cordTester: INFO: Subscriber on port veth0 timed out
cordTester: INFO: Joining channel 0 for subscriber port veth4
cordTester: INFO: Restarting olt or onu device 9d3fca92ec26
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Joining channel 0 for subscriber port veth0
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Subscriber on port veth4 timed out
cordTester: INFO: Restarting olt or onu device 9d3fca92ec26
cordTester: INFO: Subscriber on port veth0 timed out
cordTester: INFO: Joining channel 0 for subscriber port veth4
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Joining channel 0 for subscriber port veth0
cordTester: INFO: Restarting olt or onu device 9d3fca92ec26
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Subscriber on port veth4 timed out
cordTester: INFO: Subscriber on port veth0 timed out
cordTester: INFO: Joining channel 0 for subscriber port veth4
cordTester: INFO: Restarting olt or onu device 9d3fca92ec26
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Joining channel 0 for subscriber port veth0
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Subscriber on port veth4 timed out
cordTester: INFO: Subscriber on port veth0 timed out
cordTester: INFO: Joining channel 0 for subscriber port veth4
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Joining channel 0 for subscriber port veth0
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Subscriber on port veth4 timed out
cordTester: INFO: Subscriber on port veth0 timed out
cordTester: INFO: Joining channel 0 for subscriber port veth4
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Joining channel 0 for subscriber port veth0
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Subscriber on port veth4 timed out
cordTester: INFO: Subscriber on port veth0 timed out
cordTester: INFO: Joining channel 0 for subscriber port veth4
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth4 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Joining channel 0 for subscriber port veth0
cordTester: INFO: All subscribers have joined the channel
cordTester: INFO: Subscriber on port veth0 checking data traffic receiving from group 225.0.0.1, channel 0
cordTester: INFO: Subscriber on port veth4 timed out
cordTester: INFO: Subscriber on port veth0 timed out
cordTester: INFO: Joining channel 0 for subscriber port veth4
cordTester: INFO: This service is failed and other services will not run for this subscriber
cordTester: INFO: This Subscriber is tested for multiple service eligibility
cordTester: INFO: Joining channel 0 for subscriber port veth0
cordTester: INFO: This service is failed and other services will not run for this subscriber
cordTester: INFO: This Subscriber is tested for multiple service eligibility
cordTester: INFO: Deleted subscriber for device of:0000000000000001 on port  128
cordTester: INFO: Deleted subscriber for device of:0000000000000001 on port  129
cordTester: INFO: Deleted subscriber for device of:0000000000000001 on port  130
cordTester: INFO: Deleted subscriber for device of:0000000000000001 on port  131
cordTester: INFO: Deleted subscriber for device of:0000000000000001 on port  132
cordTester: INFO: Deleted subscriber for device of:0000000000000001 on port  133
cordTester: INFO: Deleted subscriber for device of:0000000000000001 on port  134
cordTester: INFO: Deleted subscriber for device of:0000000000000001 on port  135
cordTester: INFO: Deleted subscriber for device of:0000000000000001 on port  136
cordTester: INFO: Deleted subscriber for device of:0000000000000001 on port  137
cordTester: INFO: Deleted subscriber for device of:0000000000000001 on port  138
cordTester: INFO: Disabling device 9d3fca92ec26
cordTester: INFO: Deleting device 9d3fca92ec26
cordTester: INFO: Uninstalling OLT app
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 1 test in 324.012s

FAILED (failures=1)
Test volthaTest.py:voltha_exchange.test_two_subscribers_with_voltha_for_igmp_multiple_times_toggling_olt_verifying_traffic Failure
Done running tests
Removing test container cord-tester1
ubuntu@cord-rs-s8:~/cord-tester/src/test/setup$