Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 1 | # Copyright 2017-present Radisys Corporation |
| 2 | # |
| 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
| 4 | # you may not use this file except in compliance with the License. |
| 5 | # You may obtain a copy of the License at |
| 6 | # |
| 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
| 8 | # |
| 9 | # Unless required by applicable law or agreed to in writing, software |
| 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
| 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 12 | # See the License for the specific language governing permissions and |
| 13 | # limitations under the License. |
| 14 | |
| 15 | |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 16 | *** Settings *** |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 17 | Documentation Test suite for checking default maas,xos and onos containers and fabric switch default services and maas cli commands |
| 18 | Library OperatingSystem |
| 19 | Library ../cord-api/Framework/utils/onosUtils.py |
You Wang | e920610 | 2017-11-17 12:15:11 -0800 | [diff] [blame] | 20 | Library ../cord-api/Framework/utils/utils.py |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 21 | Resource ../cord-api/Framework/utils/utils.robot |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 22 | |
| 23 | *** Variables *** |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 24 | @{MAAS_SERVICE_STATUS} start/running is running |
| 25 | @{JUJU_SERVICE_STATUS} active is ready unknown |
| 26 | @{LXD_CONTAINER_STATUS} RUNNING |
| 27 | @{BOOT_RESOURCES_OUTPUT} ubuntu/trusty |
| 28 | ${FABRIC_SWITCH_PROMPT} \# |
| 29 | ${FABRIC_SWITCH_USER} root |
| 30 | ${FABRIC_SWITCH_PASSWD} onl |
| 31 | @{FABRIC_SERVICE_STATUS} is running |
| 32 | ${IP_PATTERN} (\\d+)\\.(\\d+)\\.(\\d+)\\.(\\d+) |
You Wang | e920610 | 2017-11-17 12:15:11 -0800 | [diff] [blame] | 33 | ${PUBLIC_IFACE} eth2 |
| 34 | ${NUM_OF_SWITCHES} 4 |
| 35 | ${CORD_PROFILE} rcord |
| 36 | ${DOCKER_CONTAINERS_FILE} ${CURDIR}/../diag/dockerContainers.json |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 37 | |
| 38 | *** Test Cases *** |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 39 | Verify Headnode Interfaces |
You Wang | e920610 | 2017-11-17 12:15:11 -0800 | [diff] [blame] | 40 | [Tags] fabric |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 41 | [Documentation] Verifies the headnode interface is up and has external connectivity |
| 42 | Verify HeadNode Interfaces Detected |
You Wang | e920610 | 2017-11-17 12:15:11 -0800 | [diff] [blame] | 43 | Test Ping ${PUBLIC_IFACE} www.opennetworking.org |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 44 | |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 45 | Get Compute Node and Fabric Info |
| 46 | [Documentation] Get all information pretaining to the compute nodes and fabric |
| 47 | ${nodes}= Create List |
| 48 | ${hostnames}= Create List |
| 49 | ${hostname_prefixes}= Create List |
| 50 | ${node_ips}= Create List |
| 51 | ${node_data_ips}= Create List |
| 52 | ${node_count} Run cord prov list | grep node | wc -l |
| 53 | ${node_count}= Convert To Integer ${node_count} |
| 54 | Log ${node_count} |
| 55 | ##Get hostname |
| 56 | : FOR ${INDEX} IN RANGE 1 ${node_count}+1 |
| 57 | \ ${hostname}= Run cord prov list | grep node | awk '{print $2}' | sed -n ${INDEX}p |
| 58 | \ Append To List ${hostnames} ${hostname} |
| 59 | ##Get hostname prefixes |
| 60 | : FOR ${INDEX} IN RANGE 0 ${node_count} |
| 61 | \ ${hostname_prefix}= Remove String ${hostnames[${INDEX}]} .cord.lab |
| 62 | \ Append To List ${hostname_prefixes} ${hostname_prefix} |
| 63 | ##Get compute node data ips |
| 64 | ${cordvtnnodes}= ONOS Command Execute onos-cord 8102 cordvtn-nodes | grep fabric |
| 65 | ${nds}= Split To Lines ${cordvtnnodes} |
| 66 | : FOR ${i} IN @{nds} |
| 67 | \ ${data_ip}= Get Compute Node IP ${i} |
| 68 | \ Append To List ${node_data_ips} ${data_ip} |
| 69 | ##Get compute node ips |
| 70 | : FOR ${i} IN @{hostname_prefixes} |
| 71 | \ ${node_ip}= Run cord harvest list | grep ${i} | awk '{print $4}' |
| 72 | \ Append To List ${node_ips} ${node_ip} |
| 73 | @{switch_ips}= Discover FABRIC IPs |
| 74 | Set Suite Variable ${switch_ips} |
| 75 | Set Suite Variable ${hostnames} |
| 76 | Set Suite Variable ${hostname_prefixes} |
| 77 | Set Suite Variable ${node_ips} |
| 78 | Set Suite Variable ${node_data_ips} |
| 79 | |
You Wang | e920610 | 2017-11-17 12:15:11 -0800 | [diff] [blame] | 80 | Verify Compute Nodes Pingability Through Fabric |
| 81 | [Documentation] Verifies that the two compute nodes can ping each other through the fabric |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 82 | [Tags] fabric |
| 83 | ##Verify pingablilty across compute nodes |
You Wang | e920610 | 2017-11-17 12:15:11 -0800 | [diff] [blame] | 84 | : FOR ${src} IN @{hostname_prefixes} |
| 85 | \ Ping All Compute Nodes Through Fabric ${src} |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 86 | |
| 87 | Verify Compute Nodes to Fabric Pingability |
| 88 | [Documentation] Verifies that the two compute nodes can ping the switches |
| 89 | [Tags] fabric |
| 90 | ##Verify pingability from compute nodes to fabric |
You Wang | e920610 | 2017-11-17 12:15:11 -0800 | [diff] [blame] | 91 | : FOR ${src} IN @{hostname_prefixes} |
| 92 | \ Ping All Fabric Switches ${src} |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 93 | |
| 94 | Verify CordVTN Nodes |
| 95 | [Documentation] Verifies that the cordvtn app running in onos identifies the nodes and devices (fabric) |
| 96 | ${nodes}= Execute ONOS Command onos-cord 8102 cordvtn-nodes |
| 97 | : FOR ${i} IN @{node_ips} |
| 98 | \ ${node_1}= Get Lines Containing String ${nodes} ${i} |
| 99 | \ Should Contain ${node_1} COMPLETE |
| 100 | \ Should Contain ${node_1} ${i} |
| 101 | ${ports}= Execute ONOS Command onos-cord 8102 cordvtn-ports |
| 102 | ${devices}= Execute ONOS Command onos-fabric 8101 devices |
| 103 | @{switch_ips}= Discover FABRIC IPs |
| 104 | : FOR ${i} IN @{switch_ips} |
| 105 | \ Should Contain ${devices} ${i} |
| 106 | |
| 107 | Verify MAAS Service State |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 108 | [Template] Verify MAAS Service |
| 109 | maas-dhcpd |
| 110 | maas-regiond |
| 111 | maas-clusterd |
| 112 | maas-proxy |
| 113 | bind9 |
| 114 | |
You Wang | e920610 | 2017-11-17 12:15:11 -0800 | [diff] [blame] | 115 | Verify Docker Containers State |
| 116 | ${dockerContainers} utils.jsonToList ${DOCKER_CONTAINERS_FILE} docker-containers-${CORD_PROFILE} |
| 117 | : FOR ${container} IN @{dockerContainers} |
| 118 | \ Verify Containers ${container} |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 119 | |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 120 | Verify Juju Services State |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 121 | [Template] Verify JUJU Service |
| 122 | ceilometer |
| 123 | ceilometer-agent |
| 124 | glance |
| 125 | keystone |
| 126 | mongodb |
| 127 | nagios |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 128 | neutron-api |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 129 | nova-cloud-controller |
| 130 | nova-compute |
| 131 | openstack-dashboard |
| 132 | percona-cluster |
| 133 | rabbitmq-server |
| 134 | |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 135 | Verify Openstack LXD Containers State |
| 136 | [Template] Verify Openstack LXD Containers |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 137 | ceilometer |
| 138 | glance |
| 139 | keystone |
| 140 | mongodb |
| 141 | nagios |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 142 | neutron-api |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 143 | nova-cloud-controller |
| 144 | openstack-dashboard |
| 145 | percona-cluster |
| 146 | rabbitmq-server |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 147 | |
| 148 | Verify MAAS CLI commands |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 149 | [Tags] notready |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 150 | Login MAAS Server |
| 151 | Verify MAAS CLI Commands boot-resources read | jq 'map(select(.type == "Synced"))' ubuntu/trusty |
You Wang | e920610 | 2017-11-17 12:15:11 -0800 | [diff] [blame] | 152 | Verify MAAS CLI Commands devices list | jq '.' | jq '.[]'.hostname | wc -l ${NUM_OF_SWITCHES} |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 153 | #Verify MAAS CLI Commands events query | jq '.' | jq .events[].id | wc -l 100 |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 154 | Verify MAAS CLI Commands fabrics read | jq '.' | jq .[].name | wc -l 4 |
| 155 | Verify MAAS CLI Commands networks read | jq '.' | jq .[].name | wc -l 4 |
| 156 | Verify MAAS CLI Commands node-groups list | jq '.' | jq .[].status | wc -l 1 |
| 157 | Verify MAAS CLI Commands subnets read | jq '.' | jq .[].name | wc -l 4 |
| 158 | Verify MAAS CLI Commands nodes list | jq '.' | jq .[].substatus_name | wc -l 1 |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 159 | Verify MAAS CLI Commands zones read | jq '.' | jq .[].name | wc -l 2 |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 160 | Logout MAAS Server |
| 161 | |
| 162 | Verify Fabric Switch Service |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 163 | [Tags] fabric |
| 164 | @{switch_ips}= Discover FABRIC IPs |
| 165 | : FOR ${i} IN @{switch_ips} |
| 166 | \ Verify Fabric Switch Service ${i} faultd |
| 167 | \ Verify Fabric Switch Service ${i} netplug |
| 168 | \ Verify Fabric Switch Service ${i} onlp-snmpd |
| 169 | \ Verify Fabric Switch Service ${i} onlpd |
| 170 | \ Verify Fabric Switch Service ${i} rsyslog |
| 171 | \ Verify Fabric Switch Service ${i} snmpd |
| 172 | \ Verify Fabric Switch Service ${i} ssh |
| 173 | \ Verify Fabric Switch Service ${i} udev |
| 174 | \ Verify Fabric Switch Service ${i} watchdog |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 175 | |
| 176 | *** Keywords *** |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 177 | Verify HeadNode Interfaces Detected |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 178 | ${cmd}= Catenate SEPARATOR=| sudo ethtool mgmtbr grep 'Link detected:' awk '{ print $3 }' |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 179 | ${output}= Run ${cmd} |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 180 | Should Contain ${output} yes msg= mgmtbr is not detected !!!. Reason: |
| 181 | ${cmd}= Catenate SEPARATOR=| sudo ethtool fabric grep 'Link detected:' awk '{ print $3 }' |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 182 | ${output}= Run ${cmd} |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 183 | Should Contain ${output} yes msg= fabric interface is not detected !!!. Reason: |
| 184 | |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 185 | Verify Containers |
| 186 | [Arguments] ${name} |
| 187 | ${container_id}= Get Docker Container ID ${name} |
| 188 | ${output}= Run docker inspect --format="{{ .State.Running }}" ${container_id} |
| 189 | Should Contain ${output} true msg=${name} is not running !!!. Reason: |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 190 | |
| 191 | Verify MAAS Service |
| 192 | [Arguments] ${name} |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 193 | ${cmd}= Catenate sudo service ${name} status |
| 194 | ${output}= Run ${cmd} |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 195 | Should Contain Any ${output} @{MAAS_SERVICE_STATUS} msg= ${name} is not running !!!. Reason: |
| 196 | |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 197 | Verify JUJU Service |
| 198 | [Arguments] ${name} |
| 199 | ${cmd} Catenate SEPARATOR=| juju status --format=tabular grep -v grep grep ${name}/0 awk '{ print $2,$7,$8,$9,$10}' |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 200 | ${output}= Run ${cmd} |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 201 | Should Contain Any ${output} @{JUJU_SERVICE_STATUS} msg= ${name} is not running !!!. Reason: |
| 202 | |
| 203 | Verify Openstack LXD Containers |
| 204 | [Arguments] ${name} |
| 205 | ${cmd} Catenate SEPARATOR=| sudo lxc list grep -v grep grep ${name}-1 awk '{ print $2,$4 }' |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 206 | ${output}= Run ${cmd} |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 207 | Should Contain Any ${output} @{LXD_CONTAINER_STATUS} msg= ${name} is not running !!!. Reason: |
| 208 | |
| 209 | Verify MAAS CLI Commands |
| 210 | [Arguments] ${name} ${expected} |
| 211 | ${cmd} Catenate maas cord ${name} |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 212 | ${output}= Run ${cmd} |
| 213 | Should Contain ${output} ${expected} msg=Reason: |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 214 | |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 215 | Login MAAS Server |
| 216 | ${cmd} Catenate maas login cord http://localhost/MAAS/api/1.0 $(sudo maas-region-admin apikey --user=cord) |
| 217 | ${output}= Run ${cmd} |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 218 | Should Contain ${output} You are now logged in to the MAAS msg= MAAS login failure !!!. Reason: |
| 219 | |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 220 | Logout MAAS Server |
| 221 | ${cmd} Catenate maas logout cord |
| 222 | ${output}= Run ${cmd} |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 223 | |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 224 | Discover FABRIC IPs |
| 225 | ${switches}= Run cord prov list | grep fabric | awk '{print $4}' |
| 226 | @{switch_ips}= Split To Lines ${switches} |
| 227 | [Return] ${switch_ips} |
Gunaseelan | ed0eb1b | 2017-08-08 08:12:12 -0500 | [diff] [blame] | 228 | |
| 229 | Verify Fabric Switch Service |
| 230 | [Arguments] ${ip} ${name} |
Kailash Khalasi | 2f567a4 | 2017-09-27 13:50:05 -0700 | [diff] [blame] | 231 | ${cmd}= Catenate service ${name} status |
| 232 | ${output}= Run Command On Remote System ${ip} ${cmd} ${FABRIC_SWITCH_USER} ${FABRIC_SWITCH_PASSWD} ${FABRIC_SWITCH_PROMPT} 60s False |
You Wang | e920610 | 2017-11-17 12:15:11 -0800 | [diff] [blame] | 233 | Should Contain Any ${output} @{FABRIC_SERVICE_STATUS} msg= ${name} is not running !!!. Reason: |
| 234 | |
| 235 | Ping All Compute Nodes Through Fabric |
| 236 | [Arguments] ${src_ip} |
| 237 | : FOR ${dst_ip} IN @{node_data_ips} |
| 238 | \ Verify Ping ubuntu ${src_ip} ${dst_ip} |
| 239 | |
| 240 | Ping All Fabric Switches |
| 241 | [Arguments] ${src_ip} |
| 242 | : FOR ${dst_ip} IN @{switch_ips} |
| 243 | \ Verify Ping ubuntu ${src_ip} ${dst_ip} |
| 244 | |
| 245 | Verify Ping |
| 246 | [Arguments] ${srcName} ${srcIP} ${dst} |
| 247 | ${result}= Run ssh ${srcName}@${srcIP} "ping -c 3 ${dst}" |
| 248 | Should Contain ${result} 64 bytes |
| 249 | Should Not Contain ${result} Destination Host Unreachable |