blob: d7aeb483bf9f12aa77d7ca5d5367b4004a4036a3 [file] [log] [blame]
Dan Talayco48370102010-03-03 15:17:33 -08001#!/usr/bin/env python
2"""
3@package oft
4
5OpenFlow test framework top level script
6
7This script is the entry point for running OpenFlow tests
8using the OFT framework.
9
10The global configuration is passed around in a dictionary
Brandon Heller88f709d2010-04-01 12:29:56 -070011generally called config. The keys have the following
Dan Talayco48370102010-03-03 15:17:33 -080012significance.
13
Dan Talayco2c0dba32010-03-06 22:47:06 -080014<pre>
Dan Talayco48370102010-03-03 15:17:33 -080015 platform : String identifying the target platform
16 controller_host : Host on which test controller is running (for sockets)
17 controller_port : Port on which test controller listens for switch cxn
Dan Talayco2c0dba32010-03-06 22:47:06 -080018 test_dir : (TBD) Directory to search for test files (default .)
Dan Talayco48370102010-03-03 15:17:33 -080019 test_spec : (TBD) Specification of test(s) to run
20 log_file : Filename for test logging
Dan Talayco2c0dba32010-03-06 22:47:06 -080021 list : Boolean: List all tests and exit
Dan Talayco48370102010-03-03 15:17:33 -080022 debug : String giving debug level (info, warning, error...)
Dan Talayco2c0dba32010-03-06 22:47:06 -080023</pre>
Dan Talayco48370102010-03-03 15:17:33 -080024
25See config_defaults below for the default values.
26
Dan Talayco2c0dba32010-03-06 22:47:06 -080027The following are stored in the config dictionary, but are not currently
28configurable through the command line.
29
30<pre>
31 dbg_level : logging module value of debug level
32 port_map : Map of dataplane OpenFlow port to OS interface names
Dan Talayco2c0dba32010-03-06 22:47:06 -080033</pre>
34
Dan Talaycoc24aaae2010-07-08 14:05:24 -070035Each test may be assigned a priority by setting test_prio["TestName"] in
36the respective module. For now, the only use of this is to avoid
37automatic inclusion of tests into the default list. This is done by
38setting the test_prio value less than 0. Eventually we may add ordering
39of test execution by test priority.
40
Dan Talayco2c0dba32010-03-06 22:47:06 -080041To add a test to the system, either: edit an existing test case file (like
42basic.py) to add a test class which inherits from unittest.TestCase (directly
43or indirectly); or add a new file which includes a function definition
44test_set_init(config). Preferably the file is in the same directory as existing
45tests, though you can specify the directory on the command line. The file
46should not be called "all" as that's reserved for the test-spec.
47
48If you add a new file, the test_set_init function should record the port
49map object from the configuration along with whatever other configuration
50information it may need.
51
52TBD: To add configuration to the system, first add an entry to config_default
Dan Talayco48370102010-03-03 15:17:33 -080053below. If you want this to be a command line parameter, edit config_setup
54to add the option and default value to the parser. Then edit config_get
55to make sure the option value gets copied into the configuration
56structure (which then gets passed to everyone else).
57
58By convention, oft attempts to import the contents of a file by the
59name of $platform.py into the local namespace.
60
61IMPORTANT: That file should define a function platform_config_update which
62takes a configuration dictionary as an argument and updates it for the
63current run. In particular, it should set up config["port_map"] with
64the proper map from OF port numbers to OF interface names.
65
Rich Lane8aebc5e2012-09-25 17:57:53 -070066You can add your own platform, say gp104, by adding a file gp104.py to the
67platforms directory that defines the function platform_config_update and then
68use the parameter --platform=gp104 on the command line. You can also use the
69--platform-dir option to change which directory is searched.
Dan Talayco48370102010-03-03 15:17:33 -080070
Dan Talayco48370102010-03-03 15:17:33 -080071The current model for test sets is basic.py. The current convention is
72that the test set should implement a function test_set_init which takes
73an oft configuration dictionary and returns a unittest.TestSuite object.
74Future test sets should do the same thing.
75
Dan Talayco52f64442010-03-03 15:32:41 -080076Default setup:
77
78The default setup runs locally using veth pairs. To exercise this,
79checkout and build an openflow userspace datapath. Then start it on
80the local host:
Dan Talayco2c0dba32010-03-06 22:47:06 -080081<pre>
Dan Talayco52f64442010-03-03 15:32:41 -080082 sudo ~/openflow/regress/bin/veth_setup.pl
83 sudo ofdatapath -i veth0,veth2,veth4,veth6 punix:/tmp/ofd &
84 sudo ofprotocol unix:/tmp/ofd tcp:127.0.0.1 --fail=closed --max-backoff=1 &
85
86Next, run oft:
87 sudo ./oft --debug=info
Dan Talayco2c0dba32010-03-06 22:47:06 -080088</pre>
Dan Talayco52f64442010-03-03 15:32:41 -080089
90Examine oft.log if things don't work.
Dan Talayco2c0dba32010-03-06 22:47:06 -080091
Dan Talayco1a88c122010-03-07 22:00:20 -080092@todo Support per-component debug levels (esp controller vs dataplane)
Rich Lane15f64de2012-10-04 21:25:57 -070093@todo Allow specification of priority to override prio check
Dan Talayco2c0dba32010-03-06 22:47:06 -080094
Dan Talayco1a88c122010-03-07 22:00:20 -080095Current test case setup:
Rich Lane8aebc5e2012-09-25 17:57:53 -070096 Files in the tests direcoty that contain a function test_set_init are
97considered test files.
Dan Talayco2c0dba32010-03-06 22:47:06 -080098 The function test_set_init examines the test_spec config variable
99and generates a suite of tests.
100 Support a command line option --test_mod so that all tests in that
101module will be run.
102 Support all to specify all tests from the module.
103
Dan Talayco48370102010-03-03 15:17:33 -0800104"""
105
106import sys
107from optparse import OptionParser
Dan Talayco2c0dba32010-03-06 22:47:06 -0800108from subprocess import Popen,PIPE
Dan Talayco48370102010-03-03 15:17:33 -0800109import logging
110import unittest
Dan Talayco2c0dba32010-03-06 22:47:06 -0800111import time
Brandon Heller446c1432010-04-01 12:43:27 -0700112import os
Rich Lane6b452bc2012-07-09 16:52:21 -0700113import imp
Rich Lane8592bec2012-09-03 09:06:59 -0700114import random
Rich Lane5bd6cf92012-10-04 17:57:24 -0700115import signal
Rich Lane943be672012-10-04 19:20:16 -0700116import fnmatch
Dan Talayco48370102010-03-03 15:17:33 -0800117
Rich Lanefadf3452012-10-03 16:23:37 -0700118root_dir = os.path.dirname(os.path.realpath(__file__))
119
120pydir = os.path.join(root_dir, 'src', 'python')
Rich Lane39878042012-07-09 14:45:35 -0700121if os.path.exists(os.path.join(pydir, 'oftest')):
122 # Running from source tree
123 sys.path.insert(0, pydir)
124
Rich Laneb0470142012-10-04 15:50:35 -0700125try:
126 import oftest.message
127except:
128 sys.exit("Missing OpenFlow message classes: please run \"make -C tools/munger\"")
129
Rich Laneda3b5ad2012-10-03 09:05:32 -0700130import oftest.testutils
Rich Lanee55abf72012-07-26 20:11:42 -0700131import oftest.ofutils
Dan Talaycoba3745c2010-07-21 21:51:08 -0700132
Dan Talayco02eca0b2010-04-15 16:09:43 -0700133try:
134 import scapy.all as scapy
135except:
136 try:
137 import scapy as scapy
138 except:
139 sys.exit("Need to install scapy for packet parsing")
140
Dan Talaycod7c80d12012-04-03 15:20:57 -0700141##@var Profile module
142profile_mod = None
143
Dan Talayco48370102010-03-03 15:17:33 -0800144##@var DEBUG_LEVELS
145# Map from strings to debugging levels
146DEBUG_LEVELS = {
147 'debug' : logging.DEBUG,
148 'verbose' : logging.DEBUG,
149 'info' : logging.INFO,
150 'warning' : logging.WARNING,
151 'warn' : logging.WARNING,
152 'error' : logging.ERROR,
153 'critical' : logging.CRITICAL
154}
155
156_debug_default = "warning"
157_debug_level_default = DEBUG_LEVELS[_debug_default]
158
159##@var config_default
160# The default configuration dictionary for OFT
161config_default = {
Dan Talayco551befa2010-07-15 17:05:32 -0700162 "param" : None,
Dan Talayco48370102010-03-03 15:17:33 -0800163 "platform" : "local",
Rich Lane941d1dd2012-07-30 14:27:53 -0700164 "platform_args" : None,
Rich Lane1aeccc42012-07-29 17:58:10 -0700165 "controller_host" : "0.0.0.0",
Dan Talayco48370102010-03-03 15:17:33 -0800166 "controller_port" : 6633,
Dan Talaycocf26b7a2011-08-05 10:15:35 -0700167 "relax" : False,
Dan Talayco2c0dba32010-03-06 22:47:06 -0800168 "test_spec" : "all",
Dan Talayco48370102010-03-03 15:17:33 -0800169 "log_file" : "oft.log",
Dan Talayco2c0dba32010-03-06 22:47:06 -0800170 "list" : False,
Jeffrey Townsend6614f972012-08-16 13:27:18 -0700171 "list_test_names" : False,
Dan Talayco48370102010-03-03 15:17:33 -0800172 "debug" : _debug_default,
173 "dbg_level" : _debug_level_default,
Dan Talaycoac25cf32010-07-20 14:08:28 -0700174 "port_map" : {},
Dan Talaycod7c80d12012-04-03 15:20:57 -0700175 "test_params" : "None",
Rich Lane1673b8f2012-09-25 18:35:18 -0700176 "profile" : "default",
Rich Laneee57ad02012-07-13 15:40:36 -0700177 "allow_user" : False,
Rich Lane9a84a4f2012-07-17 12:27:42 -0700178 "fail_skipped" : False,
Rich Lanee55abf72012-07-26 20:11:42 -0700179 "default_timeout" : 2,
Jeffrey Townsend5cb7ed32012-08-17 18:11:01 +0000180 "minsize" : 0,
Rich Lane8592bec2012-09-03 09:06:59 -0700181 "random_seed" : None,
Rich Lanee284b6b2012-10-03 09:19:58 -0700182 "test_dir" : os.path.join(root_dir, "tests"),
Rich Lane8aebc5e2012-09-25 17:57:53 -0700183 "platform_dir" : os.path.join(root_dir, "platforms"),
Rich Lane1673b8f2012-09-25 18:35:18 -0700184 "profile_dir" : os.path.join(root_dir, "profiles"),
Dan Talayco48370102010-03-03 15:17:33 -0800185}
186
Dan Talaycoc24aaae2010-07-08 14:05:24 -0700187# Default test priority
188TEST_PRIO_DEFAULT=100
Dan Talaycod7c80d12012-04-03 15:20:57 -0700189TEST_PRIO_SKIP=-1
Dan Talaycoc24aaae2010-07-08 14:05:24 -0700190
Dan Talayco1a88c122010-03-07 22:00:20 -0800191#@todo Set up a dict of config params so easier to manage:
192# <param> <cmdline flags> <default value> <help> <optional parser>
193
Dan Talayco48370102010-03-03 15:17:33 -0800194# Map options to config structure
195def config_get(opts):
196 "Convert options class to OFT configuration dictionary"
197 cfg = config_default.copy()
Dan Talayco2c0dba32010-03-06 22:47:06 -0800198 for key in cfg.keys():
Rich Lanea92f2522012-10-04 18:11:04 -0700199 cfg[key] = getattr(opts, key)
Dan Talayco2c0dba32010-03-06 22:47:06 -0800200
201 # Special case checks
Dan Talayco48370102010-03-03 15:17:33 -0800202 if opts.debug not in DEBUG_LEVELS.keys():
203 print "Warning: Bad value specified for debug level; using default"
204 opts.debug = _debug_default
Dan Talayco02eca0b2010-04-15 16:09:43 -0700205 if opts.verbose:
206 cfg["debug"] = "verbose"
Dan Talayco48370102010-03-03 15:17:33 -0800207 cfg["dbg_level"] = DEBUG_LEVELS[cfg["debug"]]
Dan Talayco2c0dba32010-03-06 22:47:06 -0800208
Dan Talayco48370102010-03-03 15:17:33 -0800209 return cfg
210
211def config_setup(cfg_dflt):
212 """
213 Set up the configuration including parsing the arguments
214
215 @param cfg_dflt The default configuration dictionary
216 @return A pair (config, args) where config is an config
217 object and args is any additional arguments from the command line
218 """
219
220 parser = OptionParser(version="%prog 0.1")
221
Dan Talayco2c0dba32010-03-06 22:47:06 -0800222 #@todo parse port map as option?
Dan Talayco48370102010-03-03 15:17:33 -0800223 # Set up default values
Rich Lanea92f2522012-10-04 18:11:04 -0700224 parser.set_defaults(**cfg_dflt)
Dan Talayco48370102010-03-03 15:17:33 -0800225
Dan Talayco2c0dba32010-03-06 22:47:06 -0800226 #@todo Add options via dictionary
Dan Talayco48370102010-03-03 15:17:33 -0800227 plat_help = """Set the platform type. Valid values include:
228 local: User space virtual ethernet pair setup
229 remote: Remote embedded Broadcom based switch
Dan Talayco673e0852010-03-06 23:09:23 -0800230 Create a new_plat.py file and use --platform=new_plat on the command line
Dan Talayco48370102010-03-03 15:17:33 -0800231 """
Rich Lane941d1dd2012-07-30 14:27:53 -0700232 parser.add_option("-a", "--platform-args", help="Custom arguments per platform.")
Dan Talayco48370102010-03-03 15:17:33 -0800233 parser.add_option("-P", "--platform", help=plat_help)
234 parser.add_option("-H", "--host", dest="controller_host",
235 help="The IP/name of the test controller host")
236 parser.add_option("-p", "--port", dest="controller_port",
237 type="int", help="Port number of the test controller")
Dan Talayco673e0852010-03-06 23:09:23 -0800238 test_list_help = """Indicate tests to run. Valid entries are "all" (the
Dan Talaycocfa172f2012-03-23 12:03:00 -0700239 default) or a comma separated list of:
240 module Run all tests in the named module
Dan Talayco673e0852010-03-06 23:09:23 -0800241 testcase Run tests in all modules with the name testcase
242 module.testcase Run the specific test case
243 """
Rich Lanef261a102012-07-25 13:41:38 -0700244 parser.add_option("-T", "--test-spec", "--test-list", help=test_list_help)
Dan Talayco48370102010-03-03 15:17:33 -0800245 parser.add_option("--log-file",
246 help="Name of log file, empty string to log to console")
247 parser.add_option("--debug",
248 help="Debug lvl: debug, info, warning, error, critical")
Dan Talayco02eca0b2010-04-15 16:09:43 -0700249 parser.add_option("--port-count", type="int",
Dan Talayco48370102010-03-03 15:17:33 -0800250 help="Number of ports to use (optional)")
Dan Talayco02eca0b2010-04-15 16:09:43 -0700251 parser.add_option("--base-of-port", type="int",
Dan Talayco48370102010-03-03 15:17:33 -0800252 help="Base OpenFlow port number (optional)")
Dan Talayco02eca0b2010-04-15 16:09:43 -0700253 parser.add_option("--base-if-index", type="int",
Dan Talayco2c0dba32010-03-06 22:47:06 -0800254 help="Base interface index number (optional)")
Jeffrey Townsend6614f972012-08-16 13:27:18 -0700255 parser.add_option("--list-test-names", action='store_true',
256 help="List only test names.", default=False)
Dan Talayco2c0dba32010-03-06 22:47:06 -0800257 parser.add_option("--list", action="store_true",
Brandon Heller824504e2010-04-01 12:21:37 -0700258 help="List all tests and exit")
Dan Talayco02eca0b2010-04-15 16:09:43 -0700259 parser.add_option("--verbose", action="store_true",
260 help="Short cut for --debug=verbose")
Dan Talaycocf26b7a2011-08-05 10:15:35 -0700261 parser.add_option("--relax", action="store_true",
262 help="Relax packet match checks allowing other packets")
Dan Talayco551befa2010-07-15 17:05:32 -0700263 parser.add_option("--param", type="int",
264 help="Parameter sent to test (for debugging)")
Dan Talaycod7c80d12012-04-03 15:20:57 -0700265 parser.add_option("--profile",
266 help="File listing tests to skip/run")
Dan Talaycoac25cf32010-07-20 14:08:28 -0700267 parser.add_option("-t", "--test-params",
Dan Talaycof6e76c02012-03-23 10:56:12 -0700268 help="""Set test parameters: key=val;...
269 NOTE: key MUST be a valid Python identifier, egr_count not egr-count
270 See --list""")
Rich Laneee57ad02012-07-13 15:40:36 -0700271 parser.add_option("--allow-user", action="store_true",
272 help="Proceed even if oftest is not run as root")
Rich Lane9a84a4f2012-07-17 12:27:42 -0700273 parser.add_option("--fail-skipped", action="store_true",
274 help="Return failure if any test was skipped")
Rich Lanee55abf72012-07-26 20:11:42 -0700275 parser.add_option("--default-timeout", type="int",
276 help="Timeout in seconds for most operations")
Jeffrey Townsend5cb7ed32012-08-17 18:11:01 +0000277 parser.add_option("--minsize", type="int",
278 help="Minimum allowable packet size on the dataplane.",
279 default=0)
Rich Lane8592bec2012-09-03 09:06:59 -0700280 parser.add_option("--random-seed", type="int",
281 help="Random number generator seed",
282 default=None)
Rich Lanebc3d2962012-09-25 09:34:17 -0700283 parser.add_option("--test-dir", type="string",
284 help="Directory containing tests")
Rich Lane8aebc5e2012-09-25 17:57:53 -0700285 parser.add_option("--platform-dir", type="string",
286 help="Directory containing platform modules")
Rich Lane1673b8f2012-09-25 18:35:18 -0700287 parser.add_option("--profile-dir", type="string",
288 help="Directory containing profile modules")
Jeffrey Townsend5cb7ed32012-08-17 18:11:01 +0000289
Dan Talayco48370102010-03-03 15:17:33 -0800290 # Might need this if other parsers want command line
291 # parser.allow_interspersed_args = False
292 (options, args) = parser.parse_args()
293
294 config = config_get(options)
295
296 return (config, args)
297
Rich Lane1673b8f2012-09-25 18:35:18 -0700298def load_profile(config):
Dan Talaycod15bed52012-04-04 10:39:52 -0700299 """
300 Import a profile from the profiles library
301 """
302
Dan Talaycod7c80d12012-04-03 15:20:57 -0700303 global profile_mod
Rich Lane1673b8f2012-09-25 18:35:18 -0700304 logging.info("Importing profile: %s" % config["profile"])
305 try:
306 profile_mod = imp.load_module(config["profile"], *imp.find_module(config["profile"], [config["profile_dir"]]))
307 if not "skip_test_list" in dir(profile_mod):
308 die("Profile did not define skip_test_list")
309 except:
310 logging.info("Could not import profile: %s.py" % config["profile"])
311 print "Failed to import profile: %s" % config["profile"]
312 raise
Dan Talaycod7c80d12012-04-03 15:20:57 -0700313
Dan Talayco48370102010-03-03 15:17:33 -0800314def logging_setup(config):
315 """
316 Set up logging based on config
317 """
318 _format = "%(asctime)s %(name)-10s: %(levelname)-8s: %(message)s"
319 _datefmt = "%H:%M:%S"
Dan Talayco88fc8802010-03-07 11:37:52 -0800320 logging.basicConfig(filename=config["log_file"],
321 level=config["dbg_level"],
322 format=_format, datefmt=_datefmt)
Dan Talayco48370102010-03-03 15:17:33 -0800323
Rich Lane943be672012-10-04 19:20:16 -0700324def load_test_modules(config):
325 """
326 Load tests from the test_dir directory.
Dan Talayco2c0dba32010-03-06 22:47:06 -0800327
Rich Lane943be672012-10-04 19:20:16 -0700328 Test cases are subclasses of unittest.TestCase
Dan Talayco2c0dba32010-03-06 22:47:06 -0800329
330 @param config The oft configuration dictionary
Rich Lane943be672012-10-04 19:20:16 -0700331 @returns A dictionary from test module names to tuples of
332 (module, dictionary from test names to test classes).
Dan Talayco2c0dba32010-03-06 22:47:06 -0800333 """
334
Rich Lane943be672012-10-04 19:20:16 -0700335 result = {}
Dan Talayco2c0dba32010-03-06 22:47:06 -0800336
Rich Lane943be672012-10-04 19:20:16 -0700337 for root, dirs, filenames in os.walk(config["test_dir"]):
338 # Iterate over each python file
339 for filename in fnmatch.filter(filenames, '[!.]*.py'):
340 modname = os.path.splitext(os.path.basename(filename))[0]
Dan Talayco2c0dba32010-03-06 22:47:06 -0800341
Rich Lane943be672012-10-04 19:20:16 -0700342 try:
343 if sys.modules.has_key(modname):
344 mod = sys.modules[modname]
345 else:
346 mod = imp.load_module(modname, *imp.find_module(modname, [root]))
347 except:
348 logging.warning("Could not import file " + filename)
349 raise
Rich Lane520e4152012-07-09 16:18:16 -0700350
Rich Lane943be672012-10-04 19:20:16 -0700351 # Find all testcases defined in the module
352 tests = dict((k, v) for (k, v) in mod.__dict__.items() if type(v) == type and
353 issubclass(v, unittest.TestCase))
354 if tests:
355 result[modname] = (mod, tests)
Rich Lane520e4152012-07-09 16:18:16 -0700356
Rich Lane943be672012-10-04 19:20:16 -0700357 return result
Dan Talayco2c0dba32010-03-06 22:47:06 -0800358
Rich Lane15f64de2012-10-04 21:25:57 -0700359def prune_tests(test_spec, test_modules):
360 """
361 Return tests matching a given test-spec.
362 @param test_spec A test-spec string.
363 @param test_modules Same format as the output of load_test_modules.
364 @returns Same format as the output of load_test_modules.
365 """
366 result = {}
367 for (spec_modname, spec_testname) in parse_test_spec(test_spec):
368 matched = False
369 for (modname, (mod, tests)) in test_modules.items():
370 if (spec_modname == None or spec_modname == modname):
371 for (testname, test) in tests.items():
372 if (spec_testname == None or spec_testname == testname):
373 result.setdefault(modname, (mod, {}))
374 result[modname][1][testname] = test
375 matched = True
376 if not matched:
377 if spec_modname and spec_testname:
378 el = "%s.%s" % (spec_modname, spec_testname)
379 else:
380 el = spec_modname or spec_testname or "all"
381 die("test-spec element %s did not match any tests" % el)
382 return result
383
384def parse_test_spec(test_spec):
385 """
386 The input string is split on commas and each element is parsed
387 individually into a module name and test name. Either may be None
388 for a wildcard. The case of the first letter resolves ambiguity
389 of whether a word is a test or module name. The special string
390 "all" results in both fields wildcarded.
391
392 Examples:
393 basic.Echo -> ("basic", "Echo")
394 basic -> ("basic", None)
395 Echo -> (None, "Echo")
396 all -> (None, None)
397 """
398 results = []
399 for ts_entry in test_spec.split(","):
400 parts = ts_entry.split(".")
401 if len(parts) == 1:
402 if ts_entry == "all":
403 results.append((None, None))
404 elif ts_entry[0].isupper():
405 results.append((None, ts_entry))
406 else:
407 results.append((ts_entry, None))
408 elif len(parts) == 2:
409 results.append((parts[0], parts[1]))
410 else:
411 die("Bad test spec: " + ts_entry)
412 return results
413
Dan Talayco2c0dba32010-03-06 22:47:06 -0800414def die(msg, exit_val=1):
415 print msg
416 logging.critical(msg)
417 sys.exit(exit_val)
418
Dan Talayco79f36082010-03-11 16:53:53 -0800419def _space_to(n, str):
420 """
421 Generate a string of spaces to achieve width n given string str
422 If length of str >= n, return one space
423 """
424 spaces = n - len(str)
425 if spaces > 0:
426 return " " * spaces
427 return " "
428
Dan Talaycoc24aaae2010-07-08 14:05:24 -0700429def test_prio_get(mod, test):
430 """
431 Return the priority of a test
Dan Talaycod7c80d12012-04-03 15:20:57 -0700432
433 If test is in "skip list" from profile, return the skip value
434
Dan Talaycoc24aaae2010-07-08 14:05:24 -0700435 If set in the test_prio variable for the module, return
436 that value. Otherwise return 100 (default)
437 """
Rich Lane1673b8f2012-09-25 18:35:18 -0700438 if test in profile_mod.skip_test_list:
439 logging.info("Skipping test %s due to profile" % test)
440 return TEST_PRIO_SKIP
Dan Talaycoc24aaae2010-07-08 14:05:24 -0700441 if 'test_prio' in dir(mod):
442 if test in mod.test_prio.keys():
443 return mod.test_prio[test]
444 return TEST_PRIO_DEFAULT
445
Dan Talayco48370102010-03-03 15:17:33 -0800446#
447# Main script
448#
449
450# Get configuration, set up logging, import platform from file
451(config, args) = config_setup(config_default)
Dan Talayco48370102010-03-03 15:17:33 -0800452
Rich Laned7a85c42012-09-28 15:38:45 -0700453logging_setup(config)
454logging.info("++++++++ " + time.asctime() + " ++++++++")
455
Rich Lanee284b6b2012-10-03 09:19:58 -0700456# Allow tests to import each other
457sys.path.append(config["test_dir"])
458
Rich Lane15f64de2012-10-04 21:25:57 -0700459test_modules = prune_tests(config["test_spec"], load_test_modules(config))
Dan Talaycocf26b7a2011-08-05 10:15:35 -0700460oft_config = config
Dan Talayco2c0dba32010-03-06 22:47:06 -0800461
Rich Lanec27fa002012-09-28 12:49:12 -0700462load_profile(config)
463
Dan Talayco2c0dba32010-03-06 22:47:06 -0800464# Check if test list is requested; display and exit if so
465if config["list"]:
Dan Talayco7f8dba82012-04-12 12:58:52 -0700466 mod_count = 0
467 test_count = 0
Dan Talayco2c0dba32010-03-06 22:47:06 -0800468 print "\nTest List:"
Rich Lane943be672012-10-04 19:20:16 -0700469 for (modname, (mod, tests)) in test_modules.items():
Dan Talayco7f8dba82012-04-12 12:58:52 -0700470 mod_count += 1
Dan Talayco79f36082010-03-11 16:53:53 -0800471 desc = mod.__doc__.strip()
472 desc = desc.split('\n')[0]
473 start_str = " Module " + mod.__name__ + ": "
474 print start_str + _space_to(22, start_str) + desc
Rich Lane943be672012-10-04 19:20:16 -0700475 for (testname, test) in tests.items():
Dan Talayco551befa2010-07-15 17:05:32 -0700476 try:
Rich Lane943be672012-10-04 19:20:16 -0700477 desc = test.__doc__.strip()
Dan Talayco551befa2010-07-15 17:05:32 -0700478 desc = desc.split('\n')[0]
479 except:
480 desc = "No description"
Rich Lane943be672012-10-04 19:20:16 -0700481 if test_prio_get(mod, testname) < 0:
482 start_str = " * " + testname + ":"
Dan Talaycoc24aaae2010-07-08 14:05:24 -0700483 else:
Rich Lane943be672012-10-04 19:20:16 -0700484 start_str = " " + testname + ":"
Dan Talayco551befa2010-07-15 17:05:32 -0700485 if len(start_str) > 22:
486 desc = "\n" + _space_to(22, "") + desc
Dan Talayco79f36082010-03-11 16:53:53 -0800487 print start_str + _space_to(22, start_str) + desc
Dan Talayco7f8dba82012-04-12 12:58:52 -0700488 test_count += 1
Dan Talayco79f36082010-03-11 16:53:53 -0800489 print
Rich Lane15f64de2012-10-04 21:25:57 -0700490 print "%d modules shown with a total of %d tests" % \
491 (mod_count, test_count)
492 print
493 print "Tests preceded by * are not run by default"
Dan Talayco7aa0b812010-07-20 14:51:41 -0700494 print "Tests marked (TP1) after name take --test-params including:"
Dan Talaycoac25cf32010-07-20 14:08:28 -0700495 print " 'vid=N;strip_vlan=bool;add_vlan=bool'"
Dan Talaycod7c80d12012-04-03 15:20:57 -0700496 print "Note that --profile may override which tests are run"
Dan Talayco2c0dba32010-03-06 22:47:06 -0800497 sys.exit(0)
498
Jeffrey Townsend6614f972012-08-16 13:27:18 -0700499# Check if test list is requested; display and exit if so
500if config["list_test_names"]:
Rich Lane943be672012-10-04 19:20:16 -0700501 for (modname, (mod, tests)) in test_modules.items():
Rich Lane943be672012-10-04 19:20:16 -0700502 for (testname, test) in tests.items():
503 if test_prio_get(mod, testname) >= 0:
504 print "%s.%s" % (modname, testname)
Jeffrey Townsend6614f972012-08-16 13:27:18 -0700505 sys.exit(0)
506
Dan Talayco2c0dba32010-03-06 22:47:06 -0800507# Generate the test suite
508#@todo Decide if multiple suites are ever needed
509suite = unittest.TestSuite()
510
Rich Lane15f64de2012-10-04 21:25:57 -0700511for (modname, (mod, tests)) in test_modules.items():
512 for (testname, test) in tests.items():
513 if test_prio_get(mod, testname) >= 0:
514 logging.info("Adding test " + modname + "." + testname)
515 suite.addTest(test())
Dan Talayco2c0dba32010-03-06 22:47:06 -0800516
Rich Lane8aebc5e2012-09-25 17:57:53 -0700517# Load the platform module
518platform_name = config["platform"]
519logging.info("Importing platform: " + platform_name)
520platform_mod = None
521try:
522 platform_mod = imp.load_module(platform_name, *imp.find_module(platform_name, [config["platform_dir"], config["test_dir"]]))
523except:
524 logging.warn("Failed to import " + platform_name + " platform module")
525 raise
Dan Talayco48370102010-03-03 15:17:33 -0800526
527try:
Rich Lane8aebc5e2012-09-25 17:57:53 -0700528 platform_mod.platform_config_update(config)
Dan Talayco48370102010-03-03 15:17:33 -0800529except:
530 logging.warn("Could not run platform host configuration")
Rich Laneef403e52012-07-09 14:59:43 -0700531 raise
Dan Talayco48370102010-03-03 15:17:33 -0800532
533if not config["port_map"]:
Rich Lane8aebc5e2012-09-25 17:57:53 -0700534 die("Interface port map was not defined by the platform. Exiting.")
Dan Talayco48370102010-03-03 15:17:33 -0800535
536logging.debug("Configuration: " + str(config))
537logging.info("OF port map: " + str(config["port_map"]))
538
539# Init the test sets
Rich Lane943be672012-10-04 19:20:16 -0700540for (modname, (mod, tests)) in test_modules.items():
Dan Talayco2c0dba32010-03-06 22:47:06 -0800541 try:
542 mod.test_set_init(config)
543 except:
544 logging.warning("Could not run test_set_init for " + modname)
Rich Laneef403e52012-07-09 14:59:43 -0700545 raise
Dan Talayco48370102010-03-03 15:17:33 -0800546
Dan Talayco2c0dba32010-03-06 22:47:06 -0800547if config["dbg_level"] == logging.CRITICAL:
548 _verb = 0
549elif config["dbg_level"] >= logging.WARNING:
550 _verb = 1
551else:
552 _verb = 2
Dan Talayco48370102010-03-03 15:17:33 -0800553
Rich Lanee55abf72012-07-26 20:11:42 -0700554oftest.ofutils.default_timeout = config["default_timeout"]
Rich Laneda3b5ad2012-10-03 09:05:32 -0700555oftest.testutils.MINSIZE = config['minsize']
Rich Lanee55abf72012-07-26 20:11:42 -0700556
Rich Laneee57ad02012-07-13 15:40:36 -0700557if os.getuid() != 0 and not config["allow_user"]:
Brandon Heller446c1432010-04-01 12:43:27 -0700558 print "ERROR: Super-user privileges required. Please re-run with " \
559 "sudo or as root."
560 exit(1)
561
Rich Lane8592bec2012-09-03 09:06:59 -0700562if config["random_seed"] is not None:
563 logging.info("Random seed: %d" % config["random_seed"])
564 random.seed(config["random_seed"])
565
Rich Lane5bd6cf92012-10-04 17:57:24 -0700566# Remove python's signal handler which raises KeyboardError. Exiting from an
567# exception waits for all threads to terminate which might not happen.
568signal.signal(signal.SIGINT, signal.SIG_DFL)
Dan Talaycoac25cf32010-07-20 14:08:28 -0700569
570if __name__ == "__main__":
571 logging.info("*** TEST RUN START: " + time.asctime())
Rich Lane6a1ecb82012-07-10 18:59:44 -0700572 result = unittest.TextTestRunner(verbosity=_verb).run(suite)
Rich Laneda3b5ad2012-10-03 09:05:32 -0700573 if oftest.testutils.skipped_test_count > 0:
Dan Talaycoba3745c2010-07-21 21:51:08 -0700574 ts = " tests"
Rich Laneda3b5ad2012-10-03 09:05:32 -0700575 if oftest.testutils.skipped_test_count == 1: ts = " test"
576 logging.info("Skipped " + str(oftest.testutils.skipped_test_count) + ts)
577 print("Skipped " + str(oftest.testutils.skipped_test_count) + ts)
Dan Talaycoac25cf32010-07-20 14:08:28 -0700578 logging.info("*** TEST RUN END : " + time.asctime())
Rich Lane50d42eb2012-07-16 11:57:03 -0700579 if result.failures or result.errors:
Shudong Zhoud895fcb2012-08-01 19:09:55 -0700580 # exit(1) hangs sometimes
581 os._exit(1)
Rich Laneda3b5ad2012-10-03 09:05:32 -0700582 if oftest.testutils.skipped_test_count > 0 and config["fail_skipped"]:
Shudong Zhoud895fcb2012-08-01 19:09:55 -0700583 os._exit(1)