OpenFlow Testing Framework
February, 2010

Warning
+++++++

This is still experimental and it requires root privilege to
control the dataplane ports.  As a consequence, there are
some serious risks to the machine on which this is running.
Use caution.

This is still preliminary work and there are bugs in the 
framework that need to be ironed out.  Please report any issues
to dtalayco@stanford.edu.

Quick Start
+++++++++++
    # cd <oftest>/tools/munger
    # make install
    # cd <oftest>/tests
    # ./oft --list
    # ./oft
    # ./oft --test-spec=<mod> --debug=info --platform=remote --host=...

Longer Start
++++++++++++

    1.  Pre-requisites:
        * Root privilege on host
        * Python 2.5.  You can run platforms using eth interfaces with
          Python 2.4.
        * oftest checked out (called <oftest> here)
        * scapy installed:  http://www.secdev.org/projects/scapy/
        * Doxygen and doxypy for document generation (optional)
        * lint for source checking (optional)

    2.  Build the OpenFlow Python message classes

        Important:  The OF version used by the controller is based on 
        the file in <oftest>/tools/pylibopenflow/include/openflow.h
        This is currently the 1.0 release file.

        * cd <oftest>/tools/munger
        * make install
        This places files in <oftest>/src/python/oftest/src and then
        calls setuptools to install on the local host

    3.  Edit configuration if necessary
        Local platforms work with veth interface pairs and default to
        four ports.  You can adjust this a bit with the command line
        parameters port_count, base_of_port and base_if_index.

        Starting from remote.py as a simple example, you can add your
        own <platform>.py file and then have it imported with
        --platform=<platform> on the command line.

    4.  Start the switch to test
        The switch must be running and actively attempting to 
        connect to the controller at the host/port specified above.

    5.  Run oft
        See Warning above; requires sudo to control the dataplane
        * cd <oftest>/tests
        * sudo ./oft --help

Overview
++++++++

    The directory structure is currently:

     <oftest>
         `
         |-- doc
         |-- src
         |   `-- python
         |       `-- oftest
         |-- tests
         |   `-- oft and files with test cases
         `-- tools
             |-- munger
             `-- pylibopenflow

    The tools directory is what processes the OpenFlow header
    files to produce Python classes representing OpenFlow messages.
    The results are placed in src/python/oftest and currently
    include:

        message.py:      The main API providing OF message classes
        error.py:        Subclasses for error messages
        action.py:       Subclasses for action specification
        cstruct.py:      Direct representation of C structures in Python
        class_maps.py:   Addition info about C structures

    In addition, the following Python files are present in 
    src/python/oftest:

        controller.py:   The controller representation
        dataplane.py:    The dataplane representation
        action_list.py:  Action list class
        netutils.py:     e.g., set promisc on sockets
        ofutils.py:      Utilities related to OpenFlow messages
        oft_assert.py:   Test framework level assertion

    Tests are run from the tests directory.  The file oft is the
    top level entry point for tests.  Try ./oft --help for some more.

Important Notes
+++++++++++++++

    1.  If you edit any of the files in src/python/oftest or any of the
    scripts in tools/munger/scripts, you MUST re-run make install.  This
    is easy to forget.

    2.  If your running into issues with transactions, and it appears that
    OpenFlow messages aren't quite right, start by looking at any length
    fields in the packets.  With the local platform, you can use wireshark
    on the loopback interface as well as the dataplane veth interfaces.

Adding Your Own Test Cases
++++++++++++++++++++++++++

    You can:

        * Add cases to an existing file
        * Add a new file

    If you add cases to an existing file, each case should be its own
    class.  It must inherit from unittest.TestCase or one of its 
    derivatives and define runTest (that's how test cases are discovered).

    If you add a new file, it must implement a top level function called
    test_set_init which takes a configuration dictionary.  See basic.py
    for an example.  The main point of this is to pass the port map 
    object to the test cases.  But you can access any configuration
    parameters this way.  Each test case in the new file must derive
    from unittest.TestCase.


Using CentOS/RHEL
+++++++++++++++++

    CentOS/RHEL have two challenges:  they are very tied to Python 2.4
    (and Scapy requires Python 2.5 for its latest version) and they
    require a kernel upgrade to use veth pairs for local platform
    testing.  

    If you only need to control eth interfaces for a remote platform,
    you can use CentOS/RHEL.  The key is to download scapy-1.2 from
    the following link:

    wget http://hg.secdev.org/scapy/raw-file/v1.2.0.2/scapy.py

    See: http://www.dirk-loss.de/scapy-doc/installation.html#installing-scapy-v1-2
    for more info.

    Copy scapy.py to /usr/lib/python2.4/site-packages

    If you hit an error related to importing scapy.all, you just need
    to change the import to refer to scapy (not scapy.all).  See
    examples in parse.py for example.


Other Info
++++++++++

    * Build doc with
      + cd <oftest>/tools/munger
      + make doc
    Places the results in <oftest>/doc/html

    * Run lint on sources
      + cd <oftest>/tools/munger
      + make lint
    Places results in <oftest>/lint/*.log
    The file controller.log currently has some errors indicated


To Do
+++++

    * Need to have an overview of the components of the test, how they
      connect and how they are managed by the test framework.
    * See the Regression Test component on trac:
      http://www.openflowswitch.org/bugs/openflow
      http://www.openflowswitch.org/bugs/openflow/query?component=Regression+test+suite

    * Make the framework work with OF versions other than 1.0?