Fabric/Doc/docs/flow_query_test_plan.txt

DRAFT TEST PLAN FOR TESTING TOP-HALF OF OpenFlow 1.0 SWITCH

Draft 2 - 23 Apr 2012



1.  GENERAL
===========

1.1  Test particular implementation, not OpenFlow protocol

This test plan is intended to test the function of those features of an OF
agent implementation that would be particular to a given switch,
i.e. semantics of various OF operations are correctly implemented on a given
switch.
It is NOT intended to test:
- syntactic featues, such as badly formed OF protocol messages, etc., nor
- semantic features that would be common to all implementations, e.g.
  qualifying on a VLAN id > 4095, an action to set VLAN PCP to a value > 7,
  etc.


2.  FLOW ADD
============

2.5  FLOW ADD 5
---------------

OVERVIEW
- Add flows to switch, read back and verify flow configurations

PURPOSE
- Test acceptance of flow adds
- Test ability of switch to process additions to flow table in random
  priority order
- Test correctness of flow configuration responses

INPUTS
- NUM_FLOWS: number of flows to define; 0 => maximum number of flows, as
  determined from switch capabilities

PROCESS
1. Delete all flows from switch
2. Generate NUM_FLOWS distinct flow configurations
3. Send NUM_FLOWS flow adds to switch, for flows generated in step 2 above
   - Flow mods with OFPFF_SEND_FLOW_REM = 0 (do not send remove message) and
     OFPFF_CHECK_OVERLAP = 0 (do not check for overlap)
4. Verify that no OFPT_ERROR responses were generated by switch
5. Retrieve flow stats from switch
6. Compare flow configurations returned by switch
7. Test PASSED iff all flows sent to switch in step 3 above are returned in
   step 5 above; else test FAILED

NOTES
- Will use randomized flow configuration, in an attempt to cover combinations
  of qualifier wildcarding, qualifier values, actions and action parameters


2.5.1  FLOW ADD 5.1
-------------------

OVERVIEW
- Verify flow canonicalization

PURPOSE
- Test that switch properly canonicalizes a given flow definition
  (canonicalization is defined as wildcarding out qualifiers when antecedent
  qualifiers are not set correctly, e.g. nw_tos cannot be specified, and hence
  must be wildcarded, if dl_type is not specified to be 0x0800 (IP))

INPUTS
None

PROCESS
1. Delete all flows from switch
2. Generate 1 flow definition, which is different from its canonicalization
3. Send flow to switch
4. Retrieve flow from switch
5. Compare returned flow to canonical form of defined flow
7. Test PASSED iff flow received in step 4 above is identical to canonical
   form of flow defined in step 3 above


2.6  FLOW ADD 6
---------------

OVERVIEW
- Test flow table capacity

PURPOSE
- Test switch can accept as many flow definitions as it claims
- Test generation of OFPET_FLOW_MOD_FAILED/OFPFMFC_ALL_TABLES_FULL
- Test that attempting to create flows beyond capacity does not corrupt flow
  table

INPUTS
None

PROCESS
1. Delete all flows from switch
2. Send OFPT_FEATURES_REQUEST to determine switch features
3. For each table reported by step 2 above, send OFPT_STATS_REQUEST/
   OFPST_TABLE to determine table features
4. Generate (N + 1) distinct flow configurations, where N is the flow capacity
   reported by the switch in steps 2 and 3 above
   - Flow wildcarding must obey supported wildcards, for each table
5. Send (N + 1) flow adds to switch, for flows generated in step 4 above
   - Flow mods with OFPFF_SEND_FLOW_REM = 0 (do not send remove message) and
     OFPFF_CHECK_OVERLAP = 0 (do not check for overlap)
6. Verify that OFPET_FLOW_MOD_FAILED/OFPFMFC_ALL_TABLES_FULL error response
   was generated by switch, for last flow mod sent
7. Retrieve flow stats from switch
8. Compare flow configurations returned by switch
9. Test PASSED iff:
   - error message received, for correct flow
   - last flow definition sent to switch is not in flow table
   else test FAILED


2.7  FLOW ADD 7
---------------

OVERVIEW
- Test flow redefinition

PURPOSE
- Verify that successive flow adds with same priority and match criteria
  overwrite in flow table

INPUTS
None

PROCESS
1. Delete all flows from switch
2. Generate flow definition F1
3. Generate flow definition F2, with same key (priority and match) as F1,
   but with different actions
4. Send flow adds for F1 and F2 to switch
   - Flow mods with OFPFF_SEND_FLOW_REM = 0 (do not send remove message) and
     OFPFF_CHECK_OVERLAP = 0 (do not check for overlap)
5. Retrieve flow stats from switch
6. Compare flow configurations returned by switch
7. Test PASSED iff 1 flow returned by switch, matching configuration of F2,
   and with counters equal to 0; else test FAILED


2.8  FLOW ADD 8
---------------

OVERVIEW
- Add overlapping flows to switch, verify that overlapping flows are rejected

PURPOSE
- Test detection of overlapping flows by switch
- Test generation of OFPET_FLOW_MOD_FAILED/OFPFMFC_OVERLAP messages
- Test rejection of overlapping flows
- Test defining overlapping flows does not corrupt flow table

INPUTS
None

PROCESS
1. Delete all flows from switch
2. Generate flow definition F1
3. Generate flow definition F2, with key overlapping F1, by wildcarding a
   qualifier specified in F1
4. Send flow adds to switch, for flows generated in steps 2 and 3 above
   - Flow mods with OFPFF_SEND_FLOW_REM = 0 (do not send remove message) and
     OFPFF_CHECK_OVERLAP = 1 (check for overlap)
4. Verify that OFPET_FLOW_MOD_FAILED/OFPFMFC_OVERLAP error response was
   generated by switch
5. Retrieve flow stats from switch
6. Compare flow configurations returned by switch
7. Test PASSED iff:
   - error message received, for overlapping flow
   - overlapping flow is not in flow table
   else test FAILED


3.  FLOW MODIFY
===============

3.1  FLOW MODIFY 1
------------------

OVERVIEW
- Strict modify of single existing flow

PURPOSE
- Verify that strict flow modify operates only on specified flow
- Verify that flow is correctly modified

INPUTS
- NUM_FLOWS: number of flows to define; 0 => maximum number of flows,
  as determined from switch capabilities

PROCESS
1. Delete all flows from switch
2. Generate NUM_FLOWS distinct flow configurations
3. Send NUM_FLOWS flow adds to switch, for flows generated in step 2 above
   - Flow mods with OFPFF_SEND_FLOW_REM = 0 (do not send remove message) and 
     OFPFF_CHECK_OVERLAP = 0 (do not check for overlap)
4. Pick 1 defined flow F at random
5. Generate new action list for F
6. Send flow modify for F to switch
4. Verify that no OFPT_ERROR responses were generated by switch
5. Retrieve flow stats from switch
6. Compare flow configurations returned by switch
7. Test PASSED iff all flows sent to switch in step 3 and 6 above are
   returned in step 5 above; else test FAILED


3.2  FLOW MODIFY 2
------------------

OVERVIEW
- Loose modify of existing flows

PURPOSE
- Verify that loose flow modify operates only on matching flows
- Verify that matching flows are correctly modified

INPUTS
- NUM_FLOWS: number of flows to define; 0 => maximum number of flows, as
  determined from switch capabilities

PROCESS
1. Delete all flows from switch
2. Generate NUM_FLOWS distinct flow configurations
3. Send NUM_FLOWS flow adds to switch, for flows generated in step 2 above
   - Flow mods with OFPFF_SEND_FLOW_REM = 0 (do not send remove message) and
     OFPFF_CHECK_OVERLAP = 0 (do not check for overlap)
4. Pick 1 defined flow F at random
5. Wildcard out 1 qualifier in match of F, creating F', such that F' will
   match more than 1 existing flow key, and create new actions list A' for F'
6. Send loose flow modify for F' to switch
7. Retrieve flow stats from switch
8. Compare flow configurations returned by switch
9. Test PASSED iff all flows sent to switch in steps 3 and 6 above, are
   returned in step 7 above, each with correct (original or modified) action
   list;
   else test FAILED


3.3  FLOW MODIFY 3
------------------

OVERVIEW
- Strict modify of non-existent flow

PRUPOSE
- Verify that strict modify of a non-existent flow is equivalent to a flow add

INPUTS
None

PROCESS
1. Delete all flows from switch
2. Send strict flow modify to switch
3. Retrieve flows from switch
4. Test PASSED iff single flow defined in step 2 above is returned in step 3
   above; else test FAILED


3.3.1 FLOW MODIFY 3_1
---------------------

OVERVIEW
- No-op modify

PURPOSE
- Verify that modify of a flow with new actions same as old ones operates
  correctly

PARAMETERS
None

PROCESS
1. Delete all flows from switch
2. Send single flow mod, as strict modify, to switch
3. Verify flow table in switch
4. Send same flow mod, as strict modify, to switch
5. Verify flow table in switch
6. Test PASSED iff flow defined in step 2 and 4 above verified; else FAILED


4.  FLOW DELETE
===============

4.1  FLOW DELETE 1
------------------

OVERVIEW
- Strict delete of single flows

PURPOSE
- Verify correct operation of strict delete of single defined flow

INPUTS
- NUM_FLOWS: Number of flows to define; 0 => maximum number of flows, as
  determined fro switch capabilities

PROCESS
1. Delete all flows from switch
2. Generate NUM_FLOWS distinct flow configurations
3. Send NUM_FLOWS flow adds to switch, for flows generated in step 2 above
   - Flow mods with OFPFF_SEND_FLOW_REM = 0 (do not send remove message) and
     OFPFF_CHECK_OVERLAP = 0 (do not check for overlap)
4. Pick 1 defined flow F at random
5. Send strict flow delete for F to switch
6. Retrieve flow stats from switch
7. Compare flow configurations returned by switch
8. Test PASSED iff all flows sent to switch in step 3 above, less flow
   removed in step 5 above, are returned in step 5 above; else test FAILED


4.2  FLOW DELETE 2
------------------

OVERVIEW
- Loose delete of flows

PURPOSE
- Verify correct operation of loose delete of multiple flows

INPUTS
- NUM_FLOWS: Number of flows to define; 0 => maximum number of flows, as
  determined fro switch capabilities

PROCESS
1. Delete all flows from switch
2. Generate NUM_FLOWS distinct flow configurations
3. Send NUM_FLOWS flow adds to switch, for flows generated in step 2 above
   - Flow mods with OFPFF_SEND_FLOW_REM = 0 (do not send remove message) and
     OFPFF_CHECK_OVERLAP = 0 (do not check for overlap)
4. Pick 1 defined flow F at random
5. Wildcard out 1 qualifier in match of F, creating F', such that F' will
   match more than 1 existing flow key
6. Send loose flow delete for F' to switch
7. Retrieve flow stats from switch
8. Compare flow configurations returned by switch
9. Test PASSED iff all flows sent to switch in step 3 above, less flows
   removed in step 6 above (i.e. those that match F'), are returned in step
   5 above;
   else test FAILED


4.4  FLOW DELETE 4
------------------

OVERVIEW
- Flow removed messages

PURPOSE
- Verify that switch generates OFPT_FLOW_REMOVED/OFPRR_DELETE response
  messages when deleting flows that were added with OFPFF_SEND_FLOW_REM flag

INPUTS
- NUM_FLOWS: Number of flows to define; 0 => maximum number of flows, as
  determined fro switch capabilities

PROCESS
1. Delete all flows from switch
2. Generate NUM_FLOWS distinct flow configurations
3. Send NUM_FLOWS flow adds to switch, for flows generated in step 2 above
   - Flow mods with OFPFF_SEND_FLOW_REM = 1 (do not send remove message) and
     OFPFF_CHECK_OVERLAP = 0 (do not check for overlap)
4. Pick 1 defined flow F at random
5. Send strict flow delete for F to switch
6. Verify that OFPT_FLOW_REMOVED/OFPRR_DELETE message is received from switch
7. Retrieve flow stats from switch
8. Compare flow configurations returned by switch
9. Test PASSED iff all flows sent to switch in step 3 above, less flow
   removed in step 5 above, are returned in step 5 above; else test FAILED