Add more info to the quickstart_physical.md
- How to configure route on compute node fabric interface
- How to connect switches to the controller
- How to re-initialize vtn nodes if something went wrong
- How to update fabric config
- Setup sudo nopasswd on headnode
Change-Id: I4fc29a865b19980ed916272ad09476a2acab6874
diff --git a/docs/quickstart_physical.md b/docs/quickstart_physical.md
index 587f61d..07e88fe 100644
--- a/docs/quickstart_physical.md
+++ b/docs/quickstart_physical.md
@@ -41,7 +41,7 @@
- The leaf - spine switchs are Accton 6712s
- The compute nodes are using 40G Intel NIC cards
- The compute node that is to be designated the *head node* has
-Ubuntu 14.04 LTS installed.
+**Ubuntu 14.04 LTS** installed. In addition, the user should have **password-less sudo permission**.
**Prerequisite: Vagrant is installed and operationally.**
**Note:** *This quick start guide has only been tested against Vagrant and
@@ -667,10 +667,38 @@
onos> cordvtn-nodes
hostname=nova-compute-1, hostMgmtIp=192.168.122.140/24, dpIp=10.168.0.253/24, br-int=of:0000000000000001, dpIntf=fabric, init=COMPLETE
Total 1 nodes
+
+### Troubleshoot
+If the compute node is not initialized properly (i.e. not in the COMPLETE state):
+On the compute node, run
```
+sudo ovs-vsctl del-br br-int
+```
+On the head node, run
+```
+ssh karaf@onos-cord-1 -p 8101
+```
+(password is karaf)
+and then in the ONOS CLI, run
+```
+cordvtn-node-init <compute-node-name>
+```
+(name is something like creamy-vegetable)
## Post Deployment Configuration of the ONOS Fabric
+### Manully Configure Routes on the Compute Node `br-int` Interface
+The routes on the compute node `br-int` interface need to be manually configured now.
+Run the following command on compute-1 and compute-2 (nodes in 10.6.1.0/24)
+```
+sudo ip route add 10.6.2.0/24 via 10.6.1.254
+```
+Run the following command on compute-3 and compute-4 (nodes in 10.6.2.0/24)
+```
+sudo ip route add 10.6.1.0/24 via 10.6.2.254
+```
+
+### Modify and Apply Fabric Configuration
Configuring the switching fabric for use with CORD is documented in the
[Fabric Configuration Guide](https://wiki.opencord.org/display/CORD/Fabric+Configuration+Guide) on the OpenCORD wiki.
@@ -680,29 +708,32 @@
```
Then edit the file `~/xos_services/fabric/config/network-cfg-quickstart.json` as follows:
- - **Replace the DPID of the leaf-spine switches:** The DPIDs are generated by adding 4 extra zeros
- in front of the MAC address of the management interface. If a switch has been provisioned
- by MAAS correctly, you should be able to find DPID information in `/root/connect` on each switch.
- Search for the following line:
- ```
- brcm-indigo-ofdpa-ofagent --dpid=0x0000cc37ab6182d2 --controller=$IP
- ```
- Remember to replace "0x" with "of:" . E.g. **of:0000cc37ab6182d2**
+ - **Replace the DPID of the leaf-spine switches:**
+ Locate the switches by running the following command on the haed node.
+ ```
+ admin@cord-head-1:~$ cat /etc/bind/maas/dhcp_harvest.inc | grep onl
+ onl-x86-CC37AB6182D2 IN A 10.6.0.11 ; cc:37:ab:61:82:d2
+ onl-x86-CC37AB617EC2 IN A 10.6.0.17 ; cc:37:ab:61:7e:c2
+ onl-x86-CC37AB6180CA IN A 10.6.0.8 ; cc:37:ab:61:80:ca
+ onl-x86-CC37AB618048 IN A 10.6.0.18 ; cc:37:ab:61:80:48
+ ```
+ The DPID of the switch is `of:0000` followed by the MAC address.
+ For example, the DPID is `of:0000cc37ab6182d2` for `onl-x86-CC37AB6182D2`
- **Modify the MAC address of hosts:** If a compute node has been provisioned by maas correctly,
you should be able to find out its MAC address by running the following command on each compute node.
```
- ifconfig fabric | grep HWaddr | awk {'print $5'}
+ ifconfig br-int | grep HWaddr | awk {'print $5'}
```
A valid MAC address format should look like **00:02:c9:1e:b1:21**
- - **Modify IP address and default gateway on the compute nodes:** You need to manually configure
- the IP address and route on the `fabric` bonding interface. (More details needed here.)
-
-Once these steps are done, apply the configuration (still in the XOS VM):
+Once these steps are done, delete old configuration,
+apply new configuration, and restart apps in ONOS (still in the XOS VM):
```
cd ~/service-profile/cord-pod
+make delete_fabric_config
make fabric
+make reactivate_fabric_apps
```
To verify that XOS has pushed the configuration to ONOS, log into ONOS in the onos-fabric VM and run `netcfg`:
@@ -744,6 +775,25 @@
... etc.
```
+### Connect Switches to the controller
+We need to manually connects the switches to ONOS after the network config is applied.
+Locate the switches by running the following command on the haed node.
+```
+admin@cord-head-1:~$ cat /etc/bind/maas/dhcp_harvest.inc | grep onl
+onl-x86-CC37AB6182D2 IN A 10.6.0.11 ; cc:37:ab:61:82:d2
+onl-x86-CC37AB617EC2 IN A 10.6.0.17 ; cc:37:ab:61:7e:c2
+onl-x86-CC37AB6180CA IN A 10.6.0.8 ; cc:37:ab:61:80:ca
+onl-x86-CC37AB618048 IN A 10.6.0.18 ; cc:37:ab:61:80:48
+```
+Log in to each switch (default user/pass is root/onl) and run the following command
+```
+/root/connect -bg
+```
+If something goes wrong and you need to reconnect the switch to the controller, run
+```
+/root/killit && /root/purge && /root/connect -bg
+```
+
### Complete
This step is complete when each compute node can ping the fabric IP address of all the other nodes.