AETHER-3411 Persist network config across reboot

Change-Id: I4a2df6c4fcc049986a384771f178fb1739351f34
diff --git a/developer/aiab.rst b/developer/aiab.rst
index 0e5caba..892e9b9 100644
--- a/developer/aiab.rst
+++ b/developer/aiab.rst
@@ -49,6 +49,17 @@
 
 Now change to *~/aether-in-a-box* directory.
 
+RKE2 vs. Kubespray Install
+--------------------------
+
+The AiaB installer will bring up Kubernetes on the server where it is run.  By default it
+uses `RKE2 <https://docs.rke2.io>`_ as the Kubernetes platform.  However, older versions of AiaB
+used `Kubespray <https://kubernetes.io/docs/setup/production-environment/tools/kubespray/>`_
+and that is still an option.  To switch to Kubespray as the Kubernetes platform, edit the
+Makefile and replace *rke2* with *kubespray* on this line::
+
+    K8S_INSTALL := rke2
+
 Installing the ROC
 ------------------
 
@@ -170,6 +181,8 @@
     make reset-5g-test
     make roc-clean
 
+To completely remove AiaB by tearing down the Kubernetes cluster, run *make clean*.
+
 Developer Loop
 --------------
 
diff --git a/developer/aiabhw.rst b/developer/aiabhw.rst
index 6fe38ec..9365280 100644
--- a/developer/aiabhw.rst
+++ b/developer/aiabhw.rst
@@ -176,6 +176,28 @@
 
 You should see all pods in Running status.
 
+The UPF pod connects to the ``DATA_IFACE`` specified above using macvlan networks called
+``core`` and ``access``.  Next, check that these have been successfully created, e.g. using
+``ifconfig``::
+
+    $ ifconfig core
+    core: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
+        inet 192.168.250.1  netmask 255.255.255.0  broadcast 192.168.250.255
+        ether 16:9d:c1:0f:19:3a  txqueuelen 1000  (Ethernet)
+        RX packets 513797  bytes 48400525 (48.4 MB)
+        RX errors 0  dropped 0  overruns 0  frame 0
+        TX packets 102996  bytes 26530538 (26.5 MB)
+        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
+
+    $ ifconfig access
+    access: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
+        inet 192.168.252.1  netmask 255.255.255.0  broadcast 192.168.252.255
+        ether 7a:9f:38:c0:18:15  txqueuelen 1000  (Ethernet)
+        RX packets 558162  bytes 64064410 (64.0 MB)
+        RX errors 0  dropped 0  overruns 0  frame 0
+        TX packets 99553  bytes 16646682 (16.6 MB)
+        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
+
 Sercomm eNodeB setup
 --------------------
 
@@ -291,12 +313,26 @@
 Documenting how to configure different types of devices to work
 with Aether is work-in-progress.
 
-Reinstalling AiaB
------------------
+Troubleshooting
+---------------
 
-A current limitation of AiaB is that if the host machine reboots,
-AiaB needs to be reinstalled.  We plan to fix this in the future so
-that the AiaB configuration will persist across reboots.  In the
-meantime, to reinstall AiaB on a machine where it was previously
-installed, run `make clean` and then start at the `Server setup`_
-section above.
+AiaB connects macvlan networks to ``DATA_IFACE`` so that the UPF can communicate on the network.
+To do this it assumes that the *systemd-networkd* service is installed and running, ``DATA_IFACE``
+is under its control, and the systemd-networkd configuration file for ``DATA_IFACE`` ends with
+``<DATA_IFACE>.network``, where ``<DATA_IFACE>`` stands for the actual interface name.  It
+tries to find this configuration file by looking in the standard paths.  If it fails you'll see
+a message like::
+
+    FATAL: Could not find systemd-networkd config for interface foobar, exiting now!
+    make: *** [Makefile:112: /users/acb/aether-in-a-box//build/milestones/interface-check] Error 1
+
+In this case, you can specify a ``DATA_IFACE_PATH=<path to the config file>`` argument to ``make``
+so that AiaB can find the systemd-networkd configuration file for ``DATA_IFACE``.
+
+Restarting the AiaB Server
+--------------------------
+
+AiaB should come up in a mostly working state if the AiaB server is rebooted.  If any pods are
+stuck in an Error or CrashLoopBackoff state they can be restarted using ``kubectl delete pod``.
+It might also be necessary to power cycle the Sercomm eNodeB in order to get it to reconnect to
+the SD-CORE.
diff --git a/dict.txt b/dict.txt
index 67d51cf..39d0a63 100644
--- a/dict.txt
+++ b/dict.txt
@@ -35,6 +35,7 @@
 Keycloak
 KinD
 Kubernetes
+Kubespray
 LTE
 Mbps
 Menlo
@@ -164,6 +165,7 @@
 loopback
 macOS
 macroENB
+macvlan
 mainboard
 makefile
 menlo
@@ -180,6 +182,7 @@
 natively
 nd
 netcfg
+networkd
 ng
 ng40
 nginx
@@ -244,6 +247,7 @@
 subnet
 subnets
 sudo
+systemd
 tAC
 tcp
 telegraf