blob: 3dd27c1e2975d985b746c3d94704dc43ea4ad22e [file] [log] [blame]
Charles Chan671e3982022-03-09 19:51:31 -08001.. SPDX-FileCopyrightText: 2021 Open Networking Foundation <info@opennetworking.org>
2.. SPDX-License-Identifier: Apache-2.0
3
4SD-Fabric 1.1
5=============
6
7After one year of incubation under the ONF's member-only license,
8we are excited to present the first `open-source release of SD-Fabric under the Apache 2.0 license
9<https://opennetworking.org/news-and-events/press-releases/onfs-leading-private-5g-connected-edge-platform-aether-now-released-to-open-source/>`_
10
11SD-Fabric 1.1 comes with numerous new features and improvements.
12The main focus for this release is full integration with Aether,
13ONF’s private-5G edge connectivity solution for enterprises.
14SD-Fabric brings the many benefits of network programmability to Aether,
15from a hardware-accelerated highly-available P4-UPF, to fabric-wide slicing and QoS,
16and per-packet visibility via Inband-Network Telemetry (INT).
17
18In this release, we provide two UPF options: the switch-based P4-UPF, offering the highest performance;
19and the CPU-based BESS-UPF, tailored for deployment flexibility and horizontal scalability.
20Both UPFs come with numerous bug fixes as well as integration with Aether 2.0 features,
21such as support for 5G base stations, application filtering, QoS via multi-level rate-limiting,
22and enhanced visibility with per-flow metrics and INT integration.
23
24Moreover, we provide many operational improvements such as better support for
25enterprise networks, simplified configuration, improved stability, and reduced resource usage.
26
27We hope the transition to open source will catalyze the adoption of SD-Fabric and
28attract more users and contributors from the community.
29Please visit the :ref:`SD-Fabric documentation website <index>` to learn more about
30all the features SD-Fabric offers and how to get started.
31
32Feature Highlights
33------------------
34
35Open Source Release
36^^^^^^^^^^^^^^^^^^^
37All SD-Fabric components are now completely open source under the Apache 2.0 license.
38We have published all source code, docker images, helm charts, documentation, and mailing lists.
39
40Aether Integration
41^^^^^^^^^^^^^^^^^^
42This release provides the following integrations with Aether 2.0:
43
44- **4G/5G slice based on P4-UPF**:
45 which can now be instantiated using the Aether management portal or APIs.
46 In this release, we support creating only one P4-UPF slice,
47 creation of more P4-UPF slices will come in future releases.
48 As before, more than one slice can be created when using BESS-UPF.
49- **Initial INT integration**:
50 this release adds di-metrics-exporter,
51 a new service which fetches and aggregates real-time INT-based metrics and anomalies from Intel\ |reg| DeepInsight,
52 to produce per-UE health status reports to be consumed by the Aether Analytics Engine (work-in-progress).
53 SD-fabric supports monitoring of UE traffic when using both P4-UPF and BESS-UPF.
54
55P4-UPF
56^^^^^^
57This release aims at bringing feature parity with BESS-UPF. To this end, P4-UPF now supports
58the following new features:
59
60- **Application filtering**:
61 where the UPF can drop or forward traffic based on application endpoints
62 (IPv4 prefix and port range) configured using PFCP SDF filters received from SD-Core.
63 We support all ROC policies: ALLOW-ALL, DENY-ALL, ALLOW-PUBLIC.
64- **Multi-level rate-limiting**:
65 where traffic can be policed at the application level, UE level, or slice level.
66 Per-application and per-UE rate limits can be configured by means of PFCP QERs received from SD-Core.
67
68BESS-UPF
69^^^^^^^^
70- **Port ranges in app filtering rules**:
71 BESS-UPF now supports installing PDRs with SDF filters
72 matching on more than one transport-layer port.
73- **Arbitrary IPv4 prefix lengths in app filtering rules**:
74 including Aether's **allow-public** rule set (which blocks traffic to RFC 1918 private subnets).
75- **Per-flow metrics (experimental)**:
76 when enabled, the BESS pipeline collects metrics such as throughput, latency, jitter, and packet loss.
77 Metrics are collected for each PFCP PDR rule,
78 allowing performance monitoring for multiple application flows for the same UE.
79 Metrics can be exported via the Prometheus endpoint in PFCP-Agent.
80
81Slicing & QoS
82^^^^^^^^^^^^^
83We continue to make slicing and QoS more configurable to adapt to different use cases.
84
85- **ONOS now reads slicing parameters via netcfg**:
86 to support static configuration of arbitrary slices and traffic classes.
87- **Updated config generation scripts**:
88 to automatically generate both Stratum chassis_config and ONOS netcfg file using high-level parameters.
89
90Operations
91^^^^^^^^^^
92Numerous improvements to make it easier to deploy and operate SD-Fabric.
93
94- **Support for P4Runtime port translation**:
95 which allows hiding ASIC-specific internal port numbers from user-facing configuration files.
96 Now, when configuring switch interfaces in both the ONOS netcfg and Stratum chassis config,
97 users can specify front panel port/channel numbers instead of SDK port numbers.
98- **Remove the management server from the data plane**:
99 user traffic is now processed by P4-UPF or BESS-UPF and then
100 forwarded upstream (or to edge-applications) directly by the fabric switches.
101- **Support base stations behind L3-routed network**:
102 supporting real-world enterprise deployments.
103- **Improve support for paired-leaves topology and distributed UPF**:
104 by enabling phased-recovery by default to ensure
105 traffic is not sent to a recovering switch until it is fully programmed, avoiding unnecessary packet drops.
106
107Bug fixes and other improvements
108^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
109- Made primary controller election in ONOS persistent during device and cluster events,
110 ensuring flow/group operations to always be successful.
111- Improved the UP4's distributed UPF replication protocol,
112 fixing a known issue that was causing missing or stale flows in some switches.
113- Implemented a reconciliation mechanisms in the ONOS device manager to
114 facilitate recovery from network partition scenarios
115- Fixed wrong flow handling during network partition scenarios
116- Enabled ZGC garbage collector in ONOS to reduce garbage collector pauses and their effects on the cluster.
117- Upgraded Atomix to 3.1.12 and Karaf to 4.2.14
118
119Testing
120^^^^^^^
121- Built a new Go-based integration test infrastructure for PFCP-Agent,
122 including ``pfcpsim``, a CLI-based tool to emulate SMF interactions in tests.
123 The new infrastructure allows testing PFCP-Agent interaction for both P4-UPF and BESS-UPF.
124- Nightly SD-Fabric integration tests now include PFCP-Agent
125- Added new nightly test cases for new P4-UPF features and persistent controller election
126
127Development
128^^^^^^^^^^^
129- Created a v1model version of the fabric-tna P4 program.
130 The new P4 program allows sharing the same pipeconf and PTF tests as the TNA version of the program,
131 facilitating development and testing in bmv2-based emulated environments.
132
133Upgrade Notes
134-------------
135If you are upgrading from SD-Fabric 1.0, there are several substantial changes that require additional attention.
136
137**ONOS netcfg**
138
139- We renamed all pipeconf names and replaced ``spgw`` with ``upf``
140 (e.g., ``fabric-spgw-int`` is now named ``fabric-upf-int``)
141- We bumped the supported Intel\ |reg| P4 Studio SDE from ``9_5_0`` to ``9_7_0``
142- S1U/N3 address and UE pool configuration for the UP4 app is no longer necessary.
143 Please use the PFCP-Agent ``upf.json`` or helm values to specify them.
144- When using 5G base stations, make sure to set the ``pscEncapEnabled`` flag to true in the UP4's netcfg
145- Because of P4Runtime translation,
146 all port numbers in the ONOS netcfg should match the **singleton port id** used in the Stratum chassis config
147 (instead of the ASIC-specific P4 port number).
148- See :ref:`ONOS Network Config <onos_network_config>` for more details
149
150**PFCP agent**
151
152- We made UE pool configuration in the PFCP Agent mandatory.
153 Make sure to configure ``ue_ip_pool``, under the ``cpiface`` group, in the ``upf.json`` file (or Helm values).
154- Access IP configuration parameter is now mandatory.
155 Make sure to configure ``access_ip`` in the ``upf.json`` file (or Helm values).
156- ``p4rtciface.ue_ip_pool`` has been removed. Use ``cpiface.ue_ip_pool`` instead.
157- Introduced ``slice_id`` field. It defaults to 0 if not specified.
158- See :ref:`PFCP agent config <pfcp_agent_config>` for more details
159
160**Stratum chassis config**
161
162- Since the Stratum's singleton port ID can now be used in the ONOS netcfg for all port numbers
163 (thanks to P4Runtime translation), we recommend using the following easy-to-understand convention:
164
165 - For unchannelized ports: use the same front panel port number,
166 e.g., for port “9/0” use singleton port ID “9”
167 - For channelized ports: use the formula “port number x 100 + channel”,
168 e.g., when configuring channels 9/0, 9/1, 9/2, and 9/3,
169 the corresponding singleton port IDs should be 900, 901, 902, and 903
170- See `Stratum port number <https://github.com/stratum/stratum/blob/main/stratum/hal/bin/barefoot/README.run.md#user-content-port-numbers>`_ for more details
171
172**Docker image**
173
174- All our docker images are now hosted on DockerHub.
175 Remember to update image registry, repo and tag if you were pulling images from the Aether Registry previously
176- We renamed the ``tost`` image to ``sdfabric-onos``
177
178**Helm value**
179
180- ``logging.karafVersion`` should be updated from ``4.2.9`` to ``4.2.14``
181
182
183Known Issues and Limitations
184----------------------------
185- Some INT collectors might not support topologies with dual-homed hosts.
186- In P4-UPF, configuration of slice rate limits is currently not exposed to Aether.
187 We plan to add this in the next minor release (SD-Fabric v1.1.1).
188- In BESS-UPF, when using application filtering with port ranges, we support only ranges up to 100 ports.
189
190Component Versions
191------------------
192
193SD-Fabric ONOS
194^^^^^^^^^^^^^^
195- Image
196
197 - `opennetworking/sdfabric-onos <https://hub.docker.com/r/opennetworking/sdfabric-onos>`_: ``1.1.0``
198
199- Source code
200
201 - `onos <https://github.com/opennetworkinglab/onos>`_: ``2.5.7-rc1``
202 - `trellis-control <https://github.com/opennetworkinglab/trellis-control>`_: ``3.1.0``
203 - `fabric-tna <https://github.com/stratum/fabric-tna>`_: ``1.0.0``
204 - `up4 <https://github.com/omec-project/up4>`_: ``1.0.0``
205
206Stratum
207^^^^^^^
208- Image: `stratumproject/stratum-bfrt <https://hub.docker.com/r/stratumproject/stratum-bfrt>`_: ``22.03.01-9.7.0``
209- Repo: `stratum <https://github.com/stratum/stratum>`_: ``29f391580de8d5ee2d7a207a9a4261fda4e54a6f``
210
211DBUF
212^^^^
213- Image: `opennetworking/dbuf <https://hub.docker.com/r/opennetworking/dbuf>`_: ``1.0.0``
214- Repo: `dbuf <https://github.com/omec-project/dbuf>`_: ``v1.0.0``
215
216PFCP Agent
217^^^^^^^^^^
218- Image: `omecproject/upf-epc-pfcpiface <https://hub.docker.com/r/omecproject/upf-epc-pfcpiface>`_
219
220 - For P4-UPF: ``master-fcdbc95``
221 - For BESS-UPF: ``master-9cb736e``
222
223- Repo: `pfcpiface <https://github.com/omec-project/upf/tree/master/pfcpiface>`_
224
225 - For P4-UPF: ``to-be-updated``
226 - For BESS-UPF: ``to-be-updated``
227
228BESS UPF
229^^^^^^^^
230- Image: `omecproject/upf-epc-bess <https://github.com/omec-project/upf>`_: ``master-9cb736e``
231- Repo: `upf <https://github.com/omec-project/upf>`_: ``to-be-updated``
232
233DeepInsight Utility
234^^^^^^^^^^^^^^^^^^^
235
236- Image: `registry.opennetworking.org/deepinsight/utility <https://registry.opennetworking.org/harbor/projects/38/repositories/utility>`_: ``4.2.0-ONF``
237- Repo: `bf-di-scripts <https://github.com/opennetworkinglab/bf-di-scripts/blob/master/4/utility/>`_: ``fbc6999d4dc92d8f9909df1c9f2e7b66b0edbf2e``
238
239.. note::
240 DeepInsight is a 3rd party licensed software from Intel\ |reg|.
241 It is integrated with SD-Fabric but the image is not distributed as part of the SD-Fabric open source release.
242
243Helm Chart Versions
244-------------------
245- `sdfabric <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/sdfabric>`_: ``1.1.0``
246
247 - `onos-classic chart <https://github.com/onosproject/onos-helm-charts/tree/master/onos-classic>`_: ``0.1.28``
248 - `stratum chart <https://github.com/stratum/stratum-helm-charts/tree/master/stratum>`_: ``0.1.18``
249 - `pfcp-agent chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/pfcp-agent>`_: ``0.0.12``
250 - `dbuf chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/dbuf/>`_: ``0.0.4``
251 - `int-host-reporter chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/int-host-reporter>`_: ``0.0.4``
252 - `di-topology-watchdog chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/di-topology-watchdog>`_: ``0.0.5``
253 - `di-metrics-exporter chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/di-metrics-exporter>`_: ``0.0.3``
254
255- `bess-upf chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/bess-upf>`_: ``0.0.4``
256
257.. |reg| unicode:: U+000AE .. REGISTERED SIGN