blob: 08763c908f923107d027324d79421cc768fc9263 [file] [log] [blame]
Charles Chan671e3982022-03-09 19:51:31 -08001.. SPDX-FileCopyrightText: 2021 Open Networking Foundation <info@opennetworking.org>
2.. SPDX-License-Identifier: Apache-2.0
3
4SD-Fabric 1.1
5=============
6
7After one year of incubation under the ONF's member-only license,
8we are excited to present the first `open-source release of SD-Fabric under the Apache 2.0 license
9<https://opennetworking.org/news-and-events/press-releases/onfs-leading-private-5g-connected-edge-platform-aether-now-released-to-open-source/>`_
10
11SD-Fabric 1.1 comes with numerous new features and improvements.
12The main focus for this release is full integration with Aether,
13ONF’s private-5G edge connectivity solution for enterprises.
14SD-Fabric brings the many benefits of network programmability to Aether,
15from a hardware-accelerated highly-available P4-UPF, to fabric-wide slicing and QoS,
16and per-packet visibility via Inband-Network Telemetry (INT).
17
18In this release, we provide two UPF options: the switch-based P4-UPF, offering the highest performance;
19and the CPU-based BESS-UPF, tailored for deployment flexibility and horizontal scalability.
20Both UPFs come with numerous bug fixes as well as integration with Aether 2.0 features,
21such as support for 5G base stations, application filtering, QoS via multi-level rate-limiting,
22and enhanced visibility with per-flow metrics and INT integration.
23
24Moreover, we provide many operational improvements such as better support for
25enterprise networks, simplified configuration, improved stability, and reduced resource usage.
26
27We hope the transition to open source will catalyze the adoption of SD-Fabric and
28attract more users and contributors from the community.
29Please visit the :ref:`SD-Fabric documentation website <index>` to learn more about
30all the features SD-Fabric offers and how to get started.
31
32Feature Highlights
33------------------
34
35Open Source Release
36^^^^^^^^^^^^^^^^^^^
37All SD-Fabric components are now completely open source under the Apache 2.0 license.
38We have published all source code, docker images, helm charts, documentation, and mailing lists.
39
40Aether Integration
41^^^^^^^^^^^^^^^^^^
42This release provides the following integrations with Aether 2.0:
43
44- **4G/5G slice based on P4-UPF**:
45 which can now be instantiated using the Aether management portal or APIs.
46 In this release, we support creating only one P4-UPF slice,
47 creation of more P4-UPF slices will come in future releases.
48 As before, more than one slice can be created when using BESS-UPF.
49- **Initial INT integration**:
50 this release adds di-metrics-exporter,
Carmelo Cascone827e8a52022-03-10 16:54:22 -080051 a new service which fetches and aggregates real-time INT-based metrics and anomalies
52 from Intel\ |reg| DeepInsight |tm|,
Charles Chan671e3982022-03-09 19:51:31 -080053 to produce per-UE health status reports to be consumed by the Aether Analytics Engine (work-in-progress).
54 SD-fabric supports monitoring of UE traffic when using both P4-UPF and BESS-UPF.
55
56P4-UPF
57^^^^^^
58This release aims at bringing feature parity with BESS-UPF. To this end, P4-UPF now supports
59the following new features:
60
61- **Application filtering**:
62 where the UPF can drop or forward traffic based on application endpoints
63 (IPv4 prefix and port range) configured using PFCP SDF filters received from SD-Core.
64 We support all ROC policies: ALLOW-ALL, DENY-ALL, ALLOW-PUBLIC.
65- **Multi-level rate-limiting**:
66 where traffic can be policed at the application level, UE level, or slice level.
67 Per-application and per-UE rate limits can be configured by means of PFCP QERs received from SD-Core.
68
69BESS-UPF
70^^^^^^^^
71- **Port ranges in app filtering rules**:
72 BESS-UPF now supports installing PDRs with SDF filters
73 matching on more than one transport-layer port.
74- **Arbitrary IPv4 prefix lengths in app filtering rules**:
75 including Aether's **allow-public** rule set (which blocks traffic to RFC 1918 private subnets).
76- **Per-flow metrics (experimental)**:
77 when enabled, the BESS pipeline collects metrics such as throughput, latency, jitter, and packet loss.
78 Metrics are collected for each PFCP PDR rule,
79 allowing performance monitoring for multiple application flows for the same UE.
80 Metrics can be exported via the Prometheus endpoint in PFCP-Agent.
81
82Slicing & QoS
83^^^^^^^^^^^^^
84We continue to make slicing and QoS more configurable to adapt to different use cases.
85
86- **ONOS now reads slicing parameters via netcfg**:
87 to support static configuration of arbitrary slices and traffic classes.
88- **Updated config generation scripts**:
89 to automatically generate both Stratum chassis_config and ONOS netcfg file using high-level parameters.
90
91Operations
92^^^^^^^^^^
93Numerous improvements to make it easier to deploy and operate SD-Fabric.
94
95- **Support for P4Runtime port translation**:
96 which allows hiding ASIC-specific internal port numbers from user-facing configuration files.
97 Now, when configuring switch interfaces in both the ONOS netcfg and Stratum chassis config,
98 users can specify front panel port/channel numbers instead of SDK port numbers.
99- **Remove the management server from the data plane**:
100 user traffic is now processed by P4-UPF or BESS-UPF and then
101 forwarded upstream (or to edge-applications) directly by the fabric switches.
102- **Support base stations behind L3-routed network**:
103 supporting real-world enterprise deployments.
104- **Improve support for paired-leaves topology and distributed UPF**:
105 by enabling phased-recovery by default to ensure
106 traffic is not sent to a recovering switch until it is fully programmed, avoiding unnecessary packet drops.
107
108Bug fixes and other improvements
109^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
110- Made primary controller election in ONOS persistent during device and cluster events,
111 ensuring flow/group operations to always be successful.
112- Improved the UP4's distributed UPF replication protocol,
113 fixing a known issue that was causing missing or stale flows in some switches.
114- Implemented a reconciliation mechanisms in the ONOS device manager to
115 facilitate recovery from network partition scenarios
116- Fixed wrong flow handling during network partition scenarios
117- Enabled ZGC garbage collector in ONOS to reduce garbage collector pauses and their effects on the cluster.
118- Upgraded Atomix to 3.1.12 and Karaf to 4.2.14
119
120Testing
121^^^^^^^
122- Built a new Go-based integration test infrastructure for PFCP-Agent,
123 including ``pfcpsim``, a CLI-based tool to emulate SMF interactions in tests.
124 The new infrastructure allows testing PFCP-Agent interaction for both P4-UPF and BESS-UPF.
125- Nightly SD-Fabric integration tests now include PFCP-Agent
126- Added new nightly test cases for new P4-UPF features and persistent controller election
127
128Development
129^^^^^^^^^^^
130- Created a v1model version of the fabric-tna P4 program.
131 The new P4 program allows sharing the same pipeconf and PTF tests as the TNA version of the program,
132 facilitating development and testing in bmv2-based emulated environments.
133
134Upgrade Notes
135-------------
136If you are upgrading from SD-Fabric 1.0, there are several substantial changes that require additional attention.
137
138**ONOS netcfg**
139
140- We renamed all pipeconf names and replaced ``spgw`` with ``upf``
141 (e.g., ``fabric-spgw-int`` is now named ``fabric-upf-int``)
142- We bumped the supported Intel\ |reg| P4 Studio SDE from ``9_5_0`` to ``9_7_0``
143- S1U/N3 address and UE pool configuration for the UP4 app is no longer necessary.
144 Please use the PFCP-Agent ``upf.json`` or helm values to specify them.
145- When using 5G base stations, make sure to set the ``pscEncapEnabled`` flag to true in the UP4's netcfg
146- Because of P4Runtime translation,
147 all port numbers in the ONOS netcfg should match the **singleton port id** used in the Stratum chassis config
148 (instead of the ASIC-specific P4 port number).
149- See :ref:`ONOS Network Config <onos_network_config>` for more details
150
151**PFCP agent**
152
153- We made UE pool configuration in the PFCP Agent mandatory.
154 Make sure to configure ``ue_ip_pool``, under the ``cpiface`` group, in the ``upf.json`` file (or Helm values).
155- Access IP configuration parameter is now mandatory.
156 Make sure to configure ``access_ip`` in the ``upf.json`` file (or Helm values).
157- ``p4rtciface.ue_ip_pool`` has been removed. Use ``cpiface.ue_ip_pool`` instead.
158- Introduced ``slice_id`` field. It defaults to 0 if not specified.
159- See :ref:`PFCP agent config <pfcp_agent_config>` for more details
160
161**Stratum chassis config**
162
163- Since the Stratum's singleton port ID can now be used in the ONOS netcfg for all port numbers
164 (thanks to P4Runtime translation), we recommend using the following easy-to-understand convention:
165
166 - For unchannelized ports: use the same front panel port number,
167 e.g., for port “9/0” use singleton port ID “9”
168 - For channelized ports: use the formula “port number x 100 + channel”,
169 e.g., when configuring channels 9/0, 9/1, 9/2, and 9/3,
170 the corresponding singleton port IDs should be 900, 901, 902, and 903
171- See `Stratum port number <https://github.com/stratum/stratum/blob/main/stratum/hal/bin/barefoot/README.run.md#user-content-port-numbers>`_ for more details
172
173**Docker image**
174
175- All our docker images are now hosted on DockerHub.
176 Remember to update image registry, repo and tag if you were pulling images from the Aether Registry previously
177- We renamed the ``tost`` image to ``sdfabric-onos``
178
179**Helm value**
180
181- ``logging.karafVersion`` should be updated from ``4.2.9`` to ``4.2.14``
182
183
184Known Issues and Limitations
185----------------------------
186- Some INT collectors might not support topologies with dual-homed hosts.
187- In P4-UPF, configuration of slice rate limits is currently not exposed to Aether.
188 We plan to add this in the next minor release (SD-Fabric v1.1.1).
Carmelo Cascone827e8a52022-03-10 16:54:22 -0800189- In P4-UPF, we currently don't support exporting per-port or per-flow metrics
190 via Prometheus. As a consequence, when using the Aether monitoring dashboard,
191 the corresponding graphs will not be populated. We plan to add this in the
192 next major release (SD-Fabric v1.2)
193- In BESS-UPF, when using application filtering with port ranges, we don't
194 support ranges wider than 100 ports. Please create more application filtering
195 rules to cover the desired range.
Charles Chan671e3982022-03-09 19:51:31 -0800196
197Component Versions
198------------------
199
200SD-Fabric ONOS
201^^^^^^^^^^^^^^
202- Image
203
204 - `opennetworking/sdfabric-onos <https://hub.docker.com/r/opennetworking/sdfabric-onos>`_: ``1.1.0``
205
206- Source code
207
pierventred76875b2022-03-11 11:11:30 -0800208 - `onos <https://github.com/opennetworkinglab/onos>`_: ``2.5.7-rc2``
209 - `trellis-control <https://github.com/opennetworkinglab/trellis-control>`_: ``3.2.0``
210 - `fabric-tna <https://github.com/stratum/fabric-tna>`_: ``1.1.0``
211 - `up4 <https://github.com/omec-project/up4>`_: ``1.1.0``
Charles Chan671e3982022-03-09 19:51:31 -0800212
213Stratum
214^^^^^^^
215- Image: `stratumproject/stratum-bfrt <https://hub.docker.com/r/stratumproject/stratum-bfrt>`_: ``22.03.01-9.7.0``
216- Repo: `stratum <https://github.com/stratum/stratum>`_: ``29f391580de8d5ee2d7a207a9a4261fda4e54a6f``
217
218DBUF
219^^^^
220- Image: `opennetworking/dbuf <https://hub.docker.com/r/opennetworking/dbuf>`_: ``1.0.0``
221- Repo: `dbuf <https://github.com/omec-project/dbuf>`_: ``v1.0.0``
222
223PFCP Agent
224^^^^^^^^^^
225- Image: `omecproject/upf-epc-pfcpiface <https://hub.docker.com/r/omecproject/upf-epc-pfcpiface>`_
226
pierventred76875b2022-03-11 11:11:30 -0800227 - For P4-UPF: ``master-3e29b49``
228 - For BESS-UPF: ``master-9a4d86c``
Charles Chan671e3982022-03-09 19:51:31 -0800229
pierventred76875b2022-03-11 11:11:30 -0800230- Repo: `upf <https://github.com/omec-project/upf>`_
Charles Chan671e3982022-03-09 19:51:31 -0800231
pierventred76875b2022-03-11 11:11:30 -0800232 - For P4-UPF: ``3e29b49e96e39e0ba7bf4190a8b0d0aa9413b08d``
233 - For BESS-UPF: ``9a4d86c654a9f60947ebf71d015d4781ed7a95a6``
Charles Chan671e3982022-03-09 19:51:31 -0800234
235BESS UPF
236^^^^^^^^
pierventred76875b2022-03-11 11:11:30 -0800237- Image: `omecproject/upf-epc-bess <https://github.com/omec-project/upf>`_: ``master-9a4d86c``
238- Repo: `upf <https://github.com/omec-project/upf>`_: ``9a4d86c654a9f60947ebf71d015d4781ed7a95a6``
Charles Chan671e3982022-03-09 19:51:31 -0800239
240Helm Chart Versions
241-------------------
242- `sdfabric <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/sdfabric>`_: ``1.1.0``
243
244 - `onos-classic chart <https://github.com/onosproject/onos-helm-charts/tree/master/onos-classic>`_: ``0.1.28``
245 - `stratum chart <https://github.com/stratum/stratum-helm-charts/tree/master/stratum>`_: ``0.1.18``
pierventred76875b2022-03-11 11:11:30 -0800246 - `pfcp-agent chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/pfcp-agent>`_: ``0.0.13``
Charles Chan671e3982022-03-09 19:51:31 -0800247 - `dbuf chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/dbuf/>`_: ``0.0.4``
248 - `int-host-reporter chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/int-host-reporter>`_: ``0.0.4``
249 - `di-topology-watchdog chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/di-topology-watchdog>`_: ``0.0.5``
250 - `di-metrics-exporter chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/di-metrics-exporter>`_: ``0.0.3``
251
pierventred76875b2022-03-11 11:11:30 -0800252- `bess-upf chart <https://gerrit.opencord.org/plugins/gitiles/sdfabric-helm-charts/+/refs/heads/master/bess-upf>`_: ``0.0.5``
Charles Chan671e3982022-03-09 19:51:31 -0800253
Carmelo Cascone8e79f6c2022-04-04 00:25:09 -0700254Patches
255-------
256
257.. toctree::
258 :maxdepth: 2
259 :glob:
260
261 1.1.1
Charles Chanc25edb72022-05-03 19:41:48 -0700262 1.1.2
Carmelo Cascone8e79f6c2022-04-04 00:25:09 -0700263
Charles Chan671e3982022-03-09 19:51:31 -0800264.. |reg| unicode:: U+000AE .. REGISTERED SIGN
Carmelo Cascone827e8a52022-03-10 16:54:22 -0800265.. |tm| unicode:: U+2122 .. TRADE MARK SIGN