Gitiles
Code Review Sign In
gerrit.opencord.org / aether-docs / e461233b96edcd3248c35fc26829878c854e9cbd / . / pronto_deployment_guide / acceptance_test_specification.rst
blob: da12424971e45b2fa02b6002ec1afe56e8a63e97 [file] [log] [blame]
Hung-Wei Chiu77c969e2020-10-23 18:13:07 +0000[diff] [blame]1..
2 SPDX-FileCopyrightText: © 2020 Open Networking Foundation <support@opennetworking.org>
3 SPDX-License-Identifier: Apache-2.0
4
5=============================
6Acceptance Test Specification
7=============================
8
9Objectives
10==========
11
12The purpose of this document is to create an end-user test object list (TOL) for Aether Connected Edge (ACE).
13This document will focus on the connectivity services end-user testing.
14In the future, this document will extend to other services offered through ACE.
15
16The Automated continuous testing framework for the platform is out of the scope of this document.
17
18Integration Test (eNB-LTE Core)
19===============================
20Before we start to test End-to-End connectivity, we have to check the connection (called S1-MME/S1-C interface)
21between eNB in an edge and MME in a public cloud.
22In order to verify this connectivity, the following test cases should be passed.
23
24Note that all the following test/verification cases have some assumptions:
25
261. eNB is connected to the Fabric switch;
272. eNB has correct network configurations;
283. eNB has correct ID configurations provided by the ONF PMFE team.
29
30IT-TOL01 Fabric Test 1: the connectivity test within the edge
31^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
32In order to test the fabric test, please see the following steps:
33
34
35+----------------------------------------------+------------------------------------+
36|Steps |Expected Outcome |
37+----------------------------------------------+------------------------------------+
38|1. Access to the eNB* (through SSH or Web GUI)|Able to get ICMP reply(PING reply) |
39| |from the GCP node. |
40| | |
Hyunsun Moonfc751aa2020-11-11 18:49:47 -0800[diff] [blame]41|2. Ping to "10.168.0.6" eNB S1-MME/S1-C |( )YES ( )NO |
Hung-Wei Chiu77c969e2020-10-23 18:13:07 +0000[diff] [blame]42| interface IP address* | |
43| |Comments: |
44+----------------------------------------------+------------------------------------+
45
46.. note::
47 it depends on the eNB device. Some eNBs have a single network interface for the management network, S1-U, and S1-C,
48 while other eNBs have separate interfaces. The former eNB type has a single IP address,
49 and the later eNB type has multiple IP addresses for each interface.
50
51IT-TOL02 Fabric Test 2: the connectivity test between the eNB and the public cloud
52^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
53
54In order to test the fabric test, please see the following steps:
55
56
57+----------------------------------------------+------------------------------------+
58|Steps |Expected Outcome |
59+----------------------------------------------+------------------------------------+
60|1. Access to the eNB* (through SSH or Web GUI)|Able to get ICMP reply(PING reply) |
61| |from the GCP node. |
62| | |
Hyunsun Moonfc751aa2020-11-11 18:49:47 -0800[diff] [blame]63|2. Ping to "10.168.0.6" |( )YES ( )NO |
Hung-Wei Chiu77c969e2020-10-23 18:13:07 +0000[diff] [blame]64| | |
65| |Comments: |
66+----------------------------------------------+------------------------------------+
67
68
69
70.. note::
71 it also depends on the eNB device. Some eNBs allow us to SSH into their device, other eNBs provide the PING tool through Web GUI.
72 Of course, some eNBs do not support both. In that case, it is okay to skip this test case.
73
74
75IT-TOL03 SCTP Connection Verification
76^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
77
78In order to verify the SCTP connection between MME and eNB, please see the following steps:
79
80
81+----------------------------------------------+------------------------------------+
82|Steps |Expected Outcome |
83+----------------------------------------------+------------------------------------+
84|1. SSH to the gateway device* for the eNB |Able to see Heart Beat |
85| S1-MME/S1-C traffic |messages |
86| | |
87|2. Capture the traffic with Wireshark |( )YES ()NO |
88| or the command: `sudo tcpdump -i any sctp` | |
89| |Comments: |
90+----------------------------------------------+------------------------------------+
91
92Capture the traffic with Wireshark or the command: `sudo tcpdump -i any sctp`
93
94.. note::
95 the eNB should have the gateway IP address for the S1-MME/S1-C traffic.
96 You can SSH there with the gateway IP address in the eNB and capture the traffic.
97 Normally, the gateway device can be one of those devices: (i) VPN router; (ii) Firewall device in between VPN router and the edge;
98 (iii) one of edge servers.
99
100IT-TOL04 PLMN Verification
101^^^^^^^^^^^^^^^^^^^^^^^^^^
102
103In order to verify the TAC number, please see the following steps:
104
105
106Able to see the correct MCC and MNC values*
107
108+-------------------------------------------------+------------------------------------+
109|Steps |Expected Outcome |
110+-------------------------------------------------+------------------------------------+
111|1. SSH to the gateway device for the eNB |Able to see the correct MCC and MNC |
112| S1-MME/S1-C traffic |values |
113| | |
114|2. Start to capture the traffic with Wireshark |( )YES ()NO |
115| or the command: `sudo tcpdump -i any sctp | |
116| -w FileName.pcap` |Comments: |
117|3. Reboot eNB | |
118| | |
119|4. Wait until FileName.pcap has `S1SetupRequest` | |
120| S1SetupResponse messages | |
121| | |
122|5. Stop the packet capturing and open | |
123| the FileName.pcap | |
124| | |
125|6. Find out the S1SetupRequest message and | |
126| open the detailed packet information | |
127| | |
128|7. Go to "Item 2: id-SupportedTAs" section | |
129| and check "MACC and "MNC" values | |
130+-------------------------------------------------+------------------------------------+
131
132Example (the MCC is 315 and MNC is 010)
133
134.. image:: images/it-tol04.png
135 :width: 840
136 :height: 840
137 :alt: Example (the MCC is 315 and MNC is 010)
138
139IT-TOL05 TAC Number Verification
140^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
141
142
143+-------------------------------------------------+------------------------------------+
144|Steps |Expected Outcome |
145+-------------------------------------------------+------------------------------------+
146|1. SSH to the gateway device for the eNB |Able to see the correct TAC number |
147| S1-MME/S1-C traffic | |
148| | |
149|2. Start to capture the traffic with Wireshark |( )YES ()NO |
150| or the command: `sudo tcpdump -i any sctp | |
151| -w FileName.pcap` |Comments: |
152|3. Reboot eNB | |
153| | |
154|4. Wait until FileName.pcap has `S1SetupRequest` | |
155| S1SetupResponse messages | |
156| | |
157|5. Stop the packet capturing and open | |
158| the FileName.pcap | |
159| | |
160|6. Find out the S1SetupRequest message and | |
161| open the detailed packet information | |
162| | |
163|7. Go to "Item 0: id-SupportedTAs" section | |
164| and check tAC " | |
165+-------------------------------------------------+------------------------------------+
166
167.. note::
168 if you already captured packets in IT-TOL03, you can skip steps from 1 to 5.
169 Just you can check the expected outcome with the file you captured at IT-TOL03.
170
171Example (the TAC number is 19)
172
173.. image:: images/it-tol05.png
174 :width: 840
175 :height: 840
176 :alt: Example (the TAC number is 19)
177
178IT-TOL06 eNB Verification
179^^^^^^^^^^^^^^^^^^^^^^^^^
180
181In order to test the eNB, please see the following steps:
182
183+-------------------------------------------------+------------------------------------+
184|Steps |Expected Outcome |
185+-------------------------------------------------+------------------------------------+
186|1. SSH to the gateway device for the eNB |Able to see the correct eNBID |
187| S1-MME/S1-C traffic | |
188| | |
189|2. Start to capture the traffic with Wireshark |( )YES ()NO |
190| or the command: `sudo tcpdump -i any sctp | |
191| -w FileName.pcap` |Comments: |
192|3. Reboot eNB | |
193| | |
194|4. Wait until FileName.pcap has `S1SetupRequest` | |
195| S1SetupResponse messages | |
196| | |
197|5. Stop the packet capturing and open | |
198| the FileName.pcap | |
199| | |
200|6. Find out the S1SetupRequest message and | |
201| open the detailed packet information | |
202| | |
203|7. Go to "Item 0: id-Global-ENB-ID" section | |
204| and check "eNB-ID: macroENB-ID" | |
205+-------------------------------------------------+------------------------------------+
206
207.. note::
208 if you already captured packets in IT-TOL03, you can skip steps number 1 to 5.
209 Just you can check the expected outcome with the file you captured at IT-TOL03.
210
211Example (the eNB ID is 19)
212
213.. image:: images/it-tol06.png
214 :width: 840
215 :height: 840
216 :alt: Example (the eNB ID is 19)
217
218Connectivity Services
219=====================
220
221Aether provides only data connectivity for end-user devices and systems.
222So the voice service over LTE is not available. However, users can use
223any OTT services over the Aether network for voice connectivity.
224
225The test specifications are only covering the data connectivity focused tests.
226
227
228CS-TOL01 Device Attach/Connect
229^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
230
231To test device can attach to Aether network
232
233
234+----------------------------------------------+------------------------------------+
235|Steps |Expected Outcome |
236+----------------------------------------------+------------------------------------+
237|1. Turn off the mobile device |Able to attach the device and |
238| |connect to the internet/Aether |
239|2. Turn on the mobile device |Network |
240| | |
241|3. Check whether the device is showing |( )YES ( )NO |
242| connected on the status, depending on | |
243| the device it will show "Aether" or | |
244| "MCCMNC" format. | |
245|4. Browse http://www.google.com/? |( )YES ( )NO |
246| From the device web browser | |
247| |Comments: |
248+----------------------------------------------+------------------------------------+
249
250CS-TOL02 Device Detach/Disconnect
251^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
252
253To test device can detach/disconnected by user initiation
254
255
256+----------------------------------------------+------------------------------------+
257|Steps |Expected Outcome |
258+----------------------------------------------+------------------------------------+
259|1. Make sure the device is connected to Aether|Able to detach the device and |
260| |disconnect from the internet/Aether |
261|2. Deselect the network (or forget the network|Network |
262| , depending on device configuration) | |
263|3. Try to browse http://www.google.com/? |( )YES ( )NO |
264| From your web browser | |
265| |Comments: |
266+----------------------------------------------+------------------------------------+
267
268
269CS-TOL03 Bandwidth Test - Internet
270^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
271
272To test bandwidth available to a mobile device over Aether network.
273
274Please note the following, the bandwidth test depends on the eNB hardware,
275your local breakout bandwidth, and the overall radio environment.
276If you face an unexpected result, please explain it in the comment section in the outcome column.
277
278
279+----------------------------------------------+------------------------------------+
280|Steps |Expected Outcome |
281+----------------------------------------------+------------------------------------+
282|1. Open Speedtest app from your mobile device |Expected Bandwidth/Throughput |
283| |observed |
284| | |
285|2. Run Speedtest 3 times, take the average as |( )YES ( )NO |
286| the final result | |
287| |Comments: |
288+----------------------------------------------+------------------------------------+
289
290
291CS-TOL04 Bandwidth Test - Edge Application
292^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
293
294To test bandwidth available to a mobile device over Aether network.
295
296Please note the following, the bandwidth test depends on the eNB hardware,
297your local breakout bandwidth, and the overall radio environment. If you face an unexpected result,
298please explain it in the comment section in the outcome column.
299
300
301+----------------------------------------------+------------------------------------+
302|Steps |Expected Outcome |
303+----------------------------------------------+------------------------------------+
304|1. Initiate FTP Download from a local server |Expected Bandwidth/Throughput |
305| (same location) connected to the enterprise|observed |
306| network (through local breakout) | |
307| | |
308|2. Download 3 times, take the average as the |( )YES ( )NO |
309| final result | |
310| |Comments: |
311+----------------------------------------------+------------------------------------+
312
313
314
315Monitoring Services
316===================
317
318ACE uses the Grafana dashboard for monitoring services.
319Each ACE will be provided with Read-Only Access to our centralized monitoring platform.
320
321
322Application Services
323====================
324
325Aether uses Rancher to onboard applications to ACE.
326Each ACE host will be provided with access to rancher to onboard applications on their ACE cluster.
327