External Layer 3 Connectivity
In addition to configuring networks and floating IPs within a given site (see Basic Networking), we provide a shared network named “fabnetv4”, which can route traffic via the FABRIC testbed’s layer3 experimental network. Nodes attached to this network may send traffic to “fabnetv4” on other Chameleon sites, or to anythign in the FABNET IPv4 address space, without needing public IPs or traversing a Neutron router.
Besides being easier to set up, this provides lower latency, higher bandwidth, and overall more flexiblity for multi-site experiments.
Using FabnetV4
Site |
Current status |
Fabnet Router IP |
UC |
Available |
10.191.128.1/23 |
TACC |
Available |
10.191.130.1/23 |
At sites where the feature is available, you will be able to see the fabnetv4 network in Horizon,
Or by executing:
OS_CLOUD=tacc openstack network show fabnetv4 --fit-width
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| id | 2a3f6c68-5f98-4d6f-ad9b-f405479aaec2 |
| name | fabnetv4 |
| project_id | 570aad8999f7499db99eae22fe9b29bb |
| provider:network_type | vlan |
| provider:physical_network | physnet1 |
| provider:segmentation_id | 3499 |
| router:external | Internal |
| shared | True |
| status | ACTIVE |
| subnets | 58283a6f-d834-4ba4-baf3-acfb59dc8648 |
+---------------------------+--------------------------------------+
You can therefore use it the same way as you would use sharednet1, but DHCP will provide an extra route, directing traffic towards 10.128.0.0/10 via the Fabnet router IP for the site.
To launch a server on fabnet:
openstack server create \
--network fabnetv4 \
..<other usual options>
Note
The fabnet router will not send traffic to the public internet. All traffic via floating IPs or otherwise internet bound will still traverse a neutron router at the chameleon site, as with any other isolated network.
After launching your instance, the following traceroutes demonstrate the new paths.
Tracroute from TACC to Google public DNS still traverses the Neutron rotuer:
cc@fabnet-v4-test:~$ mtr -n 8.8.8.8 --report
Start: 2024-03-01T00:55:07+0000
HOST: fabnet-v4-test Loss% Snt Last Avg Best Wrst StDev
1.|-- 10.191.131.254 0.0% 10 0.1 0.1 0.1 0.3 0.0
2.|-- 129.114.109.254 0.0% 10 4.5 4.3 1.1 12.8 4.4
3.|-- 129.114.0.142 0.0% 10 0.6 6.4 0.5 33.7 12.3
4.|-- 192.124.226.21 0.0% 10 6.0 6.1 5.8 6.9 0.3
5.|-- 192.124.228.2 0.0% 10 6.3 6.2 6.1 6.3 0.1
6.|-- 108.170.231.42 0.0% 10 7.3 7.4 7.2 7.6 0.1
7.|-- 142.251.71.113 0.0% 10 6.3 6.3 6.2 6.4 0.1
8.|-- 8.8.8.8 0.0% 10 6.3 6.3 6.1 6.4 0.1
Traceroute from TACC to closest FABNET router shows a layer 2 path:
HOST: fabnet-v4-test Loss% Snt Last Avg Best Wrst StDev
1.|-- 10.191.130.1 0.0% 10 0.6 0.7 0.5 0.7 0.1
Tracroute from TACC to FABNET router at STAR shows multiple hops through FABNET:
cc@fabnet-v4-test:~$ mtr -n 10.191.128.1 -r
Start: 2024-03-01T01:02:15+0000
HOST: fabnet-v4-test Loss% Snt Last Avg Best Wrst StDev
1.|-- 10.130.158.1 0.0% 10 0.8 0.7 0.7 0.8 0.1
2.|-- 10.130.128.158 0.0% 10 6.6 6.5 6.4 6.6 0.1
3.|-- 10.133.128.134 0.0% 10 23.2 23.2 23.1 23.3 0.1
4.|-- 10.191.128.1 0.0% 10 42.2 42.2 42.2 42.2 0.0