Network requirements and recommendations

Network requirements

• All network interfaces on a node must be connected to different subnets. A network interface can be a VLAN-tagged logical interface, an untagged bond, or an Ethernet link.
• The network for internal traffic can be non-routable, with minimum 10 Gbit/s bandwidth.
• Nodes are added to clusters by their IP addresses, not FQDNs. Changing the IP address of a node in the cluster will remove that node from the cluster. If you plan to use DHCP in a cluster, make sure that IP addresses are bound to the MAC addresses of the nodes’ network interfaces.
• Each node must have Internet access so that updates can be installed.
• Network time synchronization is required for correct statistics. It is enabled by default via the chronyd service. If you want to use ntpdate or ntpd, stop and disable chronyd first.

Recommendations for network hardware

• Network latency dramatically reduces cluster performance. Use quality network equipment with low latency links. Do not use consumer-grade network switches.
• Do not use desktop network adapters like Intel EXPI9301CTBLK or Realtek 8129 as they are not designed for heavy load and may not support full-duplex links. Also use non-blocking Ethernet switches.
• Only Mellanox ConnectX-4 and ConnectX-5 InfiniBand adapters are supported for the RDMA mode.

• If you use Mellanox adapters and AMD Epyc Rome CPU on nodes, ensure that SR-IOV is enabled in BIOS. Otherwise, this may lead to data loss. For more details, refer to the Knowledge Base at https://kb.acronis.com/content/64948.

• Adapters using the BNX2X driver, such as Broadcom Limited BCM57840 NetXtreme II 10/20-Gigabit Ethernet / HPE FlexFabric 10Gb 2-port 536FLB Adapter, are not recommended. They limit MTU to 3616, which affects the cluster performance.

Recommendations for network security

• Use separate networks (and, ideally albeit optionally, separate network adapters) for internal and public traffic. Doing so will prevent public traffic from affecting cluster I/O performance and also prevent possible denial-of-service attacks from the outside.
• To avoid intrusions, Acronis Cyber Infrastructure should be on a dedicated internal network inaccessible from outside.
• Even though cluster nodes have the necessary iptables rules configured, we recommend using an external firewall for untrusted public networks, such as the Internet.

Recommendations for network performance

• Use one 1 Gbit/s link per each two HDDs on the node (rounded up). For one or two HDDs on a node, two bonded network interfaces are still recommended for high network availability. The reason for this recommendation is that 1 Gbit/s Ethernet networks can deliver 110-120 MB/s of throughput, which is close to sequential I/O performance of a single disk. Since several disks on a server can deliver higher throughput than a single 1 Gbit/s Ethernet link, networking may become a bottleneck.
• For maximum sequential I/O performance, use one 1 Gbit/s link per each hard drive or one 10 Gbit/s link per node. Even though I/O operations are most often random in real-life scenarios, sequential I/O is important in backup scenarios.
• For maximum overall performance, use one 10 Gbit/s link per node (or two bonded for high network availability).
• It is not recommended to configure 1 Gbit/s network adapters to use non-default MTUs (for example, 9000-byte jumbo frames). Such settings require additional configuration of switches and often lead to human error. 10+ Gbit/s network adapters, on the other hand, need to be configured to use jumbo frames to achieve full performance. You will need to configure the same MTU value on each router and switch on the network (refer to your network equipment manuals), as well as on each node’s network card, bond, or VLAN. The MTU value is set to 1500 by default.

Network recommendations for clients

The following table lists the maximum network performance a client can get with the specified network interface. The recommendation for clients is to use 10 Gbps network hardware between any two cluster nodes and minimize network latencies, especially if SSD disks are used.

Maximum client network performance

Storage network interface	Node max. I/O	VM max. I/O (replication)	VM max. I/O (erasure coding)
1 Gbps	100 MB/s	100 MB/s	70 MB/s
2 x 1 Gbps	~175 MB/s	100 MB/s	~130 MB/s
3 x 1 Gbps	~250 MB/s	100 MB/s	~180 MB/s
10 Gbps	1 GB/s	1 GB/s	700 MB/s
2 x 10 Gbps	1.75 GB/s	1 GB/s	1.3 GB/s