Download MLNX_EN for Linux User Manual

Rev 3.1-1.0.4
3.4.2.3 Important Notes
•
VXLAN tunneling adds 50 bytes (14-eth + 20-ip + 8-udp + 8-vxlan) to the VM Ethernet
frame. Please verify that either the MTU of the NIC who sends the packets, e.g. the VM
virtio-net NIC or the host side veth device or the uplink takes into account the tunneling
overhead. Meaning, the MTU of the sending NIC has to be decremented by 50 bytes
(e.g 1450 instead of 1500), or the uplink NIC MTU has to be incremented by 50 bytes
(e.g 1550 instead of 1500)
•
From upstream 3.15-rc1 and onward, it is possible to use arbitrary UDP port for
VXLAN. Note that this requires firmware version 2.31.2800 or higher. Additionally, you
need to enable this kernel configuration option CONFIG_MLX4_EN_VXLAN=y.
•
On upstream kernels 3.12/3.13 GRO with VXLAN is not supported
3.5
Resiliency
3.5.1
Reset Flow
Supported in ConnectX-3 and ConnectX-3 Pro only.
Reset Flow is activated by default, once a "fatal device1" error is recognized. Both the HCA and
the software are reset, the ULPs and user application are notified about it, and a recovery process
is performed once the event is raised. The "Reset Flow" is activated by the mlx4_core module
parameter 'internal_err_reset', and its default value is 1.
3.5.1.1 Kernel ULPs
Once a "fatal device" error is recognized, an IB_EVENT_DEVICE_FATAL event is created, ULPs are
notified about the incident, and outstanding WQEs are simulated to be returned with "flush in
error" message to enable each ULP to close its resources and not get stuck via calling its
"remove_one" callback as part of "Reset Flow".
Once the unload part is terminated, each ULP is called with its "add_one" callback, its resources
are re-initialized and it is re-activated.
3.5.1.2 SR-IOV
If the Physical Function recognizes the error, it notifies all the VFs about it by marking their
communication channel with that information, consequently, all the VFs and the PF are reset.
If the VF encounters an error, only that VF is reset, whereas the PF and other VFs continue to
work unaffected.
3.5.1.3 Forcing the VF to Reset
If an outside "reset" is forced by using the PCI sysfs entry for a VF, a reset is executed on that VF
once it runs any command over its communication channel.
1. A “fatal device” error can be a timeout from a firmware command, an error on a firmware closing command, communication channel not being
responsive in a VF. etc.
Mellanox Technologies
53