Download Mellanox OFED Linux User`s Manual

Mellanox OFED for Linux User’s Manual
Rev 1.5.3-1.0.0
# cat /sys/class/infiniband/mlx4_0/ports/1/state
2: INIT
# cat /sys/class/infiniband/mlx4_0/ports/2/state
4: ACTIVE
#
2. Look at the link_layer parameter of each port. In this case port 1 is IB and port 2 is Ethernet.
Nevertheless, port 2 appears in the list of the HCA's ports. You can also run the following
commands to obtain the link_layer of the two ports:
# cat /sys/class/infiniband/mlx4_0/ports/1/link_layer
InfiniBand
# cat /sys/class/infiniband/mlx4_0/ports/2/link_layer
Ethernet
#
3. The firmware version is 2.7.700 (appears at the top). You can also run the following command to obtain the firmware version:
# cat /sys/class/infiniband/mlx4_0/fw_ver
2.7.700
#
4. The IB over Ethernet’s Port MTU is 2K byte at maximum, however the actual MTU cannot
exceed the mlx4_en interface's MTU. Since the mlx4_en interface’s MTU is 1560, port 2
will run with MTU of 1K. Please note that RoCE’s MTU are subject to IB MTU restrictions.
The RoCE’s MTU values are, 256 byte, 512 byte, 1024 byte and 2K.
Association of IB Ports to Ethernet Ports
It is useful to know how IB ports associate to network ports.
# ibdev2netdev
mlx4_0 port 2 <===> eth2
mlx4_0 port 1 <===> ib0
#
Since both RoCE and mlx4_en use the Ethernet port of the adapter, one of the drivers must carry
the task of controlling the port state. In this implementation, it is the task of the mlx4_en driver.
The mlx4_ib driver holds a reference to the mlx4_en net device for getting notifications about the
state of the port, as well as using the mlx4_en driver to resolve IP addresses to MAC that are
required for address vector creation. However, RoCE traffic does not go through the mlx4_en
driver; it is completely offloaded by the hardware.
Configre an IP Address to mlx4_en Interface
Run the following on both sides of the link.
Mellanox Technologies
39