mellanox

Mellanox: SR-IOV (Single Root IO Virtualization)

Install Mellanox Driver
http://www.panticz.de/mellanox/install-dirver

lspci | grep Mellanox
mstconfig -y -d  18:00.1 set SRIOV_EN=1 NUM_OF_VFS=16
 
#cat /etc/modprobe.d/mlnx.conf 
#options mlx4_core num_vfs=5 probe_vf=5
 
apt install -y sysfsutils
 
cat <<EOF> /etc/sysfs.d/mlnx-sriov_numvfs.conf
class/net/ens6f0/device/sriov_numvfs = 8
class/net/ens6f1/device/sriov_numvfs = 8
class/net/ens7f0/device/sriov_numvfs = 8
class/net/ens7f1/device/sriov_numvfs = 8
EOF
 
# /boot/grub/grub.cf
intel_iommu=on
 
ll /sys/class/net/en{p,s}*
echo 8 > /sys/class/net/ens6f0/device/sriov_numvfs

Configure VLAN

Mellanox ConnectX-3 Pro UEFI iPXE boot

  Device Type:      ConnectX3Pro
  Part Number:      MCX312B-XCC_Ax
  Description:      ConnectX-3 Pro EN network interface card; 10GigE; dual-port SFP+; PCIe3.0 x8 8GT/s; RoHS R6
  PSID:             MT_1200111023
  PCI Device Name:  /dev/mst/mt4103_pci_cr0
  Port1 MAC:        ec0d9a00aab1
  Port2 MAC:        ec0d9a00aab2
  Versions:         Current        Available    
     FW             2.42.5000      N/A          
     PXE            3.4.0752       N/A          

Flash UEFI firmware
Request UEFI firmware from support@mellanox.com
http://www.panticz.de/mellanox/firmware-update

Install MFT
http://www.panticz.de/install-mellanox-mft

Flash firmware

flint -y -d /dev/mst/mt4103_pci_cr0 -i firmware fw-ConnectX3Pro-rel-2_42_5000-MCX312B-XCC_Ax-FlexBoot-3.4.752-UEFI-14.11.46.bin b
  Device Type:      ConnectX3Pro
  Part Number:      MCX312B-XCC_Ax
  Description:      ConnectX-3 Pro EN network interface card; 10GigE; dual-port SFP+; PCIe3.0 x8 8GT/s; RoHS R6
  PSID:             MT_1200111023
  PCI Device Name:  /dev/mst/mt4103_pci_cr0
  Port1 MAC:        ec0d9a00aab1
  Port2 MAC:        ec0d9a00aab2
  Versions:         Current        Available    
     FW             2.42.5000      N/A          
     PXE            3.4.0752       N/A          
     UEFI           14.11.0046     N/A     

Fix iPXE boot issue (recompile ipxe.efi) when connected to LACP swith port
http://www.panticz.de/ipxe/compile

Mellanox: Install Driver

https://www.mellanox.com/products/ethernet-drivers/linux/mlnx_en

#URL=http://www.mellanox.com/downloads/ofed/MLNX_EN-5.1-1.0.4.0/mlnx-en-5.1-1.0.4.0-ubuntu$(lsb_release -rs)-x86_64.tgz
URL=http://www.mellanox.com/downloads/ofed/MLNX_EN-5.2-1.0.4.0/mlnx-en-5.2-1.0.4.0-ubuntu$(lsb_release -rs)-x86_64.tgz
 
wget ${URL} -q -O /tmp/${URL##*/}
tar -C /tmp -xzf /tmp/mlnx-en-*-ubuntu*-x86_64.tgz
cd /tmp/mlnx-en-*-ubuntu*-x86_64
./install --force
 
# prebuild driver only
./install --add-kernel-support-build-only