Optics Driver Installation
Overview
If you did not connect to the HSRN with fiber, please skip this page .
This guide is meant to walk through the installation of drivers necessary to utilize fiber connections with the HSRN. This varies by both vendor and operating system. The HSRN team will expand this guide as more network cards are tested, however they will follow the same general format.
Mellanox/NVIDIA
This guide will cover the installation of drivers for NVIDIA Cards (ConnectX). If the below steps fail or error messages such as This device cannot start appear, it is likely due to a card failure or motherboard compatibility issue. Check your motherboard to ensure the correct PCIe generation and lane count.
Download Utility
Navigate to the NVIDIA DOCA Download page. This page will ask you a series of prompts that eventually output installation instructions. This page may change as updates are released. We have added some notes below to aid in the installation process.
* Host or BlueField? - For any ConnectX cards, select the host-server option.
* Deployment package? - For any ConnectX cards, select the doca-networking option.
* Profile? - For users that only want to connect to the network, select the doca-ofed option.
Offline Installation (Linux)
If you need to install DOCA without a network installation, follow the above steps but select deb (local) for the Installer Type. The first command should be a wget to download the installation file. Enter this command on a device with internet connectivity then transfer the file to some installation media, such as a USB drive, and continue the installation process. Alternatively, use the IPMI virtual media mounting process to handle this remotely with an IPMI-enabled machine.
Switch to Ethernet (Linux)
NVIDIA VPI cards are set to Infiniband by default. This protocol cannot be supported on most networks and must be switched to Ethernet mode. This can be done through BIOS or by running the following commands.
1. Start MST
After installing doca-networking, restart the machine and then start MST sudo mst start
2. Query device path
To ensure that the drivers have been installed and to extract the device path, run the command sudo mst status. You should see a path under MST devices:. For example, we saw /dev/mst/mt4119_pciconf0. Save this path for the next step.
3. Set Ethernet mode.
- For dual port:
mlxconfig -d <device path> set LINK_TYPE_P1=2 LINK_TYPE_P2=2 - For single port:
mlxconfig -d <device path> set LINK_TYPE_P1=2
4. Verify Ethernet mode
mlxconfig -d <device path> q | grep LINK_TYPE
Switch to Ethernet (Windows)
Run the below commands in Powershell as administrator. Windows command line will not understand the command names and will require execution through the installed executable.
1. Install MFT
Install the Mellanox Firmware Tools.
2. Query device
To ensure that the drivers have been installed and to extract the device path, run the command mst status. You should see a path under MST devices:. For example, we saw mt4119_pciconf0. Save this path for the next step.
3. Set Ethernet mode.
If you get an error message saying the device cannot be opened, restart the machine.
* For dual port: mlxconfig -d <device path> set LINK_TYPE_P1=2 LINK_TYPE_P2=2
* For single port: mlxconfig -d <device path> set LINK_TYPE_P1=2
4. Reboot the machine
5. Verify Ethernet mode
mlxconfig -d <device path> q | findstr "LINK_TYPE"