iSCSI hook and IOMMU

[parthy]

I've observed issues when using iSCSI sanhook in conjunction with the IOMMU. When the loaded kernel enables the IOMMU, the traffic caused by the iSCSI hook eventually causes DMA faults, because the NIC uses DMA regions that are not registered anywhere.

Since iPXE acts like a kind of firmware in this case, shouldn't it register the NIC and corresponding memory regions in the RMRRs?

[mcb30]

(2018-11-19 14:38)parthy Wrote:  I've observed issues when using iSCSI sanhook in conjunction with the IOMMU. When the loaded kernel enables the IOMMU, the traffic caused by the iSCSI hook eventually causes DMA faults, because the NIC uses DMA regions that are not registered anywhere.

Since iPXE acts like a kind of firmware in this case, shouldn't it register the NIC and corresponding memory regions in the RMRRs?

There is no protocol by which iPXE can negotiate IOMMU registrations with the host OS. What should happen is that the kernel takes over ownership of all hardware, including the IOMMU and the NIC, and ceases to use the INT13 API.

There is some logic within iPXE to quiesce the hardware (i.e. stop DMA) at the points that iPXE expects the OS to potentially take ownership. This is currently implemented only for Hyper-V, but could be extended for other NICs.

Michael

[parthy]

Well, the situation is that the kernel does not have a NIC driver, but configures the IOMMU. The INT13 hook is transparent and continues to function if the IOMMU is configured correctly. The way to report this would be the RMRR descriptors in the DMAR table. I suspect that is how hardware iSCSI mechanisms provided by the firmware do it. The VT-d specification says:

Quote:Section 3.15 described the details of BIOS allocated reserved memory ranges that may be DMA targets. BIOS may report each such reserved memory region through the RMRR structures, along with the devices that requires access to the specified reserved memory region.

This sounds like the appropriate mechanism, if iPXE is viewed as part of the "BIOS".