Using litepcie with LiteFury (SQRL CLE-101)

[a3f]

In #38, @enjoy-digital 's recommendation was to try this out with the SQRL CLE-215. I got a LiteFury, which according to litex-boards is equivalent to the CLE-101, its little sibling.

I would like to get it running as PCIe endpoint and read the LiteFury's memory via PCIe (analogously to the standard LiteFury demo).

I am following the steps outlined in https://github.com/litex-hub/litex-boards/blob/master/litex_boards/targets/sqrl_acorn.py#L10:

# disable SATA, otherwise Vivado fails to satisfy the routing
python3 -m litex_boards.targets.sqrl_acorn --driver --variant=cle-101 --with-pcie --build --load
echo 1 > $pcidevice/remove
echo 1 > /sys/bus/pci/rescan

I see the BIOS boot up over UART, but the CLE-101 doesn't reappear in lspci output and the litepcie driver won't bind. Removing and rescanning worked with the Vivado example for LiteFury, so I'd assume it not to be the problem. I can't test with --flash, it reports Error: Unknown flash device (ID 0x00000000) (Flashing the LiteFury example via Vivado to SPI NOR worked though).

So my questions are:

• Did I miss any obvious steps?
• Do I need a specific Vivado version?
• Did you test PCIe with CLE-101? Is there maybe some PCIe specific configuration missing?
• Any other pointers how to further debug this?

Thanks!

[a3f]

I let litex generate me a mcs file that I flashed via Vivado. After a reboot, device pops up on PCIe and I can read CSR over PCIe successfully. :)

What I am still trying to figure out is how t program the dma controller on the device, so it can DMA into host memory. A similar question #50 was already asked, but I don't yet know how I'd let firmware running on the softcore make use of these 'streams'. Do you know of an example? (Classic descriptor ring: Host writes buffer into shared memory, FPGA dma's data into that buffer)

[enjoy-digital]

Hi @a3f,

I was just looking at this and doing a test on a CLE-101 that I don't think has been tested previously.

With python3 -m litex_boards.targets.sqrl_acorn --variant=cle-101 --with-pcie --load, the design builds correctly and is also correctly enumerated.

The DMAs in LitePCIe are able to read/write data from/to the Host memory and generate/absorb streams on the FPGA interface:
Host Memory --> DMA Reader --> Stream of data.
Stream of data --> DMA Writer --> Host Memory.

The core has been mostly used to create SDR or Video systems for now, and that's the reason we only have streaming interfaces. Can describe a little bit more your use-case, I'll try to guide you and tell you which other LiteX modules you can use/combine with LitePCIe's DMA to achieve the functionality you want.

[a3f]

I'd like firmware running on the soft core to draw to a frame buffer in host memory.

[enjoy-digital]

@a3f: For this you can use the AXI frontend: https://github.com/enjoy-digital/litepcie/blob/master/litepcie/frontend/axi.py#L17 that will provide you a standardized AXI interface to read/write to the Host memory and then just connect it to the LiteX SoC. This will allow your softcore to directly access the Host memory as if it was a local peripheral of the SoC.

Something like this should work:

            # Map Host Memory to SoC.
            from litex.soc.integration.soc import SoCRegion
            from litex.soc.interconnect.axi import AXILiteInterface, AXI2AXILite
            from litepcie.frontend.axi import LitePCIeAXISlave
            pcie_host_axi_mmap = LitePCIeAXISlave(self.pcie_endpoint, data_width=32)
            pcie_host_axi_lite = AXILiteInterface(data_width=32)
            self.submodules += pcie_host_axi_mmap, AXI2AXILite(axi=pcie_host_axi_mmap.axi, axi_lite=pcie_host_axi_lite)
            self.bus.add_slave("pcie_host", pcie_host_axi_lite, SoCRegion(origin=0x3000_0000, size=0x0100_0000)) # 16MB.

[a3f]

Thanks! This SoCRegion(origin=0x3000_0000, size=0x0100_0000)) is 1:1 mapped to host memory, right? Is there already a built-in way to make the window in host memory configurable? i.e.

• Host allocates buffer and passes physical address to FPGA (e.g. via CSR)
• FPGA maps this region into its own address space
• Software can DMA into dedicated host memory

[enjoy-digital]

The region will be 1:1 mapped to memory yes. I'm also interested by this use case, but haven't used it yet. I don't currently have a built-in way to do it but most of it can probably be easily adapted from the actual code. Happy to give you more directions if needed if you want to contribute this.

[axibo-reiner]

In #38, @enjoy-digital 's recommendation was to try this out with the SQRL CLE-215. I got a LiteFury, which according to litex-boards is equivalent to the CLE-101, its little sibling.

I would like to get it running as PCIe endpoint and read the LiteFury's memory via PCIe (analogously to the standard LiteFury demo).

I am following the steps outlined in https://github.com/litex-hub/litex-boards/blob/master/litex_boards/targets/sqrl_acorn.py#L10:

# disable SATA, otherwise Vivado fails to satisfy the routing
python3 -m litex_boards.targets.sqrl_acorn --driver --variant=cle-101 --with-pcie --build --load
echo 1 > $pcidevice/remove
echo 1 > /sys/bus/pci/rescan

I see the BIOS boot up over UART, but the CLE-101 doesn't reappear in lspci output and the litepcie driver won't bind. Removing and rescanning worked with the Vivado example for LiteFury, so I'd assume it not to be the problem. I can't test with --flash, it reports Error: Unknown flash device (ID 0x00000000) (Flashing the LiteFury example via Vivado to SPI NOR worked though).

So my questions are:

• Did I miss any obvious steps?
• Do I need a specific Vivado version?
• Did you test PCIe with CLE-101? Is there maybe some PCIe specific configuration missing?
• Any other pointers how to further debug this?

Thanks!

Did you find a solution for this? I am experiencing the same thing with the demo on the 101 out of the box.