Want to know the PCIe MMIO request payload unit size

32 views Asked by At

I am searching for the unit size of the MMIO request when sending to the PCIe device BAR region from CPU.

I used this code. In the below code, you can see that the MMIO accessing code between the "////////////////" line. I just used 32 bit value(pointer) to use for READ/WRITE request to PCIe device MMIO, I'm not sure what this request will be seperated?? or gathered(if the R/W request is followed more)?? and if the request will be separated or gathered, what size will be?? what the payload unit size will be??

#define MMIO_PHY_ADDR 0x20bf80000000
#define MMIO_SIZE 4096 // 4KB, that i want to map on the memory (Total BAR is 1GB but i used only 4KB for test)
#define MMIO_ACCESS_SIZE (MMIO_SIZE / sizeof(uint32_t)) // Number of 32-bit units

int main() {
    int fd = open("/dev/mem", O_RDWR | O_SYNC);
    if (fd < 0) {
        perror("Failed to open /dev/mem");
        return EXIT_FAILURE;
    }

    void* mmio_base = mmap(nullptr, MMIO_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, MMIO_PHY_ADDR);
    if (mmio_base == MAP_FAILED) {
        perror("Failed to mmap");
        close(fd);
        return EXIT_FAILURE;
    }
    
    ////////////////////////////////////////////////////////////
    printf("Memory mapped at virtual address %p.\n", mmio_base);

    uint32_t value = *((uint32_t*)mmio_base);
    std::cout << "Read value: 0x" << std::hex << value << std::endl;

    *((uint32_t*)mmio_base) = 0x12345678;
    std::cout << "Wrote 0x12345678 to the mapped memory" << std::endl;

    value = *((uint32_t*)mmio_base);
    std::cout << "Read value: 0x" << std::hex << value << std::endl;
    ////////////////////////////////////////////////////////////

Saying for just in case, my environment is Linux Ubuntu 5.4.0-172-generic and I am using Xilinx FPGA with Xilinx PCIe IP configured by myself as the PCIe device. Below is the lspci result of the PCIe device.

27:00.0 Multimedia video controller: Device ****:**** (rev ****)
    Subsystem: **** Device ****
    Physical Slot: 2
    Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort+ <TAbort- <MAbort- >SERR- <PERR- INTx-
    Interrupt: pin A routed to IRQ 16
    NUMA node: 0
    Region 0: Memory at 20bfc0000000 (64-bit, prefetchable) [size=4K]
    Region 2: Memory at 20bf80000000 (64-bit, prefetchable) [size=1G]
    Capabilities: [80] Express (v2) Endpoint, MSI 00
        DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <256ns, L1 <2us
            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 75.000W
        DevCtl: CorrErr- NonFatalErr- FatalErr+ UnsupReq-
            RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
            MaxPayload 256 bytes, MaxReadReq 512 bytes
        DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
        LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM not supported
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 8GT/s (ok), Width x8 (ok)
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Not Supported, TimeoutDis-, NROPrPrP-, LTR-
             10BitTagComp-, 10BitTagReq-, OBFF Not Supported, ExtFmt-, EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS-, TPHComp-, ExtTPHComp-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
             AtomicOpsCtl: ReqEn+
        LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+, EqualizationPhase1+
             EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-
    Capabilities: [e0] MSI: Enable- Count=1/1 Maskable- 64bit+
        Address: 0000000000000000  Data: 0000
    Capabilities: [f8] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [100 v1] Vendor Specific Information: ID=**** Rev=**** Len=**** <?>
    Capabilities: [120 v1] Address Translation Service (ATS)
        ATSCap: Invalidate Queue Depth: 00
        ATSCtl: Enable-, Smallest Translation Unit: 00
    Capabilities: [200 v1] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC+ UnsupReq+ ACSViol-
        UESvrt: DLP+ SDES+ TLP+ FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr+ BadTLP+ BadDLLP+ Rollover- Timeout- AdvNonFatalErr+
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
        AERCap: First Error Pointer: 0f, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
    Capabilities: [300 v1] Secondary PCI Express
        LnkCtl3: LnkEquIntrruptEn-, PerformEqu-
        LaneErrStat: LaneErr at lane: 0 5 6 7

I just used 32 bit value(pointer) to use for READ/WRITE request to PCIe device MMIO, I'm not sure what this request will be seperated?? or gathered(if the R/W request is followed more)?? and if the request will be separated or gathered, what size will be?? what the payload unit size will be??

0

There are 0 answers