I am searching for the unit size of the MMIO request when sending to the PCIe device BAR region from CPU.
I used this code. In the below code, you can see that the MMIO accessing code between the "////////////////" line. I just used 32 bit value(pointer) to use for READ/WRITE request to PCIe device MMIO, I'm not sure what this request will be seperated?? or gathered(if the R/W request is followed more)?? and if the request will be separated or gathered, what size will be?? what the payload unit size will be??
#define MMIO_PHY_ADDR 0x20bf80000000
#define MMIO_SIZE 4096 // 4KB, that i want to map on the memory (Total BAR is 1GB but i used only 4KB for test)
#define MMIO_ACCESS_SIZE (MMIO_SIZE / sizeof(uint32_t)) // Number of 32-bit units
int main() {
int fd = open("/dev/mem", O_RDWR | O_SYNC);
if (fd < 0) {
perror("Failed to open /dev/mem");
return EXIT_FAILURE;
}
void* mmio_base = mmap(nullptr, MMIO_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, MMIO_PHY_ADDR);
if (mmio_base == MAP_FAILED) {
perror("Failed to mmap");
close(fd);
return EXIT_FAILURE;
}
////////////////////////////////////////////////////////////
printf("Memory mapped at virtual address %p.\n", mmio_base);
uint32_t value = *((uint32_t*)mmio_base);
std::cout << "Read value: 0x" << std::hex << value << std::endl;
*((uint32_t*)mmio_base) = 0x12345678;
std::cout << "Wrote 0x12345678 to the mapped memory" << std::endl;
value = *((uint32_t*)mmio_base);
std::cout << "Read value: 0x" << std::hex << value << std::endl;
////////////////////////////////////////////////////////////
Saying for just in case, my environment is Linux Ubuntu 5.4.0-172-generic and I am using Xilinx FPGA with Xilinx PCIe IP configured by myself as the PCIe device. Below is the lspci result of the PCIe device.
27:00.0 Multimedia video controller: Device ****:**** (rev ****)
Subsystem: **** Device ****
Physical Slot: 2
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort+ <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 16
NUMA node: 0
Region 0: Memory at 20bfc0000000 (64-bit, prefetchable) [size=4K]
Region 2: Memory at 20bf80000000 (64-bit, prefetchable) [size=1G]
Capabilities: [80] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <256ns, L1 <2us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 75.000W
DevCtl: CorrErr- NonFatalErr- FatalErr+ UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM not supported
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 8GT/s (ok), Width x8 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-, NROPrPrP-, LTR-
10BitTagComp-, 10BitTagReq-, OBFF Not Supported, ExtFmt-, EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS-, TPHComp-, ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
AtomicOpsCtl: ReqEn+
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+, EqualizationPhase1+
EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-
Capabilities: [e0] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [f8] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [100 v1] Vendor Specific Information: ID=**** Rev=**** Len=**** <?>
Capabilities: [120 v1] Address Translation Service (ATS)
ATSCap: Invalidate Queue Depth: 00
ATSCtl: Enable-, Smallest Translation Unit: 00
Capabilities: [200 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC+ UnsupReq+ ACSViol-
UESvrt: DLP+ SDES+ TLP+ FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr+ BadTLP+ BadDLLP+ Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
AERCap: First Error Pointer: 0f, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [300 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn-, PerformEqu-
LaneErrStat: LaneErr at lane: 0 5 6 7
I just used 32 bit value(pointer) to use for READ/WRITE request to PCIe device MMIO, I'm not sure what this request will be seperated?? or gathered(if the R/W request is followed more)?? and if the request will be separated or gathered, what size will be?? what the payload unit size will be??