Do PCI and PCIe allow change BAR value to remap device registers to new address?

Question

(Apologize for my previous question, a code mistake in my kernel lead to wrong values of BAR's "needed mapping space", and it misleads me written a wrong question description.)

I'm implementing a simple NVMe driver for my own kernel working on x86-64 PC. Due to my poor english, although I have read PCIe spec and other materials, I'm not sure about these things:

1. What's the meaning of initial values of BARs(pre-allocated address for device done by firmware or something else, or just a random value)?
2. Does PCI or PCIe hardware allow software re-assign that address? Can the mapping adress be changed by writing another value to BAR?
3. The "PCI Express Technology 3.0" concludes that BAR with different combinations of flag bits describes 3 kind of address space: 32bit memory space, 64bit memory space and i/o space.Does that means 3 different ways to communicate with device: 32bit mmio, 64bit mmio and 16bit pio(using x86 in/out instructions)?

On VMware I defined a 32GB virtual NVMe SSD, and the corresponding values is: 0xfea00004 (initial value of BAR) 0xffffc000 (lowest non-zero bit is the 14th bit, so the "space" value is 0x4000)

Does it mean the SSD's registers have been mapped to address from 0xfea00000, the mapping space it needs is 16KB. Is 16KB absolutely enough for this SSD?

By the way, If the initial values of BAR were assigned by firmware or something else, are they been well arranged with no overlapping?

Answer 1

I'll reply in interleaved style, given that your question has many questions:

1. What's the meaning of initial values of BARs(pre-allocated address for device done by firmware or something else, or just a random value)?

The firmware set it, therefore, you don't need to do the enumeration and can talk to the device.

2. Does PCI or PCIe hardware allow software re-assign that address? Can the mapping adress be changed by writing another value to BAR?

Yes, it can be change, however you need to make sure that the bridges upstream of it are set to route the packets. All this is done as part of the PCIe enumeration process. It is common for the OS to do the PCIe bus enumeration (again) and discard the old values set by the firmware.

3. The "PCI Express Technology 3.0" concludes that BAR with different combinations of flag bits describes 3 kind of address space: 32bit memory space, 64bit memory space and i/o space.Does that means 3 different ways to communicate with device: 32bit mmio, 64bit mmio and 16bit pio(using x86 in/out instructions)?

There are 3 address spaces:

• Memory space: this can be addressed with 32 bit or with 64 bit addresses, obviously, you use 32 bit to address the first 4GB and 64 bit for the rest.
• IO space: this can be addressed with 16 bit ports or 32 bit ports (some architectures support 32 bit ports)
• Configuration space: this is what you use to set the BARs, it's 4KB at most.

Does it mean the SSD's registers have been mapped to address from 0xfea00000, the mapping space it needs is 16KB. Is 16KB absolutely enough for this SSD?

That's right, 16KB is enough for this case, the most important structures (like the command queues and the completion queues) live in RAM and therefore, they don't consume device memory space.

By the way, If the initial values of BAR were assigned by firmware or something else, are they been well arranged with no overlapping?

Yes, if the firmware did the PCIe enumeration before passing control to the OS, you can expect that it was done right and there are no overlaps