I have an NVMe SSD and want to use it as an OSD backend, but I can't create an OSD when I want to enable SPDK(it could work as a block-device).
I did the following things:
Building Ceph with SPDK enabled
I edit the CMakeLists.txt to make sure SPDK will be compiled:
CMAKE_DEPENDENT_OPTION(WITH_SPDK "Enable SPDK" ON
"CMAKE_SYSTEM_PROCESSOR MATCHES i386|i686|amd64|x86_64|AMD64|aarch64" ON)
- I set up my NVMe device by SPDK scripts and then test by:
$ sudo src/spdk/scripts/setup.sh
0000:0a:00.0 (8086 0a54): nvme -> vfio-pci
$sudo src/spdk/scripts/setup.sh status
$sudo src/spdk/examples/nvme/identify/identify
$sudo src/spdk/examples/nvme/hello_world/hello_world
- The output and test results looks good, but when I try to create OSD as Ceph docs, I meet errors:
$sudo ceph-volume lvm create --bluestore --data spdk:0000.0a.00.0
ceph-volume lvm create: error: argument --data: invalid <ceph_volume.util.arg_validators.ValidDevice object at 0x7fda562de1d0> value: 'spdk:0000.0a.00.0'
I try to replace the PCIe-id by serial number but the output is just the same, and I did some other tries but all failed...
And I found this description in Ceph Documentation » ceph-volume » Overview:
ceph-volume was designed to be a modular tool because we anticipate that there are going to be lots of ways that people provision the hardware devices that we need to consider. There are already two: legacy ceph-disk devices that are still in use and have GPT partitions (handled by simple), and lvm. SPDK devices where we manage NVMe devices directly from userspace are on the immediate horizon, where LVM won’t work there since the kernel isn’t involved at all.
So I wonder what should I do correctly or is SPDK not supported in Octopus()?
This troubled me for days and any help will be appreciated!