Which os support copy-on-write?

Question

I try run the following code in Linux and Windows:

const fs = require("fs/promises")
const {
    constants,
    existsSync
} = require("fs")

async function copy() {
    try {
        await fs.writeFile("demo.txt", "Hello World")
        await fs.copyFile("demo.txt", "copy.txt", constants.COPYFILE_FICLONE_FORCE)
    } finally {
        fs.rm("demo.txt")
        if (existsSync("copy.txt")) fs.rm("copy.txt")
    }
}

copy().catch(console.error)

Both failed, error message:

Linux:

[Error: ENOTSUP: operation not supported on socket, copyfile 'demo.txt' -> 'copy.txt'] {
  errno: -95,
  code: 'ENOTSUP',
  syscall: 'copyfile',
  path: 'demo.txt',
  dest: 'copy.txt'
}

Windows:

Error: ENOSYS: function not implemented, copyfile 'demo.txt' -> 'copy.txt'] {
  errno: -4054,
  code: 'ENOSYS',
  syscall: 'copyfile',
  path: 'demo.txt',
  dest: 'copy.txt'
}

The official documentation of nodejs says

fs.constants.COPYFILE_FICLONE_FORCE: The copy operation will attempt to create a copy-on-write reflink. If the platform does not support copy-on-write, then the operation will fail."

Most server operating systems should be Windows server or Linux.

I'm sure the developers of nodejs won't develop a feature that doesn't work, so I'd like to know on what platforms the "fs.constants.COPYFILE_FICLONE_FORCE" file copy method is available. Darwin, BSD, or some other OS?

Here is some reference information:

node version: v16.14.2

windows version: windows 10
windows file system: NTFS

linux version: 5.10.109-1-MANJARO
linux core version: 5.10
linux file system: ext4

Answer 1

Node.js was originally sponsored and heavily developed by Joyent. And Joyent was/is also a big supporter of Illumos (which is based on OpenSolaris which is based on Solaris which is a BSD distribution sold by Sun and later Oracle).

In its early days node was heavily tested on Illumos. And surprise.. Illumos's default filesystem is ZFS which has copy-on-write.

But ZFS is also available on other OSes (At one point Apple even worked on an official ZFS driver for OSX) so you can have copy-on-write if you want if you use something like ZFS. I'm not quite sure if node's copy-on-write support work on other copy-on-write filesystems like Btrfs but it should in theory.

Answer 2

It's a filesystem feature, not OS. If the FS supports it then obviously you can call driver functions directly to do a copy-on-write operation

But if the OS supports that when the underlying also supports CoW then it'll be easier because you can utilize the system calls that'll work on any CoW filesystems (ZFS, Btrfs, XFS, ReFS, APFS among the commont OSes). Since you're on nodejs there'll be another abstraction layer that depends on nodejs to implement the system calls

If you use ext4 then you're out of luck because it doesn't support CoW. On Linux you'll have to use an FS with CoW support like ZFS, Btrfs, XFS. There are a few other filesystems that have CoW support like ext3cow or Next3 but they're outdated

On the Windows side since Windows Server 2016 there's support for copy-on-write on ReFS. There's even a .NET library to avoid calling Win32 API directly. Again NTFS is out of the question, no CoW support at all. Unfortunately as of now libuv used by nodejs hasn't been updated to use the Windows CoW APIs yet. There was a closed PRs though

On macOS you'll also have CoW support with APFS but not HFS or HFS+. It looks like libuv also supports macOS

Answer 3

The hint is in the name ficlone, this refers to the ioctl_ficlone* family of system calls, which was originally part of the Linux btrfs-specific kernel interface.

The system call has since been generalised and turned to become part of the Linux general filesystem interface, so other filesystem drivers might also implement the system call. I believe XFS also implements this system call for similar CoW functionality.

ext4 and ntfs does not support copy on write, so if that's the filesystem you're using, then using ficlone to make CoW files will never work.

This likely also wouldn't work on Windows, since they use a completely different filesystem interface and a completely different way to create CoW files. It's possible that the flag could be shimmed to perform equivalent functionality against Windows Filesystem API when running on ReFS, but I don't think the current version of nodejs does that.

Answer 4

That’s a pretty advanced FS feature.

You’re asking for a specific activity (COPYFILE_FICLONE_FORCE) if the the file system on your system supports it, it will work. If not, it will fail. It’s just saying there is no /automatic/ fallback.

COW is just like letting the FS make a symlink to your copied file, but if (and ONLY IF) you modify that file it will actually make a “real” copy and apply your changes.

So, if you need to make a copy of a file very /very/ quickly.. and maybe sometimes (not all the time) it might need to be modified later, then that option makes good sense. Otherwise, you can probably just do a “regular” copy and let the file system cache take care of the housekeeping.