How can `NSUserDefaults synchronize` runs so fast?

Question

In my app I want to save user settings in a plist file for each user logs in, I write one class called CCUserSettings which has almost the same interface as NSUserDefaults and it reads and writes a plist file related to the current user id. It works but has poor performance. Every time user calls [[CCUserSettings sharedUserSettings] synchronize], I write a NSMutableDictionary(which keep the user settings) to a plist file, the code below shows synchronize of CCUserSettings omitting some trivial details.

- (BOOL)synchronize {
    BOOL r = [_settings writeToFile:_filePath atomically:YES];
    return r;
}

I suppose NSUserDefaults should write to files when we call [[NSUserDefaults standardUserDefaults] synchronize], but it runs really fast, I write a demo to test, the key part is below, run 1000 times [[NSUserDefaults standardUserDefaults] synchronize] and [[CCUserSettings sharedUserSettings] synchronize] on my iPhone6, the result is 0.45 seconds vs 9.16 seconds.

NSDate *begin = [NSDate date];
for (NSInteger i = 0; i < 1000; ++i) {
    [[NSUserDefaults standardUserDefaults] setBool:(i%2==1) forKey:@"key"];
    [[NSUserDefaults standardUserDefaults] synchronize];
}
NSDate *end = [NSDate date];
NSLog(@"synchronize seconds:%f", [end timeIntervalSinceDate:begin]);


[[CCUserSettings sharedUserSettings] loadUserSettingsWithUserId:@"1000"];
NSDate *begin = [NSDate date];
for (NSInteger i = 0; i < 1000; ++i) {
    [[CCUserSettings sharedUserSettings] setBool:(i%2==1) forKey:@"_boolKey"];
    [[CCUserSettings sharedUserSettings] synchronize];
}
NSDate *end = [NSDate date];
NSLog(@"CCUserSettings modified synchronize seconds:%f", [end timeIntervalSinceDate:begin]);

As the result shows, NSUserDefaults is almost 20 times faster than my CCUserSettings. Now I start to wonder that "Does NSUserDefaults really write to the plist files every time we call synchronize method?", but if it doesn't, how can it guarantee the data write back to file before the process exits(as the process may be killed at any time)?

These days I come up with an idea to improve my CCUserSettings, it is mmap Memory-mapped I/O. I can map a virtual memory to a file and every time user calls synchronize, I create a NSData with NSPropertyListSerialization dataWithPropertyList:format:options:error: method and copy the data into that memory, operating system will write memory back to file when process exits. But I may not get a good performance because the file size is not fixed, every time the length of data increases, I have to remmap a virtual memory, I believe the operation is time consuming.

Sorry for my redundant details, I just want to know how NSUserDefaults works to achieve so good performance, or can anyone have some good advices to improve my CCUserSettings ?

Answer 1

On modern operating systems (iOS 8+, macOS 10.10+), NSUserDefaults does not write the file when you call synchronize. When you call -set* methods, it sends an async message to a process called cfprefsd, which stores the new values, sends a reply, and then at some later time writes the file out. All -synchronize does is wait for all outstanding messages to cfprefsd to receive replies.

(edit: you can verify this, if you like, by setting a symbolic breakpoint on xpc_connection_send_message_with_reply and then setting a user default)

Answer 2

Finally I come up with an solution to improve the performance of my CCUserSettings with mmap, I call it CCMmapUserSettings.

Prerequisite

The synchronize in CCUserSettings or NSUserDefaults method writes the plist file back to the disk, it costs notable time, but we must call it in some situations like when app goes into background. Even so we take the risk of losing the settings: we apps may be killed by system because it runs out of memory or accesses a address which it hasn't permission to, at that time the settings we set after the latest synchronize may lose.

If there is a way that we can write the file to the disk when the process exits, we can modify the settings in memory all the time, it's pretty fast. But is there a way to achieve that ?

Well, I find one, it is mmap, mmap maps a file to a region of memory. When this is done, the file can be accessed just like an array in the program. So we can modify the memory as if we write the file. When process exits, the memory will write back to the file.

There are two links supporting me:

Does the OS (POSIX) flush a memory-mapped file if the process is SIGKILLed?

mmap, msync and linux process termination

Problem of using mmap

As I mentioned in my question:

These days I come up with an idea to improve my CCUserSettings, it is mmap Memory-mapped I/O. I can map a virtual memory to a file and every time user calls synchronize, I create a NSData with NSPropertyListSerialization dataWithPropertyList:format:options:error: method and copy the data into that memory, operating system will write memory back to file when process exits. But I may not get a good performance because the file size is not fixed, every time the length of data increases, I have to remmap a virtual memory, I believe the operation is time consuming.

The problem is: every time the length of data increases, I have to remmap a virtual memory, it is time consuming operation.

Solution

Now I have a solution: always create a bigger size than we need and keep the real file size in the beginning 4 bytes of the file and write the real data after the 4 bytes. As the file is bigger than what we need, when the data is increasing smoothly, we don't need to remmap memory at every call of synchronize. There is another restriction on file size : file size is always multiple of MEM_PAGE_SIZE(defined as 4096 in my app).

The synchronize method:

- (BOOL)synchronize {
    if (!_changed) {
        return YES;
    }
    NSData *data = [NSPropertyListSerialization dataWithPropertyList:_settings format:NSPropertyListXMLFormat_v1_0 options:0 error:nil];
    // even if data.length + sizeof(_memoryLength) is a multiple of MEM_PAGE_SIZE, we need one more page.
    unsigned int pageCount = (unsigned int)(data.length + sizeof(_memoryLength)) / MEM_PAGE_SIZE + 1;
    unsigned int fileSize = pageCount * MEM_PAGE_SIZE;
    if (fileSize != _memoryLength) {
        if (_memory) {
            munmap(_memory, _memoryLength);
            _memory = NULL;
            _memoryLength = 0;
        }

        int res = ftruncate(fileno(_file), fileSize);
        if (res == -1) {
            // truncate file error
            fclose(_file);
            _file = NULL;
            return NO;
        }
        // re-map the file
        _memory = (unsigned char *)mmap(NULL, fileSize, PROT_READ|PROT_WRITE, MAP_SHARED, fileno(_file), 0);
        _memoryLength = (unsigned int)fileSize;
        if (_memory == MAP_FAILED) {
            _memory = NULL;
            fclose(_file);
            _file = NULL;
            return NO;
        }
#ifdef DEBUG
        NSLog(@"memory map file success, size is %@", @(_memoryLength));
#endif
    }

    if (_memory) {
        unsigned int length = (unsigned int)data.length;
        length += sizeof(length);
        memcpy(_memory, &length, sizeof(length));
        memcpy(_memory+sizeof(length), data.bytes, data.length);
    }
    return YES;
}

An example will help to describe my thought: suppose the plist data size is 5000 bytes, the total bytes I need to write is 4 + 5000 = 5004. I write 4 bytes unsigned integer which value is 5004 first then write the 5000 bytes data. The total file size should be 8192(2*MEM_PAGE_SIZE). The reason I create a bigger file is I need a big buffer to reduce the time to re-mmap memory.

Performance

{
    [[CCMmapUserSettings sharedUserSettings] loadUserSettingsWithUserId:@"1000"];
    NSDate *begin = [NSDate date];
    for (NSInteger i = 0; i < 1000; ++i) {
        [[CCMmapUserSettings sharedUserSettings] setBool:(i%2==1) forKey:@"_boolKey"];
        [[CCMmapUserSettings sharedUserSettings] synchronize];
    }
    NSDate *end = [NSDate date];
    NSLog(@"CCMmapUserSettings modified synchronize seconds:%f", [end timeIntervalSinceDate:begin]);
}

{
    [[NSUserDefaults standardUserDefaults] setBool:YES forKey:@"key"];
    NSDate *begin = [NSDate date];
    for (NSInteger i = 0; i < 1000; ++i) {
        [[NSUserDefaults standardUserDefaults] setBool:YES forKey:@"key"];
        [[NSUserDefaults standardUserDefaults] synchronize];
    }
    NSDate *end = [NSDate date];
    NSLog(@"NSUserDefaults not modified synchronize seconds:%f", [end timeIntervalSinceDate:begin]);
}

{
    NSDate *begin = [NSDate date];
    for (NSInteger i = 0; i < 1000; ++i) {
        [[NSUserDefaults standardUserDefaults] setBool:(i%2==1) forKey:@"key"];
        [[NSUserDefaults standardUserDefaults] synchronize];
    }
    NSDate *end = [NSDate date];
    NSLog(@"NSUserDefaults modified synchronize (memory not change) seconds:%f", [end timeIntervalSinceDate:begin]);
}

The output is:

CCMmapUserSettings modified synchronize seconds:0.037747
NSUserDefaults not modified synchronize seconds:0.479931
NSUserDefaults modified synchronize (memory not change) seconds:0.402940

It shows that CCMmapUserSettings runs faster than NSUserDefaults!!!

I'm not sure

CCMmapUserSettings passes the unit settings on my iPhone6 (iOS 10.1.1), but I really not sure if it works on all iOS versions because I haven't gotten a official document to make sure the memory used to map the file will be written back to disk immediately when the process exits, if it's not, will it be written to disk before the device shuts down ?

I think I have to study the system behavior about mmap, if anyone of you knows that, please share. Thanks very much.