Using page protection to surface pointer/iterator invalidation bugs

37 views Asked by At

Context: A container class we are writing does not guarantee pointer/iterator stability over a certain operation, because the operation has to reallocate in rare circumstances. The danger is that users inadvertently write code that assumes pointer/iterator stability, only triggering the rare reallocation case much later (possibly in a seemingly unrelated change). This would lead to debugging nightmares.

We are considering ways to surface this kind of mistake earlier. As a concrete idea, in debug mode, the container could always (or at least more frequently) reallocate on this operation, even when not strictly necessary.1 Still, code that mistakenly assumes pointer stability might not really break in easily detectable ways (it might just read stale data from the deleted buffer, which could remain undetected for a long time).

Hence we are thinking about "poisoning" the deallocated memory. If we only ever allocate multiples of full pages (this would not have meaningful overhead as the allocations tend to be large anyway), we could apply PAGE_NOACCESS/PROT_NONE to the relevant memory upon deallocation. This would cause faulty code to immediately segfault when handling a stale pointer/iterator. One would presumably have to employ custom memory allocation (VirtualAlloc/mmap) for this to work.

I understand that this is probably taking a naive view on such low-level techniques. That's why I would like to understand:

  • Is this feasible/sane? Or is my model of virtual memory and page protections too simplified?

  • Are there any obvious showstoppers for this idea?

  • Does this run the risk of breaking other tooling (e.g. debuggers, profilers, sanitizers)?

  • (Optionally:) Why is this not done more often? One could imagine the runtime protecting any fully deallocated pages to immediately detect any use-after-free. I imagine there's a reason I haven't seen anything like this, but it's not clear to me why that is.


1 Technically, this would even be doable with zero overhead by messing with the page table, given that we occupy whole pages for the main idea anyway. But I have not found a sane way to do that.

0

There are 0 answers