This paper on a malloc() replacement that DOES COMPACTION even on C/C++ is making the rounds: arxiv.org/pdf/1902.04738.pdf

Scarily beautiful.

@federicomena Ooh, that's clever. I was ready to ramble about how that can already work at the OS level but they addressed that concern right away and added another layer to it that cleverly fixes the problem with that.

@fluffy I'm boggling at the part where it installs a segfault handler to catch writes to pages that it is in the process of compacting. Truly a lovely, scary hack.

@federicomena @fluffy yikes, that might kill any chance of putting it into a general purpose C library, signal handlers in general can only be set by the application

@zwol @fluffy I'd love to see a survey of how programs use signal handlers 🧐 It's hard for me to swallow these together:

* sigsegv as "game over, program is buggy"
* sigsegv as "perfectly legitimate way to catch writes to a page, who's your VMM now"

And of course I'm biased, but using this allocator on a memory safe language seems like a wonderful opportunity.

@federicomena @fluffy I was thinking about this with my “occasional contributor to glibc” hat on, and the issue there is that signal handlers are process globals, which means libraries mustn’t touch them. Also, come to think of it, nothing stops you from using sigprocmask to block delivery of SIGSEGV and (hurriedly writes test program) this causes both Linux and BSD kernels to kill the process instead of invoking a handler.

Follow

@federicomena @fluffy Chained handlers are a thing, yes, but not one I would consider reliable enough to use in the guts of malloc. That’s me though.

@zwol @federicomena yeah that’s absolutely fair and I would expect anyone who’s using this allocator to specifically know they’re doing it and only use LD_PRELOAD as a reliable means of overriding the libc one. Because overriding a default allocator in C++ at the language level is a gigantic pain in the butt.

@fluffy @federicomena I’m pretty seriously thinking about writing a variant that stops the world during copies and doing some benchmark bake-offs. It might not be slower, and stopping the world is, oddly enough, something the C library *can* safely do

@zwol thanks, now I can't unsee that loop. But it *is* pretty clever 😮

Sign in to participate in the conversation
Mastodon

Server run by the main developers of the project 🐘 It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!