Run Python Applications Efficiently With malloc_trim
www.softwareatscale.dev
Many large pieces of software - like the Instagram, YouTube, and Dropbox web applications - started off as monolith Python applications. It's extremely easy to build web applications with Python due to its ease of use and high quality open source libraries.
jemalloc wouldn't help in the naive Python interpreter. pymalloc directly uses OS virtual memory allocation routines for its object heap, which is mmap on Linux.
I think given the nature of pymalloc (arenas and pools), it would be beneficial for CPython to provide visualization into fragmentation to understand if things like malloc_trim are actually useful (no doubt that for your workload you must have measured and seen improvements).
A moving GC is actually probably the ideal solution for Python but would be incompatible with CPython semantics (specifically that `id()` is often the pointer address in memory.)
Have you measured any improvement by doing this?
jemalloc wouldn't help in the naive Python interpreter. pymalloc directly uses OS virtual memory allocation routines for its object heap, which is mmap on Linux.
I think given the nature of pymalloc (arenas and pools), it would be beneficial for CPython to provide visualization into fragmentation to understand if things like malloc_trim are actually useful (no doubt that for your workload you must have measured and seen improvements).
A moving GC is actually probably the ideal solution for Python but would be incompatible with CPython semantics (specifically that `id()` is often the pointer address in memory.)