This is a summary from this thread
in the Pylon's mailing list. Copied the summary here for future reference:
a) High pylons virtual memory usage is almost solely caused by the
huge stack allocated on Linux by default (default Linux stack size
is 8MB). It is still not quite clear whether this is really
troublesome (it looks like in typical configurations this stack
remains purely virtual number), but in some configurations may
theoretically cause trouble (like reaching VPS limits)
b) Easy way to force this memory down is to use ulimit to narrow
the stack size. For example
ulimit -s 512
to use 512kB stack (the command may be incjected in shell script
starting paste or in similar place).
More or less equivalent solution for python>=2.5
is to call
import thread
thread.stack_size(512 * 1024)
in the application initialization code. An elegant way is to move
this number to the configuration (like threadpool_worker_stack_size)
c) While tuning the memory consumption it also makes sense to
pay attention to the following runtime parameters:
[server:main]
threadpool_workers = 10 # 10 is default
[app:main]
sqlalchemy.default.pool_size = 3 # 5 is default
sqlalchemy.default.max_overflow = 7 # pool_size+max_overflow = max simultaneous database sessions
Those impact not only virtual, but also real memory usage....
The text above is related to running application under paste, but
similar idea can be applied while using mod_wsgi. If using embedded
mod_wsgi, one may consider using Apache ThreadStackSize directive, if
using mod_wsgi daemon mode (which allows one to configure amount of
working processess and threads in detail) ulimit -s may be used
(future releases of mod_wsgi are also to have configuration parameters
to tune this).
