8445

Memory Leak Using malloc fails

I am writing a program to leak memory( main memory ) to test how the system behaves with low system memory and swap memory. We are using the following loop which runs periodically and leaks memory

main(int argc, char* argv[] ) { int arg_mem = argv[1]; while(1) { u_int_ptr =(unsigned int*) malloc(arg_mem * 1024 * 1024); if( u_int_ptr == NULL ) printf("\n leakyapp Daemon FAILED due to insufficient available memory...."); sleep( arg_time ); } }

Above loop runs for sometime and prints the message "leakyapp Daemon FAILED due to insufficient available memory...." . But when I run the command "free" I can see that running this program has no effect either on Main memory or Swap.

Am I doing something wrong ?

Answer1:

Physical memory is not committed to your allocations until you actually write into it.

If you have a kernel version after 2.6.23, use mmap() with the MAP_POPULATE flag instead of malloc():

u_int_ptr = mmap(NULL, arg_mem * 1024 * 1024, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_POPULATE, -1, 0); if (u_int_ptr == MAP_FAILED) /* ... */

If you have an older kernel, you'll have to touch each page in the allocation.

Answer2:

There might be some sort of copy-on-write optimization. I would suggest actually writing something to the memory you are allocating.

Answer3:

What is happening is that malloc requests argmem * 256 pages from the heap (assuming a 4 Kbyte page size). The heap in turn requests the memory from the operating system. However, all that does is create entries in the page table for the newly allocated memory block. No actual physical RAM is allocated to the process, except that required by the heap to track the malloc request.

As soon as the process tries to access one of those pages by reading or writing, a page fault is generated because the entry in the page table is effectively a dangling pointer. The operating system will then allocate a physical page to the process. It's only then that you'll see the available physical memory go down.

Since all new pages start completely zeroed out, Linux might employ a "copy on write" strategy to optimise page allocation. i.e. it might keep a single page totally zeroed and always allocate that one when a process tries to read from a previously unused page. Only when the process tries to write to that new page would it actually allocate a completely fresh page from physical RAM. I don't know if Linux actually does this, but if it does, merely reading from a new page is not going to be enough to increase physical memory usage.

So, your best strategy is to allocate your large block of RAM and then write something at 4096 byte intervals throughout it.

Answer4:

What does ulimit -m -v print?

Explanation: On any server OS, you can limit the amount of resources a process can allocate to make sure that a single runaway process can't bring down the whole machine.

Answer5:

I'm guessing (based on the command line argument) that you're using a desktop/server OS and not an embedded system.

Allocating memory like this is probably not consuming much RAM. Your memory allocation might not have even succeeded - on some OSs (e.g. Linux), malloc() can return non-NULL even when you ask for more memory than is available.

Without knowing what your OS is and exactly what you're trying to test, it's difficult to suggest anything specific, but you might want to look at more low level ways of allocating memory than malloc(), or ways of controlling the virtual memory system. On Linux you might want to look at mlock().

Answer6:

I think caf already explained it. Linux is usually configured to allow overcommitting memory. You allocate huge chunks of memory, but internally there happens nothing but just making a note that you process wants this huge chunk of memory. It's not before you try to write that chunk, that the kernel tries to find free virtual memory to satisfy the read/write access. This is a bit like flight booking: Airlines usually overbook the flights, because there's always a percentage of passengers who do not show up.

You can force the memory to be committed by writing to the chunk with memset() after allocation. calloc should work too.

Recommend

  • mmap failed when trying to map huge page (1GB)
  • Both MAP_ANON and MAP_ANONYMOUS undeclared for sys/mmap in C
  • How to get RGB pixel values from Linux framebuffer?
  • Why mmap a 4GB file on 32-bit armv7l succeeded?
  • segmentation fault due to initialize() function when changing the size of the huge page allocation
  • How to read from /write to anonymous shared mapping?
  • How to Set Custom DataGridView Headers (with spaces) using LINQ?
  • Write non-ASCII characters to stdin of a progam on a tty (over ssh)
  • org.hibernate.AnnotationException: No identifier specified for entity using JPA XML entity-mapping
  • how to limit GPU usage in tensorflow (r1.1) with C++ API
  • What would this code do? (memory management)
  • Pointer help in C/C++ [closed]
  • How to obtain dynamic google map url of a specific map (I have static map url)
  • Compare Dictionary
  • Storing Data in a Variable vs Inline Arithmetic
  • Should concrete implementation provide any public API not present in the interface it implements?
  • “Complex Header” not responsive in current DataTables.net build?
  • App Memory Usage differs between Devices
  • How to get to older Xcode beta version?
  • Gforce min not supported for character in data.table
  • How to add the custom button on google's(device) native application in android?
  • How to call a procedure using NHibernate that returns result from multiple tables?
  • change search magnifying glass to UIActivityIndicatorView
  • SQL: Getting the physical size of a subset of a table
  • how to populate a SQLite database and use that database in phonegap?
  • Most efficient way to move table rows from one table to another
  • Blackberry 6: how to detect a long click on track pad?
  • Redshift Querying: error xx000 disk full redshift
  • std::remove_copy_if_ valgrind bytes in block are possibly lost in loss record
  • Rails Find when some params will be blank
  • Functions in global context
  • Exchange data b/w iOS devices using Bluetooth 4.0
  • Deselecting radio buttons while keeping the View Model in synch
  • htaccess rewriting URLs with multiple forward slashes
  • Display Images one by one with next and previous functionality
  • Web-crawler for facebook in python
  • How can I estimate amount of memory left with calling System.gc()?
  • XCode can't find symbols for a specific iOS library/framework project
  • A cron job substitute?
  • Does armcc optimizes non-volatile variables with -O0?