Tuesday, May 14, 2013

Valgrind Tutorial - Memory Debugging tool memcheck



1.     Introduction

Developing any program/application that is free of defects is very important and really challenging. Code review, unit testing, integration testing, system testing, functional testing, etc are different steps in the development lifecycle that helps in detecting defects in software. As the system under development becomes more complex, covering all possible scenarios as part of testing may not be possible as it may be too costly. Similarly, reproducing the fault scenarios reported from the deployment site may not possible in a lab environment under simulation mode. In addition, there can also be other defects which are very difficult to identify and reproduce such as buffer overflow, memory corruption, memory leak etc. Identifying such issues manually takes lot of time and sometimes it is hard to find.





There are many useful developer friendly tools available in the market which helps in identifying the different types of defects that may be present in the system. Some of these tools are free like Valgrind and some of them are licensed like Rational Purifier. Using these tools saves lot of developer and testing time as they help in detecting many defects which are difficult to find otherwise.
This article describes the Memcheck tool supported by Valgrind framework. This tool can be used to detect possible memory leaks that may be present in the system. Memory leak can prove to be very critical, when the application is deployed on hardware that does not have unlimited resources. It can lead to sub-optimal performance and eventually to application crashes. These defects can go undetected during the testing. It is important that memory leaks be removed from any application to the extent possible. The Memcheck tool is an excellent utility that can be made use of to achieve this.

2.     About Valgrind

Valgrind is an instrumentation framework for building dynamic analysis tools. Valgrind framework supports different tools that can be used to detect memory management and threading defects, and profile the program in detail. It also supports building new tools that can be specific to the application under development.
The Valgrind distribution currently includes six production-quality tools and these are
  • a memory error detector
  • two thread error detectors
  • a cache and branch-prediction profiler
  • a call-graph generating cache and branch-prediction profiler
  • a heap profiler
Valgrind framework also includes three experimental tools that are under various stages of development and these are
  • a heap/stack/global array overrun detector
  • a second heap profiler that examines how heap blocks are used
  • a SimPoint basic block vector generator
It runs on the following platforms: X86/Linux, AMD64/Linux, ARM/Linux, PPC32/Linux, PPC64/Linux, S390X/Linux, MIPS/Linux, ARM/Android (2.3.x and later), X86/Android (4.0 and later), X86/Darwin and AMD64/Darwin (Mac OS X 10.6 and 10.7, with limited support for 10.8).
Valgrind is Open source software, and is freely available under the GNU General Public License, version 2.

More details about each Valgrind tools can be found http://valgrind.org/info/tools.html

3.     Installing Valgrind rpm:

Redhat - yum install valgrind
Other unix: sudo apt-get install valgrind

4.     Installing Valgrind Using source:



Extract the files and run the following commands.
./configure
make
make install


5.     Memcheck tool:

The Memcheck tool is included in the Valgrind framework by default. It is aimed primarily for C and C++ programs and mainly helps in detecting memory leaks and memory errors in a program by intercepting all memory related operations.

5.1. Usage:

Valgrind –help will provide complete usage details.

user1@ubuntu:~/vagrind$ valgrind --help
usage: valgrind [options] prog-and-args

  tool-selection option, with default in [ ]:
    --tool=             use the Valgrind tool named [memcheck]

  basic user options for all Valgrind tools, with defaults in [ ]:
    -h --help                 show this message
    --help-debug              show this message, plus debugging options
    --version                 show version
    -q --quiet                run silently; only print error msgs
    -v --verbose              be more verbose -- show misc extra info
    --trace-children=no|yes   Valgrind-ise child processes (follow execve)? [no]
    --trace-children-skip=patt1,patt2,...    specifies a list of executables
                              that --trace-children=yes should not trace into
    --trace-children-skip-by-arg=patt1,patt2,...   same as --trace-children-skip=
                              but check the argv[] entries for children, rather
                              than the exe name, to make a follow/no-follow decision
    --child-silent-after-fork=no|yes omit child output between fork & exec? [no]
    --vgdb=no|yes|full        activate gdbserver? [yes]
                              full is slower but provides precise watchpoint/step
    --vgdb-error=     invoke gdbserver after errors [999999999]
                              to get started quickly, use --vgdb-error=0
                              and follow the on-screen directions
    --track-fds=no|yes        track open file descriptors? [no]
    --time-stamp=no|yes       add timestamps to log messages? [no]
    --log-fd=         log messages to file descriptor [2=stderr]
    --log-file=         log messages to
    --log-socket=ipaddr:port  log messages to socket ipaddr:port

  user options for Valgrind tools that report errors:
    --xml=yes                 emit error output in XML (some tools only)
    --xml-fd=         XML output to file descriptor
    --xml-file=         XML output to
    --xml-socket=ipaddr:port  XML output to socket ipaddr:port
    --xml-user-comment=STR    copy STR verbatim into XML output
    --demangle=no|yes         automatically demangle C++ names? [yes]
    --num-callers=    show callers in stack traces [12]
    --error-limit=no|yes      stop showing new errors if too many? [yes]
    --error-exitcode= exit code to return if errors found [0=disable]
    --show-below-main=no|yes  continue stack traces below main() [no]
    --suppressions= suppress errors described in
    --gen-suppressions=no|yes|all    print suppressions for errors? [no]
    --db-attach=no|yes        start debugger when errors detected? [no]
    --db-command=    command to start debugger [/usr/bin/gdb -nw %f %p]
    --input-fd=       file descriptor for input [0=stdin]
    --dsymutil=no|yes         run dsymutil on Mac OS X when helpful? [no]
    --max-stackframe= assume stack switch for SP changes larger
                              than bytes [2000000]
    --main-stacksize= set size of main thread's stack (in bytes)
                              [use current 'ulimit' value]

  user options for Valgrind tools that replace malloc:
    --alignment=      set minimum alignment of heap allocations [8]

  uncommon user options for all Valgrind tools:
    --fullpath-after=         (with nothing after the '=')
                              show full source paths in call stacks
    --fullpath-after=string   like --fullpath-after=, but only show the
                              part of the path after 'string'.  Allows removal
                              of path prefixes.  Use this flag multiple times
                              to specify a set of prefixes to remove.
    --smc-check=none|stack|all|all-non-file [stack]
                              checks for self-modifying code: none, only for
                              code found in stacks, for all code, or for all
                              code except that from file-backed mappings
    --read-var-info=yes|no    read debug info on stack and global variables
                              and use it to print better error messages in
                              tools that make use of it (Memcheck, Helgrind,
                              DRD) [no]
    --vgdb-poll=      gdbserver poll max every basic blocks [5000]
    --vgdb-shadow-registers=no|yes   let gdb see the shadow registers [no]
    --vgdb-prefix=    prefix for vgdb FIFOs [/tmp/vgdb-pipe]
    --run-libc-freeres=no|yes free up glibc memory at exit on Linux? [yes]
    --sim-hints=hint1,hint2,...  known hints:
                                 lax-ioctls, enable-outer, fuse-compatible [none]
    --kernel-variant=variant1,variant2,...  known variants: bproc [none]
                              handle non-standard kernel variants
    --show-emwarns=no|yes     show warnings about emulation limits? [no]
    --require-text-symbol=:sonamepattern:symbolpattern    abort run if the
                              stated shared object doesn't have the stated
                              text symbol.  Patterns can contain ? and *.




The following options are available for Memcheck tool.

  user options for Memcheck:
    --leak-check=no|summary|full     search for memory leaks at exit?  [summary]
    --leak-resolution=low|med|high   differentiation of leak stack traces [high]
    --show-reachable=no|yes          show reachable blocks in leak check? [no]
    --show-possibly-lost=no|yes      show possibly lost blocks in leak check?
                                     [yes]
    --undef-value-errors=no|yes      check for undefined value errors [yes]
    --track-origins=no|yes           show origins of undefined values? [no]
    --partial-loads-ok=no|yes        too hard to explain here; see manual [no]
    --freelist-vol=          volume of freed blocks queue      [20000000]
    --freelist-big-blocks=   releases first blocks with size >= [1000000]
    --workaround-gcc296-bugs=no|yes  self explanatory [no]
    --ignore-ranges=0xPP-0xQQ[,0xRR-0xSS]   assume given addresses are OK
    --malloc-fill=        fill malloc'd areas with given value
    --free-fill=          fill free'd areas with given value


5.2. Detect memory leak

This section describes the usage of Memcheck tool to detect memory leaks in an application with the help if a sample program. In this program, memory allocated to the pointer variable ‘x’ is not freed even when the program terminates. These types of programming mistakes are usually referred to as memory leaks. And it may become extremely difficult to detect such mistakes, when the application becomes more complex, with memory allocated by one module needs to be shared among many different modules within the application. Sharing of memory once allocated among different application modules that are interested in its contents typically results in better system performance as memory allocation and de-allocations are costly operations.
Sample Program:

user1@ubuntu:~/vagrind$ cat -n sample-prog-memleak.c
     1
     2  #include <stdio.h>
     3  #include <stdlib.h>
     4  int main()
     5  {
     6
     7  char *x = NULL;
     8
     9  x = malloc(10);
    10  if( x == NULL ){
    11     printf("Error in malloc");
    12     return -1;
    13  }
    14
    15  printf("Memory allocation successful\n");
    16
    17  return 0;
    18  }

Compile the sample program with ‘-g’ option which enables symbols. Enabling symbols gives more details about the stack and other details when the program is run later.

user1@ubuntu:~/vagrind$ gcc -g sample-prog-memleak.c -o prog
user1@ubuntu:~/vagrind$ ./prog
Memory allocation successful
user1@ubuntu:~/vagrind$



Due to the small amount of memory leak present in the above sample program, there may not be any change in the behavior of the program. But if the program is a long running one, then this type of programming mistakes (memory leaks) can lead to memory scarcity during runtime. Once this happens, the program can behave inconsistently and may even lead to crashes. It is best practice for a programmer to develop software which is free from memory leaks. The Memcheck tool can be used to identify whether the program has memory leaks or not. To identify any possible memory leaks, the program needs to be run under the supervision of valgrind. To achieve this, the following syntax is used.

Valgrind [Options] program

Below is the output when the sample program is run within the Valgrind framework.

user1@ubuntu:~/vagrind$ valgrind ./prog
==3179== Memcheck, a memory error detector
==3179== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3179== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3179== Command: ./prog
==3179==
Memory allocation successful
==3179==
==3179== HEAP SUMMARY:
==3179==     in use at exit: 10 bytes in 1 blocks
==3179==   total heap usage: 1 allocs, 0 frees, 10 bytes allocated
==3179==
==3179== LEAK SUMMARY:
==3179==    definitely lost: 10 bytes in 1 blocks
==3179==    indirectly lost: 0 bytes in 0 blocks
==3179==      possibly lost: 0 bytes in 0 blocks
==3179==    still reachable: 0 bytes in 0 blocks
==3179==         suppressed: 0 bytes in 0 blocks
==3179== Rerun with --leak-check=full to see details of leaked memory
==3179==
==3179== For counts of detected and suppressed errors, rerun with: -v
==3179== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
user1@ubuntu:~/vagrind$


The Valgrind Head summary shows that there are 10 bytes which are dynamically allocated and they are still reachable.
The Leak summary gives the details about the memory leak. In the above test program, 10 bytes are not freed and it is reported as Leak by Valgrind.
Though Valgrind reports that there is memory leak it is not quite clear which allocated memory is not freed from the output. To get more details about the location of memory allocation, use ‘leak-check=full’ option. This gives stack of the malloc call for which memory is not freed.
Below is the program output under the supervision of Valgrind with leak-check option.


user1@ubuntu:~/vagrind$ valgrind --leak-check=full ./prog
==3210== Memcheck, a memory error detector
==3210== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3210== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3210== Command: ./prog
==3210==
Memory allocation successful
==3210==
==3210== HEAP SUMMARY:
==3210==     in use at exit: 10 bytes in 1 blocks
==3210==   total heap usage: 1 allocs, 0 frees, 10 bytes allocated
==3210==
==3210== 10 bytes in 1 blocks are definitely lost in loss record 1 of 1
==3210==    at 0x402BE68: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==3210==    by 0x8048460: main (sample-prog-memleak.c:9)
==3210==
==3210== LEAK SUMMARY:
==3210==    definitely lost: 10 bytes in 1 blocks
==3210==    indirectly lost: 0 bytes in 0 blocks
==3210==      possibly lost: 0 bytes in 0 blocks
==3210==    still reachable: 0 bytes in 0 blocks
==3210==         suppressed: 0 bytes in 0 blocks
==3210==
==3210== For counts of detected and suppressed errors, rerun with: -v
==3210== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

This output shows comprehensive details about the memory leak that is present in the program. The memory allocated at line number sample-prog-memleak.c:9 is not freed. Referring to the sample program, line no 9 has the following statement.
     9  x = malloc(10);
It means that the memory allocated to variable ‘x’ is not freed when the program finishes its execution.

5.3. Fixing memory leak

To solve the above issue reported by the Memcheck tool, add free(x) at appropriate place within the sample program.


user1@ubuntu:~/vagrind$ cat -n sample-prog-memleak.c
     1
     2  #include <stdio.h>
     3  #include <stdlib.h>
     4  int main()
     5  {
     6
     7  char *x = NULL;
     8
     9  x = malloc(10);
    10  if( x == NULL ){
    11     printf("Error in malloc");
    12     return -1;
    13  }
    14
    15  printf("Memory allocation successful\n");
    16
    17  free(x);
    18
    19  return 0;
    20  }

Compile and run the sample program using Valgrind again. This time Valgrind produces the below output. The Valgrind Summary reports zero leaks.


user1@ubuntu:~/vagrind$ gcc -g sample-prog-memleak.c -o prog
user1@ubuntu:~/vagrind$ ./prog
This is a test program
user1@ubuntu:~/vagrind$ valgrind --leak-check=full ./prog
==3246== Memcheck, a memory error detector
==3246== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3246== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3246== Command: ./prog
==3246==
Memory allocation successful==3246==
==3246== HEAP SUMMARY:
==3246==     in use at exit: 0 bytes in 0 blocks
==3246==   total heap usage: 1 allocs, 1 frees, 10 bytes allocated
==3246==
==3246== All heap blocks were freed -- no leaks are possible
==3246==
==3246== For counts of detected and suppressed errors, rerun with: -v
==3246== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
User1@ubuntu:~/vagrind$

6.     Conclusion


The article explains how developers can make use of the Memcheck tool to detect and resolve any possible memory leaks that may be present in a program. As demonstrated with the help of the sample program, Valgrind Memcheck tool can report possible memory leaks that may be present in the program, accurately. And this enables the issue to be addresses quickly.

7.     References






Pin It
Related Posts Plugin for WordPress, Blogger...