Sparsemem-III

This is my third article in the series for Sparsemem. First one dealt with the logic and usability of sparsemem. Second one discussed the code flow and implementation of the Sparsemem Framework.

In this article I will be talking about various testing tools to validate your implentation of sparsemem on your system. Most of these tools are for  generic memory testing and can be used for other MM areas as well.

The most affected area of sparsemem usage is the calls to pfn_to_page() and page_to_pfn(). This can be tested by the following methods:

 1. page fault
 2. memory reclaim
 3. slab(slub/slob) (virt_to_page() will be called.)
 4. high-order page allocator

But 1 and 2 is too heavy to measure the influence. 3 or 4 can be done and out of these 3 is better.

If you don't want to test-in-the-kernel, reading /proc/kpagecount may call tons of pfn_to_page() ....but copy information between userspace <-> kernel is very heavy in general. So, in-kernel test will yield much better results.

The following patch by christoph lameter is pretty useful for doing page faults on bootup and display results on the terminal.
http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg190559.html

The other memory testing tools are as follows:
1) Memtester --  It is a GPL'd userspace utility for testing the memory subsystem for faults
It can be found at the following location:
http://pyropus.ca/software/memtester/
It has a output like the following:

<inux#> ./memtester 256 2
memtester version 4.2.0 (32-bit)
Copyright (C) 2010 Charles Cazabon.
Licensed under the GNU General Public License version 2 (only).
pagesize is 4096
pagesizemask is 0xfffff000
want 256MB (268435456 bytes)
got  256MB (268435456 bytes), trying mlock ...locked.
Loop 1/2:
  Stuck Address       : ok        
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : ok        
  Block Sequential    : ok        
  Checkerboard        : ok        
  Bit Spread          : ok        
  Bit Flip            : ok        
  Walking Ones        : ok        
  Walking Zeroes      : ok        
  8-bit Writes        : ok
  16-bit Writes       : ok
Loop 2/2:
  Stuck Address       : ok        
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : ok        
  Block Sequential    : ok        
  Checkerboard        : ok        
  Bit Spread          : ok        
  Bit Flip            : ok        
  Walking Ones        : ok        
  Walking Zeroes      : ok        
  8-bit Writes        : ok
  16-bit Writes       : ok
Done.

2) Stress -- stress is a deliberately simple workload generator for POSIX systems. It imposes a configurable amount of CPU, memory, I/O, and disk stress on the system.
It can be found here: http://weather.ou.edu/~apw/projects/stress/

Output:
<Linux#> ./stress --cpu 2 --io 1 --vm 1 --vm-bytes 128M --timeout 10s --verbose
stress: info: [419] dispatching hogs: 2 cpu, 1 io, 1 vm, 0 hdd
stress: dbug: [419] using backoff sleep of 12000us
stress: dbug: [419] setting timeout to 10s
stress: dbug: [419] --> hogcpu worker 2 [420] forked
stress: dbug: [419] --> hogio worker 1 [421] forked
stress: dbug: [419] --> hogvm worker 1 [422] forked
stress: dbug: [422] allocating 134217728 bytes ...
stress: dbug: [422] touching bytes in strides of 4096 bytes ...
stress: dbug: [419] using backoff sleep of 3000us
stress: dbug: [419] setting timeout to 10s
stress: dbug: [419] --> hogcpu worker 1 [423] forked
stress: dbug: [422] freed 134217728 bytes
stress: dbug: [422] allocating 134217728 bytes ...
stress: dbug: [422] touching bytes in strides of 4096 bytes ...
stress: dbug: [422] freed 134217728 bytes
stress: dbug: [422] allocating 134217728 bytes ...
stress: dbug: [422] touching bytes in strides of 4096 bytes ...
stress: dbug: [422] freed 134217728 bytes
stress: dbug: [422] allocating 134217728 bytes ...
stress: dbug: [422] touching bytes in strides of 4096 bytes ...
stress: dbug: [422] freed 134217728 bytes
stress: dbug: [422] allocating 134217728 bytes ...
stress: dbug: [422] touching bytes in strides of 4096 bytes ...
stress: dbug: [419] <-- worker 421 signalled normally
stress: dbug: [419] <-- worker 420 signalled normally
stress: dbug: [419] <-- worker 422 signalled normally
stress: dbug: [419] <-- worker 423 signalled normally
stress: info: [419] successful run completed in 10s
Apart from these two you can use googles stress tester tools as well. They can be found here

Let me know of your feedback and comments.