WineHQ

Wine Developer's Guide/Other Debugging Techniques: Difference between revisions

Line 98: Line 98:
gcov registry.c
gcov registry.c
less registry.c.gcov</pre>
less registry.c.gcov</pre>
The interesting code part looks like this in <code>registry.c.gcov</code>:
The interesting code part looks like this in <tt>registry.c.gcov</tt>:


<source lang="c">    4: 1273:    if (name && name[0])  /* need to create the subkey */
<source lang="c">    4: 1273:    if (name && name[0])  /* need to create the subkey */

Revision as of 17:47, 8 February 2016

__NUMBEREDHEADINGS__

Understanding undocumented APIs

Some background: On the i386 class of machines, stack entries are usually dword (4 bytes) in size, little-endian. The stack grows downward in memory. The stack pointer, maintained in the esp register, points to the last valid entry; thus, the operation of pushing a value onto the stack involves decrementing esp and then moving the value into the memory pointed to by esp (i.e., push p in assembly resembles *(--esp) = p; in C). Removing (popping) values off the stack is the reverse (i.e., pop p corresponds to p = *(esp++); in C).

In the stdcall calling convention, arguments are pushed onto the stack right-to-left. For example, the C call myfunction(40, 20, 70, 30); is expressed in Intel assembly as:

push 30
push 70
push 20
push 40
call myfunction

The called function is responsible for removing the arguments off the stack. Thus, before the call to myfunction, the stack would look like:

         [local variable or temporary]
         [local variable or temporary]
          30
          70
          20
esp ->    40

After the call returns, it should look like:

         [local variable or temporary]
esp ->   [local variable or temporary]

To restore the stack to this state, the called function must know how many arguments to remove (which is the number of arguments it takes). This is a problem if the function is undocumented.

One way to attempt to document the number of arguments each function takes is to create a wrapper around that function that detects the stack offset. Essentially, each wrapper assumes that the function will take a large number of arguments. The wrapper copies each of these arguments into its stack, calls the actual function, and then calculates the number of arguments by checking esp before and after the call.

The main problem with this scheme is that the function must actually be called from another program. Many of these functions are seldom used. An attempt was made to aggressively query each function in a given library (ntdll.dll) by passing 64 arguments, all 0, to each function. Unfortunately, Windows NT quickly goes to a blue screen of death, even if the program is run from a non-administrator account.

Another method that has been much more successful is to attempt to figure out how many arguments each function is removing from the stack. This instruction, ret hhll (where hhll is the number of bytes to remove, i.e. the number of arguments times 4), contains the bytes 0xc2 ll hh in memory. It is a reasonable assumption that few, if any, functions take more than 16 arguments; therefore, simply searching for hh == 0 and ll < 0x40 starting from the address of a function yields the correct number of arguments most of the time.

Of course, this is not without errors. ret 00ll is not the only instruction that can can have the byte sequence 0xc2 ll 0x0; for example, push 0x000040c2 has the byte sequence 0x68 0xc2 0x40 0x0 0x0, which matches the above. Properly, the utility should look for this sequence only on an instruction boundary; unfortunately, finding instruction boundaries on an i386 requires implementing a full disassembler -- quite a daunting task. Besides, the probability of having such a byte sequence that is not the actual return instruction is fairly low.

Much more troublesome is the non-linear flow of a function. For example, consider the following two functions:

somefunction1:
    jmp  somefunction1_impl

somefunction2:
    ret  0004

somefunction1_impl:
    ret  0008

In this case, we would incorrectly detect both somefunction1 and somefunction2 as taking only a single argument, whereas somefunction1 really takes two arguments.

With these limitations in mind, it is possible to implement more stubs in Wine and, eventually, the functions themselves.

How to do regression testing using Git

A problem that can happen sometimes is “it used to work before, now it doesn't anymore...”. Here is a step by step procedure to try to pinpoint when the problem occurred. This is NOT for casual users.

  1. Clone the “Git” repository from winehq. It's more than 100Mb, so it may take some time with a slow Internet connection.

  2. If you found that something broke between wine-1.1.42 and wine-1.1.44 (these are [WWW] release tags). To start regression testing we run:

    git bisect start
    git bisect good wine-1.1.42
    git bisect bad wine-1.1.44

    If you have exact date/time instead of a release you will need to use sha1 IDs from git log.

  3. Having told Git when things were working and when they broke, it will automatically “position” your source tree to the middle. So all you need to do is build the source:

    ./configure && make clean && make depend && make
    ./wine 'c:\test.exe'

    If the version of Wine that Git picked still has the bug, run:

    git bisect bad

    and if it does not, run:

    git bisect good

    When you run this command, Git will checkout a new version of Wine for you to rebuild, so repeat this step again. When the regression has been isolated, git will inform you.

    To find out what's left to test, try:

    git bisect visualize.
  4. When you have found the bad patch and want to go back to the current HEAD run:

    git bisect reset
  5. If you find the patch that is the cause of the problem, you have almost won; report about it to Wine Bugzilla or subscribe to wine-devel and post it there. There is a chance that the author will jump in to suggest a fix; or there is always the possibility to look hard at the patch until it is coerced to reveal where is the bug :-)

Which code has been tested?

Deciding what code should be tested next can be a difficult decision. And in any given project, there is always code that isn't tested where bugs could be lurking. This section goes over how to identify these sections using a tool called gcov.

To use gcov on wine, do the following:

  1. In order to activate code coverage in the wine source code, when running make set EXTRACFLAGS and LDFLAGS like so: make EXTRACFLAGS=--coverage LDFLAGS=--coverage. Note that this can be done at any directory level. Since compile and run time are significantly increased by these flags, you may want to only use these flags inside a given dll directory.
  2. Run any application or test suite.
  3. Run gcov on the file which you would like to know more about code coverage.

The following is an example situation when using gcov to determine the coverage of a file could be helpful. We'll use the dlls/advapi32/registry.c file. At one time the code in this file was not fully tested (as it may still be). For example at the time of this writing, the function RegSetValueW had the following lines in it:

if (name && name[0])  /* need to create the subkey */
{
    if ((ret = RegCreateKeyW( hkey, name, &subkey )) != ERROR_SUCCESS) return ret;
}

Currently there are a few tests written to test this function. However, these tests don't check that everything is correct. Using gcov and directed tests, we can validate the correctness of this line of code. First, we see what has been tested already by running gcov on the file. To do this, do the following:

cd dlls/advapi32
make clean && make EXTRACFLAGS=--coverage LDFLAGS=--coverage
cd tests
make test
cd ..
gcov registry.c
less registry.c.gcov

The interesting code part looks like this in registry.c.gcov:

    4: 1273:    if (name && name[0])  /* need to create the subkey */
    -: 1274:    {
#####: 1275:        if ((ret = RegCreateKeyW( hkey, name, &subkey )) != ERROR_SUCCESS) return ret;
    -: 1276:    }

gcov output consists of three components: the number of times a line was run, the line number, and the actual text of the line. Note: If a line is optimized out by the compiler, it will appear as if it was never run. Line 1275 is never executed, most likely because name is never passed to the function. In order to validate this line, we need to do two things. First, we must write the test:

ret = RegSetValueW(hkey_main, name1W, REG_SZ, string1W, sizeof(string1W));
ok(ret == ERROR_SUCCESS, "RegSetValueW failed: %d, GLE=%d\n", ret, GetLastError());
test_hkey_main_Value_A(name1A, string1A, sizeof(string1A));
test_hkey_main_Value_W(name1W, string1W, sizeof(string1W));

Once we add in this test case, we now want to know if the line in question is run by this test and works as expected. You should be in the same directory as in the previous command example. The only difference is that we have to remove the *.gcda files in order to start the count over. (If we leave the files then the number of times the line is run is just added, e.g. line 545 below would be run 19 times.) We remove the *.gcov files because they are out of date and need to be recreated:

rm *.gcda *.gcov
cd tests
make
make test
cd ..
cd ..
gcov registry.c
less registry.c.gcov

The interesting code part looks like this in registry.c.gcov:

 5: 1273:    if (name && name[0])  /* need to create the subkey */
 -: 1274:    {
 1: 1275:        if ((ret = RegCreateKeyW( hkey, name, &subkey )) != ERROR_SUCCESS) return ret;
 -: 1276:    }

Based on gcov, we now know that line 1275 is executed once. And since all of our other tests have remain unchanged, we can assume that the one time it is executed is to satisfy the testcase we added where we check for it. Thus we have validated a line of code. While this is a cursory example, it demonstrates the potential usefulness of this tool.

For a further in depth description of gcov, the official gcc compiler suite page for gcov is http://gcc.gnu.org/onlinedocs/gcc/Gcov.html.

This page was last edited on 8 February 2016, at 17:47.