How to debug the VM?

Hi. Whether you are experimenting and hacking in the VM (where it is likely that some things will go wrong) or you are running an application in production, it is always useful to know how to debug the VM.  In this post, we will see how to compile the VM with all the debug information, how to run the VM from GDB (the GNU Project Debugger) and how to debug it using an IDE’s debugger.

Before going further, I would like to tell you something I did last weekend (apart from sleeping quite a lot).  Laurent Lauffont, creator of the PharoCasts, started a new sequence of screencasts called ” PharoCasts with Experts“. The idea is to do a more or less 1 hr “interview” about certain topic. Laurent calls by Skype to the “expert” and they can talk, ask questions, etc. Using TeamViewer, the “expert” shares his desktop to Laurent, who is in addition recording the screencast. The screencast is finally edited and uploaded to . Ok, I am not a VM expert at all, but last week we did a 1:30 screencast about compiling and debugging the VM 🙂   This screencast contemplates all what we have learned in the previous posts and what you will learn today.

So…let’s start the show.

Prepared image for you

In previous posts, I told you that Pharo guys recommend us to use the latest PharoCore for building the CogVMs. There are several reasons for this but the basic idea is that they want the VM can be build in latest versions, and if cannot, then fix it as soon as possible. But we have a much better testers than you, and it is called Hudson. So…I don’t want to force you to use the latest unstable PharoCore (which in fact, it is unstable sometimes). Even more, if you are starting in the VM world, I don’t want to add even more problems. So…from now onward, I have prepared and image so that you can use. At the time of this writing, there is no PharoDev 1.3 yet, but all the VM tools doesn’t load directly in a Pharo 1.2. There are 3 little changes that are needed. So…you can grab a PharoDev 1.2.1 and file in these changes, or directly load this image that I have prepared for you. This image has only those 3 changes. It does not contain any Cog or CMakeVMMaker package, that’s your homework 😉   The good news is that since we use a Dev image now we have all the development tools like refactorings, code completion, code highlighting, etc.

When do we need to debug the VM?

I think it is a good idea to start thinking when we need to debug the VM. In my little experience, I have found the following reasons:

  1. When there is a crash and you cannot find why it was.
  2. When you are developing something on the VM.
  3. For learning purposes.
  4. When you are just a hacky geek 🙂

I think most people will need the first one.

Debugging and optimizing a C program

If you are reading this post, you are probably a Smalltalker. And if you are a Smalltalker, I know how much you love the debugger. We all do. But there are guys that are not as lucky as we are 😉 Now, being serious, how do we debug a program written in C ? I mean, the VM, at the end, is a program compiled in C and executed like any other program. I am not a C expert so I will do a quick introduction.

When you normally compile a C program the binary you get does not contain the C source code used to generate such binaries. In addition, the names of the variables or the functions may be lost. Hence, it is really hard to be able to debug a program. This is why C compilers (like GCC) usually provide a way to compile a C program with “Debugging Symbols“, which add the necessary information for debugging to the binaries. Normally, as we will see today, these symbolic symbols are included directly in the binary. However, they can be sometimes separated in a different file. The GCC flag for the debugging symbols is -g  where -g3 is the maximum and -g0 the minimal (none). There are much more flags related to debugging but that is not the main topic of this post.

In addition to the debugging, a C compiler usually provides what they call “Compiler Optimizations”. This means that the compiler try to optimize the resulted binary in different ways: speed, memory, size, etc. When compiling for debugging it is common to disable optimizations, why? because, as its documentation says some variables you declared may not exist at all; flow of control may briefly move where you did not expect it; some statements may not be executed because they compute constant results or their values were already at hand; some statements may execute in different places because they were moved out of loops. The GCC flag for the optimization is -O  where -O3 is the maximum and -O0 no optimization (default).

But we are Smalltalkers, so much better if someone can take care about all these low-level details. In this case, CMakeVMMaker is that guy.

Building a Debug VM

As I told you in the previous post, there are several CMakeVMMAker configurations classes, and some of them are for compiling a debug VM. Examples: MTCocoaIOSCogJitDebugConfig, StackInterpreterDebugUnixConfig, CogDebugUnixConfig, etc. What is the difference between those configurations and the ones we have used so far ? The only difference is the compiler flags. These “Debug” VMs use special flags for debugging. Normally, these flags include things like -g3 (maximum debugging information), -O0 (no optimization), etc. For more details, check implementors of #compilerFlagsDebug. But…since there were a couple of fixes in the latest commit, I recommend you to load the version “CMakeVMMaker-MarianoMartinezPeck.94” of the CMakeVMMaker package.

So…how do we build the debug VM?  Exactly the same way as the regular (“deploy”)  VMs (I have explained that here and here), with only one difference. Guess which one? Yes, of course, use the debug configuration instead of the normal one. That means that in Mac for example, instead of doing “MTCocoaIOSCogJitConfig  generateWithSources”  we do “MTCocoaIOSCogJitDebugConfig  generateWithSources”. And that’s all. Once you finish all the build steps, you have a debug VM.

What are you waiting? Fire up your image and create a debug VM 😉  Now…how do you know if you really succeeded or not to create a debug VM?  Easy: it will be much slower and the VM binary will be bigger (because of the symbolic symbols addition). In a regular CogMTVM, if you evaluate “1 tinyBenchmarks” you may get something arround  ‘713091922 bytecodes/sec; 103597027 sends/sec’ whereas with a debug VM something arround ‘223580786 bytecodes/sec; 104885548 sends/sec’. you can notice, a debug VM is at least 3 times slower.

Debugging the VM with GDB

Disclaimer: I am not a GDB expert, since most of the times I debug from the IDE (XCode). I will give you a quick introduction. So…we have compiled our with all the debug flags which means that the VM binaries now have extra information so that we can debug. GDB is a command line debugger from GNU Project and it is the “standard” when debugging C programs. In addition, it works in most OS. Notice that for Windows there are not VMMakeVMMaker debug configurations. I think this is just because nobody tried it. As far as I know gdb works with MSYS, so it may be a matter of just implementing #compilerFlagsDebug for Windows confs and try to make it work. If you give it a try and it works, please let me know! So far, I did a little test with some compiler flags like -g3 and -O0 and it compiles and runs. But (as we will see later) when I do CTRL+C instead of get the gdb prompt, I kill GDB heheheh. Googling, seems to be a known problem.

So….open a console/terminal. The following are the minimal steps to run the VM under GDB in Mac OS:

cd blessed/results/
#If you compiled a StackVM or CogVM instead of a CogMTVM, then the .app and the executable will have another name
gdb CogMTVM

In Linux/Windows since you don’t have the .app directory of the Mac OS, it should be something like this;

cd blessed/results
#If you compiled a StackVM or CogVM instead of a CogMTVM, then the executable will those names instead
gdb CogMTVM

With the previous step, you should have arrived to a gdb console that looks like this:


So now we need to start the VM. In Mac, the VM automatically raises the File Selection popup that lets you choose which image to run. So the following line is enough:

(gdb) run

Choose the image and continue. In the Linux  you have to specify to the VM the .image to run by parameter. So, you have to do something like this:

(gdb) run  /home/mariano/Pharo/images/Pharo1.3.image

The idea is that you send by parameter the .image you want to run. So…at this point you have already launched your image, its time to play a bit. Open a Transcript. Open a workspace and evaluate:

9999 timesRepeat: [Transcript show: 'The universal answer is 42.']

Now…come back to the console, and do a CTRL+C. This will get a SIGINT Interruption and in the GDB console you should get something like:

Program received signal SIGINT, Interrupt.
0x9964a046 in __semwait_signal ()

What happened?  I am not an expert, but with such interruption we can “pause” the VM execution and the gdb takes the control. If you see your image at this moment, you will se that you cannot do anything in it. It is like “frozen”. If you now do:

(gdb) continue

You can also type “c” instead of “continue”. The image should continue running, and you Transcript should continue printing the universal answer 😉   So…what is cool about being able to interrupt the VM?  We can, for example:

  • Inspect the value of some variables and stack. Type “help data” and “help stack” in the gdb prompt.
  • Put breakpoints in the code and then after go step by step.
  • Invoke functions from our VM (this is really cool!!)

Let’s see a couple of examples….hit CTRL+C again to get the control of the gdb prompt and do:

(gdb) bt

Which should look similar this (why similar? because it depends where you stop it):

(gdb) bt
#0  0x0002dbd0 in longAtPointerput (ptr=0x22efa6e4 "\017??\024?\001", val=349999375) at sqMemoryAccess.h:82
#1  0x0002daf2 in longAtput (oop=586131172, val=349999375) at sqMemoryAccess.h:99
#2  0x0006a2ae in sweepPhase () at /Users/mariano/Pharo/vm/git/cogVM2/blessed/src/vm/gcc3x-cointerpmt.c:44356
#3  0x0003d8b4 in incrementalGC () at /Users/mariano/Pharo/vm/git/cogVM2/blessed/src/vm/gcc3x-cointerpmt.c:17846
#4  0x00069f61 in sufficientSpaceAfterGC (minFree=0) at /Users/mariano/Pharo/vm/git/cogVM2/blessed/src/vm/gcc3x-cointerpmt.c:44214
#5  0x00032e56 in checkForEventsMayContextSwitch (mayContextSwitch=1) at /Users/mariano/Pharo/vm/git/cogVM2/blessed/src/vm/gcc3x-cointerpmt.c:11398
#6  0x0003ccad in handleStackOverflowOrEventAllowContextSwitch (mayContextSwitch=1) at /Users/mariano/Pharo/vm/git/cogVM2/blessed/src/vm/gcc3x-cointerpmt.c:17346
#7  0x000326c4 in ceStackOverflow (contextSwitchIfNotNil=525336580) at /Users/mariano/Pharo/vm/git/cogVM2/blessed/src/vm/gcc3x-cointerpmt.c:11136
#8  0x1f40032e in ?? ()
#9  0x0006ab4c in threadSchedulingLoop (vmThread=0x828a00) at /Users/mariano/Pharo/vm/git/cogVM2/blessed/src/vm/gcc3x-cointerpmt.c:44714
#10 0x0003dd96 in initialEnterSmalltalkExecutive () at /Users/mariano/Pharo/vm/git/cogVM2/blessed/src/vm/gcc3x-cointerpmt.c:17970
#11 0x0003ebdc in initStackPagesAndInterpret () at /Users/mariano/Pharo/vm/git/cogVM2/blessed/src/vm/gcc3x-cointerpmt.c:18435
#12 0x00022211 in interpret () at /Users/mariano/Pharo/vm/git/cogVM2/blessed/src/vm/gcc3x-cointerpmt.c:2037
#13 0x00072142 in -[sqSqueakMainApplication runSqueak] (self=0x211090, _cmd=0x121c80) at /Users/mariano/Pharo/vm/git/cogVM2/blessed/platforms/iOS/vm/Common/Classes/sqSqueakMainApplication.m:172
#14 0x96339cbc in __NSFirePerformWithOrder ()
#15 0x91b9ee02 in __CFRunLoopDoObservers ()
#16 0x91b5ad8d in __CFRunLoopRun ()
#17 0x91b5a464 in CFRunLoopRunSpecific ()
#18 0x91b5a291 in CFRunLoopRunInMode ()
#19 0x920bde04 in RunCurrentEventLoopInMode ()
#20 0x920bdaf5 in ReceiveNextEventCommon ()
#21 0x920bda3e in BlockUntilNextEventMatchingListInMode ()
#22 0x9307378d in _DPSNextEvent ()
#23 0x93072fce in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()
#24 0x93035247 in -[NSApplication run] ()
#25 0x9302d2d9 in NSApplicationMain ()
#26 0x0006fca5 in main (argc=2, argv=0xbffff864, envp=0xbffff870) at /Users/mariano/Pharo/vm/git/cogVM2/blessed/platforms/iOS/vm/Common/main.m:52

Guess what it was doing ? Sure…an incremental GC 😉

(gdb) call printAllStacks()

Which should look similar to the following (notice that it is not the same place as the previous example and this is because the stacktrace in the middle of the incremental Garbage Collector was too long to put in the post, so I did a “bt” in another moment):

(gdb) call printAllStacks()
Process 0x2197b5f8 priority 10
0xbff650b0 I ProcessorScheduler class>idleProcess 526237088: a(n) ProcessorScheduler class
0xbff650d0 I [] in ProcessorScheduler class>startUp 526237088: a(n) ProcessorScheduler class
0xbff650f0 I [] in BlockClosure>newProcess 563590428: a(n) BlockClosure

Process 0x219edf1c priority 50
0xbff5f0b0 I WeakArray class>finalizationProcess 526237628: a(n) WeakArray class
0xbff5f0d0 I [] in WeakArray class>restartFinalizationProcess 526237628: a(n) WeakArray class
0xbff5f0f0 I [] in BlockClosure>newProcess 564059712: a(n) BlockClosure

Process 0x2067e180 priority 80
0xbff600d0 M Delay class>handleTimerEvent 526243268: a(n) Delay class
0xbff600f0 I Delay class>runTimerEventLoop 526243268: a(n) Delay class
543678476 s [] in Delay class>startTimerEventLoop
543678752 s [] in BlockClosure>newProcess

Process 0x2197b430 priority 60
0xbff610b0 I SmalltalkImage>lowSpaceWatcher 527603592: a(n) SmalltalkImage
0xbff610d0 I [] in SmalltalkImage>installLowSpaceWatcher 527603592: a(n) SmalltalkImage
0xbff610f0 I [] in BlockClosure>newProcess 563589972: a(n) BlockClosure

Process 0x219eba78 priority 60
0xbff62030 M [] in Delay>wait 564050624: a(n) Delay
0xbff62050 M BlockClosure>ifCurtailed: 565737144: a(n) BlockClosure
0xbff6206c M Delay>wait 564050624: a(n) Delay
0xbff62084 M InputEventPollingFetcher>waitForInput 526769836: a(n) InputEventPollingFetcher
0xbff620b0 I InputEventPollingFetcher(InputEventFetcher)>eventLoop 526769836: a(n) InputEventPollingFetcher
0xbff620d0 I [] in InputEventPollingFetcher(InputEventFetcher)>installEventLoop 526769836: a(n) InputEventPollingFetcher
0xbff620f0 I [] in BlockClosure>newProcess 564050332: a(n) BlockClosure

Process 0x212b6170 priority 40
0xbff5e03c M [] in Delay>wait 565740688: a(n) Delay
0xbff5e05c M BlockClosure>ifCurtailed: 565741260: a(n) BlockClosure
0xbff5e078 M Delay>wait 565740688: a(n) Delay
0xbff5e098 M WorldState>interCyclePause: 529148776: a(n) WorldState
0xbff5e0b4 M WorldState>doOneCycleFor: 529148776: a(n) WorldState
0xbff5e0d0 M PasteUpMorph>doOneCycle 527791904: a(n) PasteUpMorph
0xbff5e0f0 I [] in Project class>spawnNewProcess 528638248: a(n) Project class
556491024 s [] in BlockClosure>newProcess

Notice that in this case we are calling from GDB a exported function of the VM. Where does it come from?  For the moment, take a look to #printAllStacks. Now, it is time to kill the executable and finally to quit gdb:

(gdb) kill
(gdb) Quit

Notice that after doing the “kill” you can call “run” again an start everything again.  Hopefully, you got an idea of how to debug the VM. However, using gdb from command line is not the only option.

Debugging the VM with XCode

In the previous post we saw that CMake provides “generators”, i.e, a way to generate specific makefiles for different IDEs. Hence, we can generate makefiles for our IDE and debug the VM from there. Remember that executing “cmake help” in a console shows the help and at the end, there is a list of the available generators. In this particular example, I will use XCode in Mac OS. To do so, we need to follow the same steps than compiling a debug VM (that is, the same as building a release VM but using a debug CMakeVMMaker conf) but using the XCode generator in particular. That means that instead of doing “cmake .” we do “cmake -GXcode”. Notice that if we previously build another VM you will need to remove the CMake cache: /blessed/build/CMakeCache.txt. So..

cd blessed/build
cmake -G Xcode

Notice that it is not needed to do a “make” since we will do that from inside XCode. “cmake -G Xcode” generates a /blessed/build/CogMTVM.xcodeproj (the name will change if you generate a CogVM or a StackVM) that we directly open with XCode. Now…normally we should be able to select the target Debug (on the top left) and then from the menu “Build -> Build and Debug – Breakpoints On”. That should compile the VM and automatically run it with GDB. However, I found two problems:

  • External plugins are not correctly placed. I tried to fix it but I couldn’t. They are placed in blessed/results/  but they should be together in blessed/results/Debug/ The problem is that when XCode compiles, it adds a “Debug” in the directory when compiling with the “debug target” and the same for “release”. I have spent two days completely trying to solve it from the CMakeVMMaker and I couldn’t. If you know how to do it, please let me know. Anyway, the workaround is to copy all those .dylib from the first directory to the second one.
  • No matter that you change the settings in the project (like compiler flags, gcc version, etc) it will always the values from the CMake generated makefiles. This is a pity because I would like to change settings from XCode…

Now you are able to “Build -> Build and Debug – Breakpoints On”. If everything goes well, you should be able to open an image and the VM will be running with gdb. You will notice there is a little gdb button that opens a gdb console where you can do exactly the same as  if it were by command line. In addition, there is another button for the debugger. I attach a screenshot.

So…there you are, you are running the VM in debug mode from gdb and inside XCode. Interesting things from the gdb console is that you have buttons for “Pause” (what we did with gdb but from command line with the CTRL+C) and for “Continue” (what we type from gdb command line). And you have even a “Restart”.

Now….the nice thing about being able to debug the VM with XCode is the ability to easily put breakpoints in the code. What where? in which file? Without giving much details (because that’s the topic of another post), we will say that the “VM Core” (basically Interpreter classes + ObjectMemory classes) is translated into a big .c file called gcc3x-cointerpmt.c (in case of cogMT). A regular CogVM will be called “gcc3x-cointerp.c” the same as for StackVM. You can check this files by yourself. You should know where they are. Remember?  yes, they are (if you did the default configuration) in /blessed/src/vm  (for Cocoa configurations it may be in /blessed/stacksrc/vm). In such folder you will see that there is another file cointerpmt.c (or cointerp.c). Which is the difference between the one that has the “gcc3x” and the one that has not? for me moment let’s just say the one with the “gcc3x” has some optimizations (Gnuifier) that were automatically done in the C code and with the intention of compiling such sources with a gcc3x compiler.

We already know which file to debug “most of the times”. Maybe you need to debug another C file like plugins or the machine code generator, but most of the times, you will debug the “VM core” which is the file gcc3x-cointerpmt.c or its variant. Now what you have to do is to open it and put a breakppoint. On the top part of XCode you have a list of the files included in the project. Search for that one and select it. Be careful that XCode is really slow with this big file. Let’s put a breakpoint in the method lookup when the #doesNotUnderstand is thrown. This is done in the function “static sqInt lookupMethodInClass(sqInt class)” and here is a screenshot:

Once you set breakpoints you can “Build and Debug”. Now, instead of testing by executing something in a workspace, we will create a simple method anywhere (why not to use the workspace is not at my knowledge right now, sorry). So…I create the method foo in TestCase doing like this:

TestCase >> foo
self methodNonExistent.

And then I executed (this can be done in a workspace):

TestCase new foo

Once this is done, the VM should have been paused and you should have available the gdb prompt. You can now open the debugger or GDB console and do whatever you want. With the Debugger you can do “Step Over”, “Step Into”, “Step Out”, etc…And with the GDB console you can do exactly the same like if you were executing gdb from a terminal. Here is the screenshot:

There are much things I would like to talk about regarding debugging a VM, but the post is already too long and I think this is enough to start. I recommend you to watch the screencast.  Further in this blog sequence, we will do a second part. Now, it is time to understand some internal parts of the Squeak/Pharo VM.

3 thoughts on “How to debug the VM?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s