Building the VM from scratch using Git and CMakeVMMaker

So…this is the post all hackers were waiting for! Take a beer, open your Pharo image and prepare your terminal 🙂   In this post, we will see how to build a VM from scratch. I will use Mac OS along this post, but I will also explain how to build in Linux and Windows (yes, I had to do that for this post!). I will follow the instructions together with you so that they are correct and working heehhehe. In addition, I am new with both Git and CMake…so if I say something wrong, please correct me.

The VM is a huge beast and no matter all the effort people do to ease its build, it can fail. But don’t get disappointed, if there are problems, please ask!

Remember (I told you this in the previous post) that there are different ways of building the from scratch. In this case we will use GIT (instead of the SVN repository) and CMakeVMMaker (instead of VMMakerTool) and we will build a Cog VM. These tools are part of what I called “new infrastructure” in the previous post.

Installing necessary tools on each OS

For all the OS we need a couple of things: Git client, CMake, and gcc/make. I will tell you what I think is needed in each OS. I know it is painful, but at least, you have to do it only once…Don’t be lazy. If this get’s longer, you have the excuse for the second beer 😉

Mac OS

  1. “Just” (4gb) installing the whole XCode is enough for compiling. XCode package comes with everything: gcc, make, cmake, etc. You can download it from Apple’s website. For Lion or XCode 4.2 users, read the last section of the blog (“Problems I have found so far”) .
  2. The Git client for Mac OS. Try to put Git in PATH.
  3. To check if everything is installed, open a terminal and try to execute the different commands: git, make and cmake.

Linux

  1. You need to install the development tools such as gcc, make, etc. In Ubuntu, you need to install the “buildessential” package (sudo apt-get install build-essential).
  2. Install cmake -> if it is not already included something like “sudo apt-get install cmake” should work in all Ubuntu and forks.
  3. Install Git client (in my Ubuntu this was with “sudo apt-get install git-core”). You may want to put Git in PATH. Usually, when installing with a package system it automatically adds binaries to the PATH.
  4. To check if everything is installed, open a terminal and try to execute the different commands: git, make and cmake.

Windows

  1. Download and install MinGW and MSYS, with C/C++ compiler support:  http://www.mingw.org/wiki/msys. To install mingw+msys using single, painless step, one should download latest installer from here: http://sourceforge.net/projects/mingw/files/Automated%20MinGW%20Installer/mingw-get-inst/
    Today, the mingw-get-inst-20110530 is latest one.
  2. Download and install Git: http://code.google.com/p/msysgit/ During the installation, I choose the option “run git from from the windows command prompt”. Optional: add git to the PATH variable so that you can see git from msys. To do this, add path to git for msys:
    Control panel -> System -> System Properties / Advanced  [ Environment Variables ].   There should be already: ‘C:\Program Files\Git\cmd’. Add ‘C:\Program Files\Git\bin’  Notice that the path may not be exactly ‘C:\Program Files\Git\’ but similar…
  3. Install CMake: http://www.cmake.org/cmake/resources/software.html (during installation, in install options , make sure that you choose to add CMake to PATH).
  4. To check if everything is installed, open MSYS program (which should look like a UNIX terminal) and try to execute the different commands: git, make and cmake.

From now, when we refer to “terminal” we refer to whatever it is: iTerm and friends in Mac OS, Unix/Linux terminals, and MSYS in Windows.

Downloading platform code from GIT

If and only if you plan to modify and publish your cahnges, you will need to create an account in https://gitorious.org/ in case you don’t have one already. If you just want to download a project, then you don’t need an account. Gitorious is not more than a nice, free and public GIT hosting. Creating an account there is a little tricky because you need to SSH key. But try it and if you have problems check in the web because there is a lot of documentation and blog posts about that. Once you have your account working, continue with this…

The goal of this post is not to talk about GIT so we will do it fast 🙂 If you are reading this post, you may probably be a Smalltalk hacker. A real Smalltalk hacker doesn’t use a UI program, but instead command line jeheheh. Seriously, for this post it is easier to just tell you the git commands instead of using a Git front end. So..all git commands should be run from a command line. What are you waiting for? Open a terminal go to your prefer directory and create your workspace for today:

mkdir cogVM
cd cogVM

The Cog VM repository is https://gitorious.org/cogvm. We need to download the “platform code” from there. To do this, we have two options. The first one is to just clone the CogVM project to your local directory. This option is the most common and it is what you will do if you just want to load CogVM (and not modify it or commit changes). It is like doing a “svn co” for the SVN guys.

git clone git://gitorious.org/cogvm/blessed.git

Normally, you can also pass the argument “–depth 1” and do “git clone –depth 1 git://gitorious.org/cogvm/blessed.git”. This is just for avoiding to download all the history and it just downloads the HEAD (at least that’s what I think it does). In this post we are not going to use “–depth 1” and I will explain later why not. The second option is to clone Cog VM to your own Git repository. This has to be done from the Git website (maybe it can be done from command line but I don’t know it): login in Gitorius, search the project CogVM in the search input, select the CogVM project and you will find a “clone repository” button. Click on it and wait. Once it finishes, you will have your own fork of CogVM, and the Git repository is something like this: git://gitorious.org/~marianopeck/cogvm/marianopecks-blessed.git. Now, the last step is to clone from your own fork to your local directory (as we did in the first option). This should be something like:

git clone git://gitorious.org/~marianopeck/cogvm/marianopecks-blessed.git

For this post, I recommend to take the first option if you may be a beginner. I told you that I wanted my posts to be reproducible. With the previous commands, you will clone the latest version in the repository. Since I don’t know when you are going to do it (if there is someone), I would like that you load the specific version I know it works. What I am suggesting is doing a kind of “svn co http://xxxx -r 2202”. I checked how to do this in git, and it seems not to provide a clone of a specific version. Instead, you just clone (from the latest one) and then you checkout or revert to a previous one. Execute:

cd blessed
git checkout f3fe94c828f66cd0e7c37cfa3434e384ff65915e

Notice that you can do this because we have downloaded the full history of the repository. If we would have added the “–depth 1” parameter during the first clone, then we should be having an error like “fatal: reference is not a tree: f3fe94c828f66cd0e7c37cfa3434e384ff65915e”.

f3fe94c828f66cd0e7c37cfa3434e384ff65915e is the commit hash of the version I want. You can do “git log” to see the latest commits or “git rev-parse HEAD” to see the last one.

Ok, if you could successfully load everything from Gitorious you should have something like this:

mariano @ Aragorn : ~/Pharo/vm/git/cogVM/blessed
$ls -la
total 72
drwxr-xr-x  16 mariano  staff    544 Apr  7 00:14 .
drwxr-xr-x   3 mariano  staff    102 Apr  7 00:07 ..
drwxr-xr-x  13 mariano  staff    442 Apr  7 00:14 .git
-rw-r--r--   1 mariano  staff   6651 Apr  7 00:13 .gitignore
-rw-r--r--   1 mariano  staff     41 Apr  7 00:13 CHANGES
-rw-r--r--   1 mariano  staff   1112 Apr  7 00:13 LICENSE
-rw-r--r--   1 mariano  staff  13597 Apr  7 00:13 README
-rw-r--r--   1 mariano  staff     17 Apr  7 00:13 VERSION
drwxr-xr-x   8 mariano  staff    272 Apr  7 00:13 codegen-scripts
drwxr-xr-x  13 mariano  staff    442 Apr  7 00:13 cygwinbuild
drwxr-xr-x   3 mariano  staff    102 Apr  7 00:13 image
drwxr-xr-x  24 mariano  staff    816 Apr  7 00:13 macbuild
drwxr-xr-x   7 mariano  staff    238 Apr  7 00:14 platforms
drwxr-xr-x   4 mariano  staff    136 Apr  7 00:14 processors
drwxr-xr-x   8 mariano  staff    272 Apr  7 00:14 scripts
drwxr-xr-x   6 mariano  staff    204 Apr  7 00:14 unixbuild

I have highlighted two lines that represent two important directories: “/platforms” and “/image”. For the moment, lets explain what “/platforms” is…come on, you should guess it! Yes, there, in that folder, it is the famous “platform code”. You can enter to such directory and see the C code by yourself.

Downloading Cog and dependencies

So far we have loaded from Git the “platform code”. We are missing the other part, VMMaker. In the previous post I told you it may not be necessary to always download VMMAker and generate sources, because such sources may also be commited in the repository. Having the auto-generated source code in the repository is a trade-off, it has advantages and disadvantages. In this “new infrastrucutre” under Git, it was decided to remove it. So if you want to compile the VM you have to load VMMaker and translate it to C. You can read the explanations here and here.

So…we need to download VMMaker (Cog branch) and translate it to C. But of course, we first need a Smalltalk image. I told you in the previous post that I want all my posts to be reproducible. So, take this PharoCore 1.3 image. Notice that zip comes not only with the .image and .sources files, but also the .sources. If you don’t know what .sources file is, you should read Pharo By Example chapter 1, section 1.1 “Getting Started” 🙂  You need the .sources because it is necessary in order to generate the sources of the VMMaker. The image of such zip can be opened with both, Interpreter VM and Cog VM. However, if you open it with Cog, and you save it, then you won’t be able to run it with the Interpreter VM (this was fixed in the latest code of Interpreter VM but there isn’t yet an official VM release for all OS that contains this fix). Thus, I recommend you to run the image with a CogVM. If you don’t have one already, you can download this one.

Now, let’s load VMMaker branch for Cog and all its dependencies. You have already learned that Cog has dependencies in other packages and to solve that, among other problems, we use a Metacello configuration for it. The following code may take time because since we are evaluating this in a PharoCore image where Metacello is not present, Metacello needs to be installed first. In addition, VMMaker is a big package…Fortunatly we are running with a CogVM 🙂    So, take the image and evaluate:

Deprecation raiseWarning: false.
Gofer new
squeaksource: 'MetacelloRepository';
package: 'ConfigurationOfCog';
load.
((Smalltalk at: #ConfigurationOfCog) project version: '1.5') load.

Gofer new
squeaksource: 'VMMaker';
package: 'CMakeVMMaker';
version: 'CMakeVMMaker-MarianoMartinezPeck.83';
load.

IMPORTANT: How do you know you have to load version ‘1.5’ ? Yes, because I am telling you hahha. But how do I know that 1.5 is the one that works with the current version of Git?  This is answered in the next post under the title “Synchronization between platform code (Git) and VMMaker”.

One of the cool things I like from Metacello configurations is the possibility to query them. For example, do you want to all the packages that are installed while doing the previous code? Just inspect or print:


(ConfigurationOfCog project version: '1.5') packages.
ConfigurationOfCog project versions.

Now, if you are curious about how defining versions and baselines is achieved in Metacello, take a look to the methods ConfigurationOfCog >>#version15:  and ConfigurationOfCog >>#baseline13:.

Generating VM sources and CMake files

What have we done so far? We have just download all the VMMaker branch for Cog with all its required dependencies. Now, its turn to translate from SLANG to C and to generate the CMake outputs so that we can compile after. This C code generated from VMMaker is known in the Squeak/Pharo VM world as “the sources”. But don’t confuse, the platform code is also source code…but anyway, when usually someone says the “sources” it means the autogenerated C code from VMMaker. This is why this C code is placed (by default) in /src.

To do both things (generate VM sources and CMake outputs), we use one of the available CMakeVMMaker configurations. Metacello configurations are represented by classes (instead of XML like maven or similar package management systems). How do you think CMakeVMMaker configurations are represented?? Of course!! with classes too. So, we need to find the accurate class for us. I won’t go further now because that’s topic of another post, but for the moment, lets use these classes CogUnixConfig, CogMsWindowsConfig and CocoaIOSCogJitConfig depending in which OS you are.

A little remark for Mac users: you can notice that there are two categories with CMake Mac configurations, ‘CMakeVMMaker-MacOS’ and ‘CMakeVMMaker-IOS’. The first one, is for Mac OS versions that use Carbon library. ‘CMakeVMMaker-IOS’ contains CMake configurations that use Cocoa instead, which is the new library for Mac OS. Carbon is legazy, and it may be removed soon in next MacOS versions. So, the new ones, and the ones you should use, are those configurations under ‘CMakeVMMaker-IOS’.

These configurations are flexible enough to set specific directories for sources, platforms, results, etc. In addition, if you follow certain conventions (defaults), the build is more automatic. For the purpose of this post, we will follow the conventions and use the expected default directories. The only real convention we should follow is that the .image should be in a subdirectory of the directory where you downloaded the GIT code. In my case (see the bash example at the beginning of the post), it is ~/Pharo/vm/git/cogVM/blessed.  So, I moved the .image to ~/Pharo/vm/git/cogVM/blessed/image. You can create your own directory  ~/Pharo/vm/git/cogVM/generator  and place it there. The only requirement is that the ‘platforms’ directory is found in ‘../platforms’.  So…did you move your image?   Perfect, let’s continue.

No…wait. Why you need ‘platforms’ directory if we are not really compiling right now?  Ask yourself…do you think VMMaker translation to C needs the platform code?  Nooo!  So…we only need the platform directory for the second part, for  CMake. Now yes, we continue…take the Pharo image (which should have been moved) and evaluate:

"CocoaIOSCogJitConfig is an example. If you are not in Mac, replace it with CogMsWindowsConfig or CogUnixConfig"
CocoaIOSCogJitConfig  new
"Using VMMaker we translate Cog to C"
generateSources;
"We generate all the CMake necessary directories and files"
generate.

Ok…As my comments say, #generateSources uses VMMaker class to translate from SLANG to C. Instead of using a UI (VMMakerTool) we directly translate by code…but..do you remember that for compiling the VM we needed to say which plugins to compile and whether to compile them like internal or external? Ok…At that moment I told you that most of the developers shouldn’t be aware of that. In this case, CMakeVMMaker does the job for us. We will come later to this topic, but if you want to know which plugins are compiled, check implementors of #internalPlugins and #externalPlugins. Once again, CMakeVMMaker has defaults for these things, but you can customize and change them.

Where is the C generated code ? By default (yes, it can be changed) is placed in ‘../src’. In my example, it should be in ~/Pharo/vm/git/cogVM/blessed/src and should look like this:

mariano @ Aragorn : ~/Pharo/vm/git/cogVM/blessed/src
ls -la
total 16
drwxr-xr-x   6 mariano  staff   204 Apr  9 14:55 .
drwxr-xr-x  18 mariano  staff   612 Apr  9 14:55 ..
-rw-r--r--@  1 mariano  staff   776 Apr  9 14:55 examplePlugins.ext
-rw-r--r--@  1 mariano  staff    83 Apr  9 14:55 examplePlugins.int
drwxr-xr-x  43 mariano  staff  1462 Apr  9 14:55 plugins
drwxr-xr-x  11 mariano  staff   374 Apr  9 14:55 vm

In a future post, we will go deeper in how is the C translated cog…but if you want to take a look, go ahead!! Inspect the file /src/vm/cointerp.c  for example 🙂   So…do you already love SLANG? hehehe

With the method #generate we create all the directories and files needed by CMake so that we can after use CMake to generate different makefiles. You will notice that this method creates a directory /build. In my case, it is ~/Pharo/vm/git/cogVM/blessed/build. If you check inside that directory, there are a couple of important files generated for CMake (so that we can use it after), such as CMakeLists.txt, directories.cmake, etc. If you are curious, take a look to them.

If you are interested, I strongly recommend you to take a look to both methods: #generateSources and #generate. Now that I have explained the two big steps, I can tell you that there is a shortcut:

CocoaIOSCogJitConfig generateWithSources

Using CMake and compiling the VM

We are almost done…we already have all the necessary C code, and all the CMake files and directories. The next step is to just use CMake. I am newbie in both Git and CMake. But as far as I could see, CMake is a wonderful tool for being able to generate different makefiles from the same “definition”. So, we have already tell CMake which was our code, the directories, compiler flags, ec, etc. Then CMake can take such information and generate different makefiles: UNIX makefiles, MSYS (for Windows), XCode, Visual Studio, etc…In this post we will see just how to use regular Unix makefiles.

Now…come back to your terminal. We need to first go the /build directory and then execute CMake. In MacOS and Linux, evaluate:

cd build
cmake .

Now, in Windows we are compiling in MSYS, so we need to create special makefiles for it. The way to do this with CMake is using the parameter -G”Generator Name” where “Generator Name” is “MSYS Makefiles” in this case. So, in MSYS (in Windows) we evaluate:

cd build
 cmake . -G"MSYS Makefiles"

Once that is done, we have created all the necessary makefiles. Now, the last pending thing is just to “make”. It is the same whether you’re on windows or not. The makefiles have been generated, son only a make is needed:

make

Hopefully, you didn’t receive any compilation error and you can find your VM binary in /results (in Windows it is under /build/results) . Again, in my case it is ~/Pharo/vm/git/cogVM/blessed/results.

Problems I have found so far

Linux

It seems that the default CogUnixConfig needs OpenGL dev files (headers) and libs.This is because some plugins like Croquet or Balloon3D require such lib. And those plugins are being included by default in CogUnixConfig. So in my case I’ve got the error “The file was not found sqUnixOpenGL.h” which I fixed by installing the dev package:

sudo apt-get install mesa-common-dev

Then, I have a problem at linking time, “/usr/bin/ld: cannot find -lGL”, which I solved by doing:

delete /usr/lib/libGL.so
cd /usr/lib/
sudo ln -s libGL.so.1.2 libGL.so

Notice that other people have experienced the same problem but with the libSM.so (lSM) and libICE.so (lICE). Such problem could be resolved the same way. You may  want to use the command “locate” first to see where the library is and then do the “ln”.

Another solution (but I couldn’t test it by myself) to avoid using “ln”, could be to simply install the package libgl1-mesa-dev (sudo apt-get install libgl1-mesa-dev).

After having done all that, I then realised there is a special  CogUnixNoGLConfig  which seems to avoid linking to OpenGL. The VM will work until you use something like Croquet or the Ballon3D. For a more detailed explanation, read this thread.

If you have the error “alsa/asoundlib.h: No such file or directory” then you should install libasound2-dev, for example, by doing “sudo apt-get install libasound2-dev”. Be aware that depending on your Linux distro and what packages you have installed on in, you may require to install a couple of packages or non. If you have a problem with a C header file not found (a .h file) you will probably need to install the “dev” package of the project. And if what it is not found is a library (a .so for example), then it is likely you will need to install the package that contains such libs. How do you know which package contains a specific header file or lib ? I have no idea. I always go to Google and I find the answer.

Windows

As you can read in this thread I could generate the VM in Windows by myself but I had a problem at compile time: “cc1.exe: error: invalid option `no-fused-madd'”. To solve that problem I edited the method CPlatformConfig >> configureFloatMathPlugin:.  I’ve changed the line “maker addDefinitions: ‘-O0 -mno-fused-madd’.”  to “maker addDefinitions: ‘-O0’.”. The effects of this change, may be that Croquet doesn’t work properly. For further details, read another thread.

Ok, this is the end of the post. I still remember (ok, only two years ago) the first day I could compile the VM. I felt so hacky and happy at the same time. I remember I chatted with my SqueakDBX companions about that ehehhehe. So…if I could help you to successfully build your own first VM, you own me a beer (yes, I will be at ESUG hahaha). If you spent a couple of hours and you couldn’t…..mmmm… ask the mailing list 🙂    Seriously, as I told you, compiling the VM from scratch is complicated, even with all the effort made to ease that. If you had problems, please ask in the mailing lists. They will probably be able to help you, and your questions will make this process easier in a near future.

In the next post we will see some advanced topics of compiling the VM, and after that we will start to take a look to the VM inside. Once again, thanks Esteban, Igor, and everbody who answered my questions in the mailing list.

Mac OSX

As far as I understand, if you update to the latest XCode (4.2) from an older version (in Snow Leopard) it still  includes the GCC but it sets LLVM as the default compiler. In a fresh installation of Lion/XCode, not only LLVM is the default compiler but also GCC is not installed. To check which compiler area you using, execute in a console “gcc –version”. If it says something like “i686-apple-darwin11-gcc-4.2.1” then it is correct, you are using GCC. If it says something like “686-apple-darwin11-llvm-gcc-4.2” then it means you are using LLVM by default.

If GCC is not installed, you can install it via Mac Ports by doing “sudo port install apple-gcc42”.  You can follow http://stackoverflow.com/a/8593831/424245 to get it to appear in  Xcode, the last two steps will probably look like:
a) sudo ln -s /opt/local/bin/gcc-apple-4.2 /Developer/usr/bin/gcc-4.2
b) sudo ln -s /opt/local/bin/g++-apple-4.2 /Developer/usr/bin/g++-4.2

The CogVM needs GCC and cannot be compiled right now with LLVM so you have to use GCC. There are a couple of possibilities:
1) Change the default compiler for the whole system. To do this you have to edit the symbolic link of gcc to point to the real GCC and not LLVM. I do not recommend that much this option since you may be affecting the whole system.
2) When you do the cmake, reacher than simply do “cmake .” do: “cmake -D CMAKE_C_COMPILER=gcc-4.2 -D CMAKE_CXX_COMPILER=g++-4.2 . ” If that doesn’t work, try “cmake -D CMAKE_C_COMPILER=gcc-apple-4.2 -D CMAKE_CXX_COMPILER=g++-apple-4.2 .”. I would recommend this option for those who are building for the first time.
3) Instead of doing 2) by hand, you can use a patch in CMakeVMMaker that does that for you. In such a case you should use the config CogMTCocoaIOSGCC42Config. Notice, however, that such patch was added in the latests versions of CMakeVMMaker. In this post, I put specific versions of each part of the VM building system. Therefore, if you want to use the latest version of CMakeVMMaker you should also use the latest code from git and from ConfigurationOfCog.

For more details see this thread and this one in the VM mailing list:

Advertisements

47 responses to “Building the VM from scratch using Git and CMakeVMMaker

  • Jason Olson

    Excellent posts Mariano! Perfect timing for me as I had just come back to trying to disect the VM when I came across this (I never endured the pain in the past). Your instructions were perfect. I’m now successfully compiling the VM just fine.

    I’m not going to be at ESUG, but if I ever meet you in person, I guarantee you that I will buy you a beer 🙂

  • Building the VM – Second Part « Mariano Martinez Peck

    […] post is mostly about a couple of things that I wanted to mention in the previous post, but I couldn’t because it was already too long. If you read such post, you may think that […]

  • How to debug the VM? « Mariano Martinez Peck

    […] debug VM?  Exactly the same way as the regular (“deploy”)  VMs (I have explained that here and here), with only one difference. Guess which one? Yes, of course, use the debug configuration […]

  • The first part of the journey is over « Mariano Martinez Peck

    […] that you knew which repositories to use and where everything was version, I explained you step by step how to build a CogVM using Metacello configurations, the Git repository and CMakeVMMaker. The post covered how to build […]

  • Class formats and CompiledMethod uniqueness « Mariano Martinez Peck

    […] my post about compiling the VM I told you to use a PharoCore image since it was the “recommended” way. However, in the […]

  • Named Primitives « Mariano Martinez Peck

    […] VM, as well as telling the VM which plugins to compile, is explained in a previous posts such as this one and this […]

  • Community driven VM « Mariano Martinez Peck

    […] everybody can do it. That’s all I have tried with my first posts of this blog. My post about how to build the VM from scratch has already like 800 visists….So…hopefully more people can build Cog VM […]

  • Ben Coman

    Having done…
    $ git clone –depth 1 git://gitorious.org/cogvm/blessed.git

    I am getting….
    Admin@Ben-Tecra ~/cogVM/blessed
    $ git checkout f3fe94c828f66cd0e7c37cfa3434e384ff65915e
    fatal: reference is not a tree: f3fe94c828f66cd0e7c37cfa3434e384ff65915e

    And I get the following…
    Admin@Ben-Tecra ~/cogVM/blessed
    $ git log
    commit 4a65655f0e419248d09a2502ea13b6e787992691
    Merge: 82ad789 ba737db
    Author: Esteban Lorenzano
    Date: Sat Dec 3 22:00:58 2011 -0300

    Merge branch ‘master’ of gitorious.org:~estebanlm/cogvm/cog-osx

    commit 82ad789bb444b182a0de292600aa6e29037aa5a4
    Author: Esteban Lorenzano
    Date: Sat Dec 3 21:59:52 2011 -0300

    using version 1.4-1

    commit ba737db8e171ade42fc4a493ffe1b3a5ec1ed397
    Author: Esteban Lorenzano
    Date: Sat Dec 3 21:56:37 2011 -0300

    missing icon files

    Has something that was in the repository been moved out?
    What hash should I use?

  • Stefan

    Ben,

    if you check out with –depth 1 then you are checking out only a single (1) revision. And that is usually the git HEAD. In your case the commit id is 4a65655f0e419248d09a2502ea13b6e787992691 for what you checked out. And that is the only thing you got.

    If you want to contribute to the source, using git, then you want to clone without ‘–depth 1’.

    From the manual: –depth
    Create a shallow clone with a history truncated to the specified number of revisions. A shallow repository has a number of limitations (you cannot clone or fetch from it, nor push from nor into it), but is adequate if you are only interested in the recent history of a large project with a long history, and would want to send in fixes as patches.

  • Ben Coman

    Thanks Stefan. I now ran…
    $ git clone git://gitorious.org/cogvm/blessed.git
    and after downloading all 200MiB then…
    $ git log | grep f3fe94c828f66cd0e7c37cfa3434e384ff65915e
    is successful, whereas before it wasn’t, and…
    $ git checkout f3fe94c828f66cd0e7c37cfa3434e384ff65915e
    is now successful.

    That then leaves the question, in order to satisfy Mariano’s intent of reproducability, is there some shortcut to checkout a specific hashkey without having the clone the entire repository?

    • marianopeck

      Hi Ben. In the post I say:

      For this post, I recommend to take the first option if you may be a beginner. I told you that I wanted my posts to be reproducible. With the previous commands, you will clone the latest version in the repository. Since I don’t know when you are going to do it (if there is someone), I would like that you load the specific version I know it works. What I am suggesting is doing a kind of “svn co http://xxxx -r 2202″. I checked how to do this in git, and it seems not to provide a clone of a specific version. Instead, you just clone (from the latest one) and then you checkout or revert to a previous one. Execute:

      1 cd blessed
      2 git checkout f3fe94c828f66cd0e7c37cfa3434e384ff65915e

      f3fe94c828f66cd0e7c37cfa3434e384ff65915e is the commit hash of the version I want. You can do “git log” to see the latest commits or “git rev-parse HEAD” to see the last one.

      Isn’t that enough?

      • Ben Coman

        That bit is fine and straightforward, except that it doesn’t work with the preceeding given instruction
        $ git clone –depth 1 git://gitorious.org/cogvm/blessed.git
        rather than
        $ git clone git://gitorious.org/cogvm/blessed.git

        I was happy to wait for the full repository to enhance reproducabiity, but other might not.

        Another option for future reference might be to put a spcific version into a separate gitorious account that isn’t updated.

      • marianopeck

        Thanks Ben, you are right. As soon as I have some minutes I will update the post with all this information. Thank you very much for the feedback. And thanks Stefan for helping him 🙂

    • Stefan

      Hi Ben:

      Not normally, no. You would need to guess how old it is…
      The manual is here by the way: http://schacon.github.com/git/git-clone.html

      From my perspective, the best in terms of reproducibility you can do is a tarball coming with a script that is able to recreate the tarball.

      Regards
      Stefan

  • Ben Coman

    After
    Smalltalk>> CogMsWindowsConfig new generateSources; generate.
    $ cd ~/cogVM/blessed/build
    build$ cmake . -G”MSYS Makefiles”
    build$ make

    I hit this problem….
    C:/Apps/MinGW/msys/1.0/home/Admin/cogVM/blessed/platforms/Cross/vm/sqSCCSVersion.h:17:54: fatal error: vmVersionInfo.h: No such file or directory

    For which I found a solution at http://forum.world.st/About-sqSCCVersion-h-td3684142.html, such that from the build directory:
    build$ ../codegen-scripts/extract-commit-info.sh
    build$ make
    successfully passed this issue and on to the next problem (which I will report on later).

    btw I am using the Microsoft Windows 7 Professional 32-bit.

    • marianopeck

      Ahhh yes, when I wrote the post, such thing about ../codegen-scripts/extract-commit-info.sh didn’t exist. I will try to update the post to include such information. Thanks Ben.

    • marianopeck

      Hi Ben. Now that I was updating the post I thought… Why you have the error of vmVersionInfo.h: if you were building from the git version I said (git checkout f3fe94c828f66cd0e7c37cfa3434e384ff65915e). This is weird because this stuff of the extract-commit-info.sh was commited only in “latest” vm sources….

      Are you sure you are building agains f3fe94c828f66cd0e7c37cfa3434e384ff65915e ?

      • Ben Coman

        Sorry to have confused two separate issues. The first issue was that f3fe94c… was not available with the
        `git clone –depth 1 git://gitorious.org/cogvm/blessed.git` command that was originally in your post, which I see you have corrected.

        The second issue was sqSCCSVersion.h occured because I proceeded with default available from the `git clone –depth 1 git://gitorious.org/cogvm/blessed.git` , which in my first comment `git log` showed as 4a65655…

        I have now checked out f3fe94c… in a fresh clone and the sqSCCSVersion.h error does not occur. In fact, trying to run extract-commit-info.sh generates an error because it does not exist – as you mentioned. You might change your post to have an aside or footnote which says “At this point in later versions you need to run extract-commit-info.sh”

        …or alternatively that script might be included as its own make target.

      • marianopeck

        Thanks Ben. I understand now. So I have just changed the post 🙂

  • ncalexan

    The stanza

    “CocoaIOSCogJitConfig is an example. If you are not in Mac, replace it with CogMsWindowsConfig or CogUnixConfig”
    CocoaIOSCogJitConfig new
    “Using VMMaker we translate Cog to C”
    generateSources;
    “We generate all the CMake necessary directories and files”
    generate.

    failed for me because an instance variable (MacOSConfig>>resourcesDir) did not exist. It at least runs after I define the instance variable.

    This was with everything listed on this page — image, versions, git repo, etc.

    Nick

    • marianopeck

      Thanks Nick. Indeed, it was wrong. What I don’t understand is how I couldn’t see it before….Anyway, I have updated the post to use version CMakeVMMaker-MarianoMartinezPeck.83 where I notice I have already fixed that. I checked with the metioned version of git, and everything works. So I just updated the post to use CMakeVMMaker-MarianoMartinezPeck.83 rather than .73.
      Thanks!

  • thekilon

    Does not work for me, it displays an error that it cant find the sources when it tries to generate sources. The same error I get when I open the pharo image you supplied. I am linux mint 32. I am also trying to compile with Nativeboost support.

    • marianopeck

      Hi. The problem is that you need the .sources files in order to generate the VM sources. You can download the .surces file from: https://gforge.inria.fr/frs/download.php/24391/PharoV10.sources.zip
      You have to put the .sources in the same directory where the binary of the VM is, or, in the same directory where the .image is. I will update the post with this information. Let me know if it works.
      Thanks

      • thekilon

        thanks it worked but now I get a new error . I am trying to generate the sources but with the nativeboost configuration

        this is my workspace

        Deprecation raiseWarning: false.
        Gofer new
        squeaksource: ‘MetacelloRepository’;
        package: ‘ConfigurationOfCog’;
        load.
        ((Smalltalk at: #ConfigurationOfCog) project version: ‘1.5’) load.

        Gofer new
        squeaksource: ‘VMMaker’;
        package: ‘CMakeVMMaker’;
        version: ‘CMakeVMMaker-MarianoMartinezPeck.83’;
        load.

        NBInstaller installPlugin.
        NBCogUnixConfig generateWithSources.

        The error is message not understood: receiver of setTargetPlatform is nil. So i assume that someway the target platform is not set ? Sorry I am a noob with squeak and this is way above my level of skill.

      • marianopeck

        Hi. I have never tried NativeBoost before, but maybe this thread helps: http://forum.world.st/About-installing-NativeBoost-in-Linux-td2715504.html ?

      • Igor Stasenko

        perhaps because you using pretty old config version.
        ((Smalltalk at: #ConfigurationOfCog) project version: ’1.5′) load.

        browse the ConfigurationOfCog and check what is the last version there. and then try loading it.

  • kilon

    thank you very much my friend seem i was missing loads and loads of steps which igor for some reason has not documented in his wiki. But thats the nature of open source software fighting your way through weird implementation and weak documentation 😀 At least its free and open, no complaints here 😀 Again congratulation for taking the time to explain a process that is far from easy.

    • marianopeck

      I am happy you made it 🙂 I have also update the post to link with an zip that includes the .sources as well as explaining in the post that such file is needed.
      Thanks for your feedback.

  • kilon

    I tried in macos Lion, with a cog machine and a pharo 1.4 image, I get all of dependencies issues when I try the first 2 gopher instructions. I had to click “proceed” 5-6 times. I feel the whole process is abit out of my league, I am very confused by it. I took a look at Vmaker packages and nativeboost dont know why but only half the packages are installed by Gopher, tried to install them some got me an error that they tried to rewrite a class or something.

    I will keep trying , maybe one day I will be sucessfull. I plan to retry on linux , I have not been sucessfull with it so far .

    Another thing I dont understand is the purpose of an image in all of this, and can I use any kind of image with cog ? Can I use a pharo compiled cog with squeak image ? Can I use an existing non cog squeak image with squeak cog ? It would help if the community sit down and writes some detailed documentation on this because I see a gazillion things that can go wrong.

    Simplifying the installation would be alot more desirable, we got morphic why not use it to make a nice simple GUI for building Cog and Nativeboost and this way encourage people to use Cog and Nativeboost ? Is this hard to do for Cog and NB developers ?

    Anyway, I have already Cog with Nativeboost from Pharo image 1.4 (pre made build), but would like to learn to build it myself (so I can keep up with any new progress in both of them) so I will keep trying.

    Oh and Igor hey there, cogratulations to you both guys for making such amazing software I can wait to use all the features of Cog and Nativeboost. Yes I have checked and in Macos i run the latest versions, but I was not able to install all classes for the above reasons. I will make sure this applies for my Linux efforts too.

    I hope with your help will enevntually be successful . I am planing a project that hopefully will change the look of squeak and pharo and Cog + Opengl (through nativeboost) can help alot as my project will deal with graphics , sound and visual programming.

    here is my project , still in planning phase , I got so much to learn 😀

    http://www.squeaksource.com/ephestos.html

    • marianopeck

      What I recommend you is to first try to build the VM EXACTLY as I say in this post. Using exactly the mentioned version, images and links. The post is REPRODUCIBLE. Once you were successful, you can do a second step and try with the latest code. Finally, try NB.

      I tried in macos Lion, with a cog machine and a pharo 1.4 image, I get all of dependencies issues when I try the first 2 gopher instructions. I had to click “proceed” 5-6 times.

      For Lion, there is a problem with GCC. Read all the threads:
      http://forum.world.st/template/NamlServlet.jtp?macro=search_page&node=104410&query=lion+gcc
      http://forum.world.st/template/NamlServlet.jtp?macro=search_page&node=3068605&query=lion+gcc

      The probem you find in latest 1.4 is because you are trying the bleedinge edge of everything, and indeed, there is a problem now loading Cog with the latest Pharo 1.4 image. My recomendation is to use the Pharo 1.3 image that I include in this post. Over that image, you can try to load if you want the last version from Git and 3.8 of ConfigurationOfCog.

      Another thing I dont understand is the purpose of an image in all of this, and can I use any kind of image with cog ?

      The image is because the core of the VM is written in Smalltalk itself (a subset in fact). Hence, you need it. Please, read:
      https://marianopeck.wordpress.com/2011/04/02/departure-vm-introduction/

      here is my project , still in planning phase , I got so much to learn 😀
      http://www.squeaksource.com/ephestos.html

      Nice. Welcome to the VM world. I hope you can make it.

    • kilon

      I used exactly , the instruction you provide . In summary

      1) Downloaded YOUR image

      2) Downloaded YOUR cogvm

      3) Opened the image with cogvm

      4) Run all the instructions, and I have been successful with building sources

      bare in mind i put image from the very start in the cogvm folder image subfolder

      I soon as I try to make this happens

      —>

      cmake .

      — The C compiler identification is GNU
      — The CXX compiler identification is GNU
      — Checking whether C compiler has -isysroot
      — Checking whether C compiler has -isysroot – yes
      — Checking whether C compiler supports OSX deployment target flag
      — Checking whether C compiler supports OSX deployment target flag – yes
      — Check for working C compiler: /usr/bin/gcc
      — Check for working C compiler: /usr/bin/gcc — works
      — Detecting C compiler ABI info
      — Detecting C compiler ABI info – done
      — Checking whether CXX compiler has -isysroot
      — Checking whether CXX compiler has -isysroot – yes
      — Checking whether CXX compiler supports OSX deployment target flag
      — Checking whether CXX compiler supports OSX deployment target flag – yes
      — Check for working CXX compiler: /usr/bin/c++
      — Check for working CXX compiler: /usr/bin/c++ — works
      — Detecting CXX compiler ABI info
      — Detecting CXX compiler ABI info – done

      Debug;Release
      Adding internal plugin: ADPCMCodecPlugin
      Adding internal plugin: BMPReadWriterPlugin
      Adding internal plugin: B2DPlugin
      Adding internal plugin: BitBltPlugin
      Adding internal plugin: DSAPrims
      Adding internal plugin: ZipPlugin
      Adding internal plugin: DropPlugin
      Adding internal plugin: FFTPlugin
      Adding internal plugin: FilePlugin
      Adding internal plugin: FloatArrayPlugin
      Adding internal plugin: GeniePlugin
      Adding internal plugin: HostWindowPlugin
      Adding internal plugin: JPEGReadWriter2Plugin
      Adding internal plugin: JPEGReaderPlugin
      Adding internal plugin: Klatt
      Adding internal plugin: LargeIntegers
      Adding internal plugin: Matrix2x3Plugin
      Adding internal plugin: MiscPrimitivePlugin
      Adding internal plugin: RePlugin
      Adding internal plugin: SecurityPlugin
      Adding internal plugin: SocketPlugin
      Adding internal plugin: SoundCodecPrims
      Adding internal plugin: SoundPlugin
      Adding internal plugin: StarSqueakPlugin
      Adding internal plugin: SurfacePlugin
      Adding external plugin: SqueakFFIPrims
      Adding external plugin: UUIDPlugin
      Adding external plugin: FloatMathPlugin
      Adding external plugin: AsynchFilePlugin
      Adding external plugin: SerialPlugin
      Adding external plugin: Mpeg3Plugin
      Adding external plugin: CroquetPlugin
      Adding external plugin: JoystickTabletPlugin
      Adding external plugin: MIDIPlugin
      Adding external plugin: B3DAcceleratorPlugin
      Adding external plugin: LocalePlugin
      — Configuring done
      CMake Error at CMakeLists.txt:46 (add_executable):
      Cannot find source file:

      /Users/kilon/cogvm/blessed/macbuild/resources/ProjectBuilder/MainMenu.nib

      Tried extensions .c .C .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp
      .hxx .in .txx

      — Build files have been written to: /Users/kilon/cogvm/blessed/build

      I got Lion , MacOSX . Have a nice day .

      • marianopeck

        Hi. It looks like you forget one step, which is the one of loading EXACTLY the version of git repository I say. In the post I say:

        For this post, I recommend to take the first option if you may be a beginner. I told you that I wanted my posts to be reproducible. With the previous commands, you will clone the latest version in the repository. Since I don’t know when you are going to do it (if there is someone), I would like that you load the specific version I know it works. What I am suggesting is doing a kind of “svn co http://xxxx -r 2202″. I checked how to do this in git, and it seems not to provide a clone of a specific version. Instead, you just clone (from the latest one) and then you checkout or revert to a previous one. Execute:

        cd blessed
        git checkout f3fe94c828f66cd0e7c37cfa3434e384ff65915e

        It seems MainMenu.nib is present at that version I say you should checkout, but it is not present in latest git version. So that’s why I guess you are using the lastest and not the specific version I said 🙂

      • kilon

        I hope you dont think it would be THAT easy, do you 😀 . I got a new error for you , since I am getting a ton of warnings too I used pastebin to save space . Here it the link , have fun 😀

        http://pastebin.com/qYn2zTj1

      • marianopeck

        Hi Kilon. I have no clue about that error. I would do 2 things:
        1) Clean/Remove everything: /blessed/build, /blessed/src and /blessed/results
        2) Be SURE to be using GCC rather than LLVM.
        If doing so, you still have the error, then I would send an email to the VM mailing list because I have no idea:
        http://lists.squeakfoundation.org/mailman/listinfo/vm-beginners
        http://lists.squeakfoundation.org/mailman/listinfo/vm-dev

        BTW…yesterday I fixed ConfigurationOfCog and created version 3.9. In latest Pharo 1.4 it should load and generate sources without problems 🙂

  • kilon

    oh Igor forgot to say that my latest version is 3.8 according to the class you mention, version38: method. Nice to meet you Igor, I enjoyed your nativeboost presentation in ESUG . Mariono I am about to watch your pharocast too, I am still very much a pathetic noob but I am slowly learning so sorry if I miss something obvious here… I am slow but I a reliable 😀

  • kilon

    as always you were right it was LLVM. I installed gcc from macports and , did

    sudo ln -s /opt/local/bin/gcc-apple-4.2 /Developer/usr/bin/gcc-4.2

    sudo ln -s /opt/local/bin/g++-apple-4.2 /Developer/usr/bin/g++-4.2

    cmake -D CMAKE_C_COMPILER=gcc-4.2 -D CMAKE_CXX_COMPILER=g++-4.2 .

    The above is wrong in LION the correct is

    cmake -D CMAKE_C_COMPILER=gcc-apple-4.2 -D CMAKE_CXX_COMPILER=g++-apple-4.2 .

    so you may want to correct that in your tutorial at least for Lion.

    I am still getting erros , but different ones this time, I posted a question together with the log in Pharo-project mailing list. Thank you for all your help so far. I will keep trying. Never Surrender 😀

    • marianopeck

      Thanks. I have updated the tip for the CMake. As you can imagine I don’t have a box here with Lion 😦
      I saw your email, but I don’t have a clue this time. Maybe you need the 32 bits library for some libs? no idea..let’s see if someone can answer…

      • kilon

        no worries that why I am here to try to get this working on Lion (and linux / windows 7), I will keep trying till I get it right and try to learn as much I can for the whole process. I had other issues with building apps for Lion so I know that compiling in Lion can be abit problematic. I will certainly googlinvestigate it deeply to see what might go wrong under these circumstances. I am not in any hurry to build it as already the NBCogVm that I have downloaded works pretty well, but helping you guys provide latest builds frequently and even debug cogvm and nativeboost is something that can benefit my project greatly. So I will keep trying till I get it right.

  • damienpollet

    Typo: you mention “CMMake” several times, I believe you actually mean CMake. Small detail I know, but it’s already confusing enough…

  • Halting when VM sends a particular message or on assertion failures « Mariano Martinez Peck

    […] with CogVM? Of course, we first need to build the VM in “debug mode”. Here you can see how to build the VM, and here how to do it in debug mode. Then we can do something like this (taken from an […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: