Boosting SandstoneDB with Fuel

Hi guys. I continue to find nice Fuel examples to show at ESUG, and I while doing so, I post my results 😉 . It may be useful for someone else apart from myself.  SandstoneDB is a database similar to Ruby’s ActiveRecord. SandstoneDB is a simple object database that uses (by default) SmartRefStreams to serialize clusters of objects to disk. It seems that currently there is also a GOODS database backend.

For more details about SandstoneDB please read: http://onsmalltalk.com/sandstonedb-simple-activerecord-style-persistence-in-squeak. There is also an excellent screencast about it in PharoCasts.

In summary, SandstoneDB is a simple database which I think it may be useful for small/medium applications. Anyway….the goal of this short post is to show you how to speed-up SandstoneDB by using Fual serializer instead of SmartRefStream. Ramon Leon, author of SandstoneDB did the integration between the both. This was pretty straighforward and worked out of the box in the lass one hour. It was necessary to only create the class SDFuelSerializer with two methods: #serialize: anObject toFile: aFile  and #materializeFromFile: aFile.

So….how to install and use SandstoneDB with Fuel? Quite easy in fact:

1) Grab a Pharo image and install Fuel. Otherwise, you can download an image from Jenkis: https://ci.lille.inria.fr/pharo/job/Fuel/. Anyway, if you want to take a Pharo image and install Fuel in it, just execute:

Gofer new
squeaksource: 'MetacelloRepository';
package: 'ConfigurationOfFuel';
load.

((Smalltalk at: #ConfigurationOfFuel) project version: '1.6') load.

2) Install SandstoneDB and its integration with Fuel:

Gofer new
squeaksource: 'SandstoneDb';
package: 'SandstoneDb';
package: 'SandstoneDbTests';
package: 'SandstoneDbFuel';
load.

3) If you run now the tests with SandstoneDB you will still be using SmartRefStream instead of Fuel, even if we have downloaded the package “SandstoneDbFuel”. So if you want to use Fuel as the default serializer you have to evaluate:

SDFileStore serializer: SDFuelSerializer new.

4) That’s all 🙂 You can now run all SandstoneDB tests, they all should be using Fuel and they all should be green 🙂

Now, how much speed-up you can have? Well…that depends on the graph you are serializing. The bigger it is the graph to serialize, the faster Fuel will be over SmartRefStream. If the graph is large, Fuel can be like 10x faster than SmartRefStream. If it is small, much less. The following is a benchmark where we individually serialize 500 persons. Each person is small, hence there is not that big difference. Nevertheless, in this test, Fuel is 2.5 times faster in serialization 1.5 times in materialization.

| commitTime people lookupTime loadTime |
 "  SDFileStore serializer:  SDSmartRefStreamSerializer new."
 SDFileStore serializer: SDFuelSerializer new.
 SDActiveRecord
 setStore: SDFileStore new;
 warmUpAllClasses.
 "only want to warm up test models, not anything else that might be in this image"
 SDFooObject warmUp.
 SDPersonMock withAllSubclasses do: [ :each | each warmUp ].

people := (1 to: 500) collect: [ :it | SDPersonMock testPerson ].
 commitTime := [ people do: [ :each | each save ] ] timeToRun.
 lookupTime := [ people do: [ :each | SDPersonMock atId: each id ] ] timeToRun.
 loadTime := [
 SDActiveRecord resetStoreForLoad.
 SDPersonMock
 withAllSubclassesDo: [ :each | SDActiveRecord store ensureForClass: each ];
 withAllSubclassesDo: [ :each | each warmUp ].
 SDActiveRecord store ensureForClass: SDFooObject.
 SDFooObject warmUp ] timeToRun.
 Transcript
 show: 'Serialiation time: ', commitTime asString;
 cr;
 show: 'Materialization time: ', loadTime asString;
 cr;
 cr.
 SDPersonMock do: [ :each | [each delete] on: Error do: [] ].
 SDPersonMock coolDown.
 SDFooObject do: [:each | [each delete] on: Error do: [] ].
 SDPersonMock allSubclassesDo: [ :each | each coolDown ].
 Smalltalk garbageCollect
 

If we take the same example what instead of serializing each person individually we save serialize an object that contains all those persons, Fuel is 12x faster in serialization and 6x in materialization.

Anyway…all this was just a proof of concept. All I can tell you is that all tests are green and that Fuel seems to speed-up SandstoneDB. If you want to give it a try, please go ahead. And if you already have an application running with SandstoneDB and SmartRefStream I would appreciate if you can benchmark both since it is interesting to see a real case.

See you.