[Pharo-project] A tinyBenchmark

Eliot Miranda eliot.miranda at gmail.com
Thu Feb 10 19:17:50 CET 2011


2011/2/10 Felipe Bañados Schwerter <fbanados at dcc.uchile.cl>

> Hi Everybody! This is my first post on the list, even though i've been
> subscribed for more than 6 months.
>
> 2011/2/10 Levente Uzonyi <leves at elte.hu>
>
> On Thu, 10 Feb 2011, Torsten Bergmann wrote:
>>
>>  see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark
>>>
>>> Maybe Eliot is able to say something here on the main
>>> differences between Cog <-> VW VM.
>>>
>>
>> You made several mistakes:
>> - You didn't post the source code for #tinyBenchmarks. Do all those
>> Smalltalks implement it? Are the implementations equal?
>>
>
> No, they don't implement it. Just Pharo and Squeak. I just added the same
> implementation to Integer in VA, VisualWorks and GNU Smalltalk (
> Integer>>tinyBenchmarks, Integer>>benchmark and Integer>>benchFib), so the
> source code is the same.
>
> - You didn't specify the version numbers for the VMs used. Or if you built
>> them yourself, then the sourcecode + compiler version + configuration
>> parameters. Without these your measurements can't be reproduced.
>>
>
> Every System was the one available from their websites. (Got Pharo 1.2 from
> Hudson about 3 days ago.)  However, VisualWorks is not available from its
> website right now. I installed it from a Cincom CD-ROM I got on Smalltalks
> 2010 .
>
>
>> - There are two different Cog VMs. You didn't specify which one did you
>> use with Pharo.
>>
>
> Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter
> VMMaker-oscog.35]').
>

But Smalltalk getSystemAttribute: 1008 returns the JIT version, which is
much more important for this performance measurement.


>
>
>> - You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak
>> 4.2. What's even worse is that you based your conclusion on this mistake.
>>
>
> Yes, it is true. Daniel Galdames also pointed this out last night on the
> blog. But as I told before, it is just the benchmarks you get from the
> "download the latest version available" approach. I couldn't find a
> Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the
> time to repeat this until next week, then I'll try this.
>
>
>> - Does #tinyBenchmarks alone give fair performance comparison? Hardly.
>>
>> Well, it was the benchmark I knew of. Do you know a better performance
> benchmark available? If so, I could retry the experiment.
>

As large a benchmark that is as close to your working case as possible.
 tinyBenchmarks is little more than a smoke test (which is what we use it
for at work; if tinyBenchmarks hasn't plummeted ten the build is OK
performance-wise).

In the Cog-Benchmarks (in Cog package) that I uploaded to squeaksource
recently you'll find ShootoutTests which has 4 tests from the computer
language shootouts that are pure Smalltalk.


>
>
>>
>> Levente
>>
>>
>>
>>> Thx
>>> T.
>>> --
>>> NEU: FreePhone - kostenlos mobil telefonieren und surfen!
>>> Jetzt informieren: http://www.gmx.net/de/go/freephone
>>>
>>>
>>>
>>
>
>
> --
> Felipe Bañados Schwerter
> Estudiante de Ingeniería Civil en Computación
> http://www.dcc.uchile.cl/~fbanados
> Universidad de Chile
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gforge.inria.fr/pipermail/pharo-project/attachments/20110210/302b3de8/attachment.htm>


More information about the Pharo-project mailing list