[Pharo-project] Remove #class as a Compiler optimization

Camillo Bruni camillo.bruni at inria.fr
Tue Apr 5 14:53:27 CEST 2011


This is exactly why you have to provide some confidence interval / deviation, otherwise it is hard to make any reasonable conclusion.

run it 100 times and take the average and provide the standard deviation.

I am not a big fan of relying on incomplete benchmarking results:

Please read: http://portal.acm.org/citation.cfm?id=1297033

http://www.squeaksource.com/p.html provides a basic benchmarking framework under the NBenchmark package. You subclass from PBenchmarkSuite implement a method #benchXXX and run it.

r := PBFloat run: 100.
r asString

which will give decent results back :). This way it is much easier to make sense out of the numbers.

So here again to remember:

- number of samples
- average run times
- standard deviation

If one of these results is missing the benchmark results are incomplete.

best regards,
camillo



On 2011-04-05, at 13:56, Igor Stasenko wrote:

> VariableNode initialize.
> Compiler recompileAll.
> 
> [
> TestCase allSubclasses do: [ :cls|
>        cls isAbstract
>                ifFalse: [cls suite run]].
> ] timeToRun
> 
> 178938
> 183963
> 
> 
> 
> (ParseNode classVarNamed:  'StdSelectors') removeKey: #class ifAbsent: [].
> Compiler recompileAll.
> 
> [
> TestCase allSubclasses do: [ :cls|
>        cls isAbstract
>                ifFalse: [cls suite run]].
> ] timeToRun
> 
> 187168
> 184992
> 
> the deviation is too big to see if its really so big overhead.
> 
> if you compare worst , you got  187/178 ~ 5%
> and if you compare the best you got
> 184/183  ~ 0.5%
> 
> 
> -- 
> Best regards,
> Igor Stasenko AKA sig.
> 




More information about the Pharo-project mailing list