[Pharo-project] Remove #class as a Compiler optimization

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Tue Apr 5 15:02:10 CEST 2011


Better than average, take the median

Nicolas

2011/4/5 Camillo Bruni <camillo.bruni at inria.fr>:
> This is exactly why you have to provide some confidence interval / deviation, otherwise it is hard to make any reasonable conclusion.
>
> run it 100 times and take the average and provide the standard deviation.
>
> I am not a big fan of relying on incomplete benchmarking results:
>
> Please read: http://portal.acm.org/citation.cfm?id=1297033
>
> http://www.squeaksource.com/p.html provides a basic benchmarking framework under the NBenchmark package. You subclass from PBenchmarkSuite implement a method #benchXXX and run it.
>
> r := PBFloat run: 100.
> r asString
>
> which will give decent results back :). This way it is much easier to make sense out of the numbers.
>
> So here again to remember:
>
> - number of samples
> - average run times
> - standard deviation
>
> If one of these results is missing the benchmark results are incomplete.
>
> best regards,
> camillo
>
>
>
> On 2011-04-05, at 13:56, Igor Stasenko wrote:
>
>> VariableNode initialize.
>> Compiler recompileAll.
>>
>> [
>> TestCase allSubclasses do: [ :cls|
>>        cls isAbstract
>>                ifFalse: [cls suite run]].
>> ] timeToRun
>>
>> 178938
>> 183963
>>
>>
>>
>> (ParseNode classVarNamed:  'StdSelectors') removeKey: #class ifAbsent: [].
>> Compiler recompileAll.
>>
>> [
>> TestCase allSubclasses do: [ :cls|
>>        cls isAbstract
>>                ifFalse: [cls suite run]].
>> ] timeToRun
>>
>> 187168
>> 184992
>>
>> the deviation is too big to see if its really so big overhead.
>>
>> if you compare worst , you got  187/178 ~ 5%
>> and if you compare the best you got
>> 184/183  ~ 0.5%
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>
>
>



More information about the Pharo-project mailing list