[Pharo-project] Remove #class as a Compiler optimization

Camillo Bruni camillo.bruni at inria.fr
Tue Apr 5 15:29:35 CEST 2011


On 2011-04-05, at 15:11, Igor Stasenko wrote:

> I don't care :)
> 
> What i found that measurements confirms my previous expectation:
> the 'slowdown' of removing #class from set of optimized selectors lies
> in the range of macro-benchmark deviation.
> 
> And running all tests for 100 times is not fun (it could take half of
> the day), and i'm not ready to leave my computer for half of the day
> and simply wait till it finish, because if i will do something
> else,then it will affect the performance, and measurements will be
> even more inaccurate.

thats why you run it over night :) (or the day, depending on your sleep rhythm).
or you reduce the number of runs to something at least bigger than 1 :D.

> On 5 April 2011 14:53, Camillo Bruni <camillo.bruni at inria.fr> wrote:
>> This is exactly why you have to provide some confidence interval / deviation, otherwise it is hard to make any reasonable conclusion.
>> 
>> run it 100 times and take the average and provide the standard deviation.
>> 
>> I am not a big fan of relying on incomplete benchmarking results:
>> 
>> Please read: http://portal.acm.org/citation.cfm?id=1297033
>> 
>> http://www.squeaksource.com/p.html provides a basic benchmarking framework under the NBenchmark package. You subclass from PBenchmarkSuite implement a method #benchXXX and run it.
>> 
>> r := PBFloat run: 100.
>> r asString
>> 
>> which will give decent results back :). This way it is much easier to make sense out of the numbers.
>> 
>> So here again to remember:
>> 
>> - number of samples
>> - average run times
>> - standard deviation
>> 
>> If one of these results is missing the benchmark results are incomplete.
>> 
>> best regards,
>> camillo
>> 
>> 
>> 
>> On 2011-04-05, at 13:56, Igor Stasenko wrote:
>> 
>>> VariableNode initialize.
>>> Compiler recompileAll.
>>> 
>>> [
>>> TestCase allSubclasses do: [ :cls|
>>>        cls isAbstract
>>>                ifFalse: [cls suite run]].
>>> ] timeToRun
>>> 
>>> 178938
>>> 183963
>>> 
>>> 
>>> 
>>> (ParseNode classVarNamed:  'StdSelectors') removeKey: #class ifAbsent: [].
>>> Compiler recompileAll.
>>> 
>>> [
>>> TestCase allSubclasses do: [ :cls|
>>>        cls isAbstract
>>>                ifFalse: [cls suite run]].
>>> ] timeToRun
>>> 
>>> 187168
>>> 184992
>>> 
>>> the deviation is too big to see if its really so big overhead.
>>> 
>>> if you compare worst , you got  187/178 ~ 5%
>>> and if you compare the best you got
>>> 184/183  ~ 0.5%
>>> 
>>> 
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>> 
>> 
>> 
>> 
> 
> 
> 
> -- 
> Best regards,
> Igor Stasenko AKA sig.
> 




More information about the Pharo-project mailing list