[Pharo-project] Remove #class as a Compiler optimization

Stefan Marr pharo at stefan-marr.de
Tue Apr 5 23:26:12 CEST 2011


Hi:

On 05 Apr 2011, at 21:27, Nicolas Cellier wrote:

> But you probably have more ellaborated and well thought method :)
I think we are talking here mostly about quick and dirty benchmarks to confirm intuitions.

The provable best and most cost effective way to do it is: flip a coin (and that is serious, has the same or even better reliability and quality of result)

If you think that is not good enough, well then the absolute minimum is something like this:

- run your benchmark 5 times
  - 5 times with your optimization
  - 5 times without

- look at the collected raw values i.e. measure runtime
- and use your gut feeling to decide whether you have meaningful values and whether your time to do the optimization was well spent (don't eliminate outliers)

If that is still to much to do, well, again, just flip a coin, and you have a high chance that the result of that are actually more reliable.
(the approach above does not account for systematic errors...)

However, once you got your optimization into the VM your favorite build server should do its thing and that should include performance regression testing which is using a "slightly" more sophisticated approach than the described one.


Best regards
Stefan





-- 
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525




More information about the Pharo-project mailing list