[Pharo-project] Basic tricks for improving a serializer?
Mariano Martinez Peck
marianopeck at gmail.com
Tue May 17 20:15:59 CEST 2011
On Tue, May 17, 2011 at 3:51 PM, <csrabak at bol.com.br> wrote:
> It is early in the morning right here and I might have read too hastily to
> your post, but from the diagonal reading I gave it seems you're going
> quickly to reinvent the wheel!
Hi. Good evenening.
In fact, we are not reinventing the whele. As I said, Fuel takes the
concepts of Parcels.
> Those issues you rise are typical of database (engine) programming.
Database programming are just ONE usage of serializacion. But there are much
- session management in a webframework.
- remote messages and whatever remote.
- code versioning
In summary, whenever you need to take a subgraph and get a chunk of bytes
from it and vice-versa.
Also, IIRC Smalltalk has another approach to persistence of objects called
> BOSS, have you given a look at it and check if a mix of Fuel + BOSS would
> accomplish your intent?
Yes. It is in VW. I promise we have analyzed all serializers available:
Magma serializer, MC2 serializer, BOSS, ReferenceStream, ImageSegment, etc.
Some of them, like BOSS focus on portability. I don't care at all in
portability. Some other do not even support class reshape or they are too
If it works in other dialects, perfect. But I won't loose performance just
because of that. The main focus of Fuel is to be FAST. And even more, be
fast at deserializacion time (like Parcels). The idea is that at
serializacion time you can do some calculus and groups objects some way so
that the deserializacion is really fast.
> Cesar Rabak
> Em 17/05/2011 10:21, Mariano Martinez Peck < marianopeck at gmail.com >
> Hi guys. Together with Martin we are developing a fast object graph
> serializer called Fuel. The idea is to use a pickle format similar to VW
> Parcels. Right now we are able to correctly serialize (almost) all kind of
> objects (from different classes and so on). The algorithm is working quite
> well and we are already quite fast. We also know how to encode more or less
> efficient some of the "primitive" classes like Floats, SmallIntegers,
> ByteString, ByteArray, etc, etc.
> What we really don't know anything about streams. For example, until now,
> we were using just a MultiByteFileStream and doing all the time things like
> #next: #nextPutAll: #nextWordPut: etc...
> Now Igor told us for example, to use a buffer like this:
> | bufferStream |
> bufferStream := ByteArray new writeStream.
> (FLSerializer on: bufferStream)
> serialize: anArrayToSerialize.
> aFileStream nextPutAll: bufferStream contents.
> and that it at least 2 times faster than we were doing.... I guess it is
> because it goes to the disk only once. But MultiByteFileStream uses a
> buffer, doesn't it ?
> For reading (deserialization) we are not doing nothing special. We just
> open the file stream and we read with #next #nextString ... etc.
> Another thing is which kind of file stream should we use.
> MultiByteFileStream ? StandardFileStream ?
> So...before spending time in some optimizations, are there "known" things
> that we should check?
> Thanks in advance,
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Pharo-project