[Simgrid-user] Excessive Memory Use

Martin Quinson martin.quinson at loria.fr
Sat Jul 23 23:32:59 CEST 2011


Thanks for using SimGrid. Propeling actual science is the whole
purpose of our work, which you prove useful. That's great. The tone of
my mails are too often harsh, sorry about it. I try to work on it.

For SIMIX_process_empty_trash(), I'll fix it in the git tomorrow. I'll
also try to see if I can understand why task_cancel was raising the
impossible exception. This exception is a bug marker that you
shouldn't swallow in your code. I'd appreciate if you could report the
next time that you're getting it so that we can fix it. I may come
back to you at some point to check my fix for task_cancel.

As for non blocking communications, it's clear that spawning a new
process is quite a lot of work for the simulator where running an
asyncronous communication is really really (really) cheap. I wouldn't
be suprised if told that the performance ratio is about or even over
1000-1 between these operations. 

If run time is not limiting for your simulations so far and if
emptying the process trash works for you, you can certainly continue
that way, but if time become limitating at some point, that's
certainly the feature that you want to leverage to solve your issue.
That's maybe not trivial to use, and certainly awfully
underdocumented, but it really helped us for the chord example.

Bye, Mt.

On Sat, Jul 23, 2011 at 11:38:53AM -0300, Wagner Kolberg wrote:
> It seems that we have found the problem! Using the
> SIMIX_process_empty_trash() function did the trick.
> 
> And you are absolutely right, I guess that simgrid's non-blocking
> functions like MSG_task_isend() will solve my problem without spawning
> new processes, my bad. And that will probably use even less memory?
> 
> So thank you for your effort and cooperation.
> 
> Wagner
> 
> On Fri, Jul 22, 2011 at 8:52 PM, Martin Quinson <martin.quinson at loria.fr> wrote:
> >
> > It seems that you trigered a bug in the recent rewrite of SIMIX. We
> > forget to clean the terminated processes until the very end of the
> > simulation. Try calling
> >
> >  SIMIX_process_empty_trash();
> >
> > somewhere in your code to see whether it helps (just add a fake
> > prototype in your code if you need to, that's an internal function
> > which does not live in any public header). This function is in charge
> > of freeing the context of dead processes.
> >
> > I'd say that in previous version it was called in each scheduling
> > round (ie main loop of the simulator), and now the only remaining call
> > is in SIMIX_process_killall(), itself only called from SIMIX_clean().
> >
> >
> > I think we found your bug (in our code, sorry). Now. Why the heck are
> > you spawning a process per communication? I fear that it's to achieve
> > non blocking communication. If so, you may want to switch to
> > MSG_task_isend(), MSG_task_irecv(), MSG_task_test() and
> > MSG_task_wait(). Before you ask, MSG_task_dsend() is a detached send
> > where you send the data in best effort and don't care of whether it
> > gets received or raises an error (see the chord example). And
> > MSG_task_isend_with_matching() allows to mimick the MPI tag mechanism.
> >
> > Thanks for your time,
> > Mt.
> >
> > --
> > Those are my principles, and if you don't like them... well, I have others.
> >  -- Groucho Marx
> >
> > _______________________________________________
> > Simgrid-user mailing list
> > Simgrid-user at lists.gforge.inria.fr
> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/simgrid-user
> >
> 
> _______________________________________________
> Simgrid-user mailing list
> Simgrid-user at lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/simgrid-user

-- 
The only merit of this paper is to demonstrate all what you have to
not do when writing an article.      -- Bastard Reviewer From Hell



More information about the Simgrid-user mailing list