[Simgrid-user] Scalability of PlatformGenerator improved...

Kurt Vanmechelen kurt.vanmechelen at ua.ac.be
Thu Jul 24 11:16:59 CEST 2008


Hello,

I tried out the PlatformGenarator under the contrib packages which generates SimGrid 3.3 platforms. When generating platforms with a very large number of clusters using BARABASI (only 
method I tested up until now), the generator does not seem to scale. More specifically, the transformation from the graph representation to the platform file is very slow. 

This is caused by the fact that the platform XML is built up using += operators on String objects. This causes a lot of copying and resulting slowdowns. I chabged the following two 
methods, for improved scalability (routes and platform are StringBuffer members):

private void makeRoute(ArrayList route, StringBuffer routes) {	
		routes.append("<route:multi src=\"" + (String)route.get(0) + "\" dst=\"" + (String)route.get(route.size()-1) + "\" symmetric=\"NO\">");
		for (int i=1; i<route.size() - 1; i++) {
			String id = "bb_" + (String)route.get(i);// + "-" + (String)route.get(i);
			String idSymmetric = "bb_" + (String)route.get(i);// + "-" + (String)route.get(i-1);
			if (alreadyCreatedLinks.containsKey(idSymmetric) == false) {
				routes.append("<link:ctn id=\"" + id + "\"/>");
				alreadyCreatedLinks.put(id, id);
			}
			else {
				routes.append("<link:ctn id=\"" + idSymmetric + "\"/>");				
			}
		}
		routes.append("<link:ctn id=\"$dst\"/></route:multi>\n");		
	}


	private String makeFromGraph(WeightedGraph t, boolean fromGraphFile) {
		StringBuffer platform = new StringBuffer(1024*1024*5);

		/* make cluster tags */
		String id;
		for (int start=0; start<t.size(); start++) {
			if ((id = (String)t.getLabel(start)) != null) {				
				if (id.startsWith("@") == false)
					platform.append(makeCluster(id, fromGraphFile));
			}
		}
		StringBuffer routes = new StringBuffer(1024*1024*5);
		Hashtable l = new Hashtable();
		ArrayList route = null;
		int [] pred;
		for (int start=0; start<t.size(); start++) {				
			if (t.getLabel(start) != null) {					
				pred = Dijkstra.dijkstra (t, start);
	      			for (int other=0; other<t.size(); other++) {	      				
					if (t.getLabel(other) != null) {						
						if (((String)t.getLabel(start)).startsWith("@") == false && ((String)t.getLabel(other)).startsWith("@") == false) {					
							route = Dijkstra.getPath(t, pred, start, other);
							//Dijkstra.printPath(t, pred, start, other);
							if (route.size() > 1)
								makeRoute(route, routes);
							for (int k=1;k<route.size()-1; k++) {
								//add bb_paris_nancy but not bb_nancy_paris. they are identical
//								if (l.containsKey("bb_"+route.get(k)+"-"+route.get(k-1)) == false)
									final int e = getIndexFromLabel(t, (String)route.get(k));
									final int s = getIndexFromLabel(t, (String)route.get(k -1));
									l.put("bb_"+route.get(k), ""+t.getWeight(s, e));//+"-"+route.get(k), "bb_"+route.get(k-1)+"-"+route.get(k));
							}
						}
					}					
				}
			}
		}
		/* make links */
		String links = "";
		Iterator it = l.keySet().iterator();
		int index = 1;
		while (it.hasNext()) {
			final String name = (String)it.next();
			links += makeLink(name, Double.parseDouble((String)l.get(name)), fromGraphFile);
			System.out.println("Link " + index + "/" + l.keySet().size());
			index++;
		}
		platform.append(links);
		platform.append(routes);

		return platform.toString();
	}

Note that I also removed the router elements in the routes. Is this OK? The original code generated for every link :
	//Snippet of makeRoute... 
	for (int i=1; i<route.size() - 1; i++) {
		String id = "bb_" + (String)route.get(i);
		r += "<link:ctn id=\"" + id + "\"/>";
		r += "<router id=\"router_" + id + "\"/>";
But as far as I know, SimGrid doesn't really consider these router elements (apart from GTNeTS)?

Thanks for the feedback!

Cheers,

Kurt
-- 
================================================
Kurt Vanmechelen
Phd Student
Dept. of Mathematics and Computer Sciences
Group CoMP (Computational Modelling and Programming)
University of Antwerp
Middelheimlaan 1
2020 Antwerpen, Belgium
G2.14

Phone : 03/265 3477
Mail  : kurt.vanmechelen at ua.ac.be
================================================




More information about the Simgrid-user mailing list