Introducing SPE Execution Management System (spexms)

Investigation into how Linux on the PS3 might lead to homebrew development.

Moderators: cheriff, emoon

Post Reply
unsolo
Posts: 155
Joined: Mon Apr 16, 2007 2:39 am
Location: OSLO Norway

Introducing SPE Execution Management System (spexms)

Post by unsolo »

So people have been wondering why the activity has been low in spu-medialib.

The reason is that i have been working on a framework for easy execution of multiple application across the spe's

The project is listed here
http://sourceforge.net/projects/spexms

And i am still looking for members

Please note this is still in development and that im working on extending the buildt in functions however the code below is not very much compared to what one achives.

The example code to make spexms load two spe binaries and execute them in random order is currently something like this

Code: Select all

 

#include <spexms/spexms.h>
#include <spexms/spexms_types.h>

#include <stdlib.h>

#include <stdio.h>


int main &#40;int nArg, char* cArg&#91;&#93;&#41; &#123;

	int number_of_tasks=0;
	exms_t *xms=init_spes&#40;&#41;;
	
	spu_program_info_t *spi&#91;2&#93;;
	
	spi&#91;0&#93;=register_program&#40;"spe_hello_world"&#41;;	
	spi&#91;1&#93;=register_program&#40;"spe_goodbye_world"&#41;;
	//printf&#40;"First command in the binary is %x",&spi->

	if &#40;nArg > 1&#41;
		number_of_tasks=atoi&#40;cArg&#91;1&#93;&#41;;

	///lets make spexms first task ever;

	spexms_task_header_t xms_th&#91;2&#93;;

	xms_th&#91;0&#93;.program_id=spi&#91;0&#93;->id;
	xms_th&#91;0&#93;.program_location=spi&#91;0&#93;->location;
	xms_th&#91;0&#93;.program_size=spi&#91;0&#93;->size;
	xms_th&#91;0&#93;.no_of_dma=0;

	xms_th&#91;1&#93;.program_id=spi&#91;1&#93;->id;
	xms_th&#91;1&#93;.program_location=spi&#91;1&#93;->location;
	xms_th&#91;1&#93;.program_size=spi&#91;1&#93;->size;
	xms_th&#91;1&#93;.no_of_dma=0;
		
	int i;
	int id;
	for &#40;i=0;i < number_of_tasks;i++&#41;
	&#123;
		id=rand&#40;&#41;%2;
		register_task&#40;xms,spi&#91;id&#93;->id,0,&xms_th&#91;id&#93;&#41;;
	&#125;
	sleep&#40;1&#41;;

&#125;

The code needed in the binary is something like this

Code: Select all

#include <spexms/spexms_buildtin.h>

int main &#40;spexms_buildtin_t *spexms,spexms_pointers_t *pointers&#41;
&#123;
	int value=55;
	//char *mychar="Hello World this is my int";
	spexms->printInt&#40;"Goodbye world",value&#41;;
	return value;

&#125;
Don't do it alone.
ouasse
Posts: 80
Joined: Mon Jul 30, 2007 5:58 am
Location: Paris, France

Post by ouasse »

I've noticed using libspe that it was already possible to run more that 6 spe threads. Some kind of trivial (non-preemptive ?) multithreading across spe's is then performed.

What does your library provide that is not already possible ?
unsolo
Posts: 155
Joined: Mon Apr 16, 2007 2:39 am
Location: OSLO Norway

Post by unsolo »

it moves the code as data and double buffer the code on the spe so techically you can use 65536 programs without doing a context switch. and without any wait*. provided this and that ofc

*one will wait for the few things in the kernel to execute.

The other thing it provides is a fifo scheduler of your tasks so you fire and forget on the ppu and the spu's execute your task as soon as possible.

In addition to that it provides and abstraction layer making it easy to develop things for the spu.
As you can se you dont need to worry about pthread etc and libspe and mbox commands you simply schedule a job and it gets done voila.

Now when it comes to knowing a job is done etc im currently working on that but my priority thus far has been getting fire and forget working.
Don't do it alone.
ps2devman
Posts: 259
Joined: Mon Oct 09, 2006 3:56 pm

Post by ps2devman »

Great! Thanks a lot!
ouasse
Posts: 80
Joined: Mon Jul 30, 2007 5:58 am
Location: Paris, France

Post by ouasse »

this sounds really interesting. Have you planned support for task dependencies, which should schedule all spawned tasks in order to respect some data dependence between them ?

It should be great to be able to specify that task X's input is the output of tasks Y and Z, for instance. Thus, tasks Y and Z must be executed before task X.
unsolo
Posts: 155
Joined: Mon Apr 16, 2007 2:39 am
Location: OSLO Norway

Post by unsolo »

There is a flag for dependency buildt into it as well as a register to keep track of executed tasks in current and "previous" fifo.

But i have not yet buildt that part in ..

This is very complicated and any help will be of interest.
Don't do it alone.
cchalpha
Posts: 9
Joined: Sun Feb 24, 2008 7:08 pm

Post by cchalpha »

unsolo, it's a great idea to make this project.

I believe in homebrews game programming, the spe scheduling should be setup manually for best efficiency. But under linux, it is another case, linux is a general proposed OS, it is necessary to have such a scheduling mechanism to make the spe code executing as easy as possible.

I just co the project, and go through some of them, not finish yet. Here is my questions,
SPE scheduling is known by the process which loaded spexms. But if spexms lib are loaded by many processes, is there any conflict among each other?

I don't know how much I can help, but at least I can add some comments to the code while I read through them.
unsolo
Posts: 155
Joined: Mon Apr 16, 2007 2:39 am
Location: OSLO Norway

Post by unsolo »

Spexms is compatible with the current kernel spu task scheduler so provided tasks are small it should automatically swap.

This would be good even for otheros applications..

so using spu-top + two programs using spexms one should be able to se it even interleaving the two.

Havin said that im trying to work out some good regulation algo (pd regulator in its own pthread) that makes sure sufficient spe's are running to complete your queued tasks as soon as possible
Don't do it alone.
unsolo
Posts: 155
Joined: Mon Apr 16, 2007 2:39 am
Location: OSLO Norway

Post by unsolo »

small video made of spexms + draw_rect (v2) in action from spu.medialib

http://www.customweb.no/unsolo/DSC00055.3GP


Its doing about 1.1GPixels/second read modify and write of 200x200 squares mode is fire and forget so overlapping may cause conflict
Don't do it alone.
Chambers1
Posts: 7
Joined: Fri Jan 25, 2008 6:02 am
Location: US

Post by Chambers1 »

Wow! I don't know what this means but it's getting closer to a fully working x264 media player for ps3 so I'm happy :) Great job unsolo!!
You mean people actually do that?
ouasse
Posts: 80
Joined: Mon Jul 30, 2007 5:58 am
Location: Paris, France

Post by ouasse »

unsolo, this looks very good ! there should be no overlapping problem for drawing code where spes share pre-defined independent parts of the screen memory for drawing stuff.

The question I asked before was mainly for handling 3d code, where spe's would first be used for computing triangles, and then for drawing them.

The dependencies should simply be handled with a global barrier : all computing tasks must be finished before the drawing tasks. this should be easier to implement than dependencies between specific tasks.
unsolo
Posts: 155
Joined: Mon Apr 16, 2007 2:39 am
Location: OSLO Norway

Post by unsolo »

Im going to look at a global Spexms barrier task that waits untill all previous tasks are completed. One also has the dma sync alternatives.. but anyways this is how its currently set up
Don't do it alone.
cchalpha
Posts: 9
Joined: Sun Feb 24, 2008 7:08 pm

Post by cchalpha »

According to your example code in spexms, the call to spu is asynchronous currently, but synchronous calls should be also provided as another choice of programming model/pattern. I think the complete I/O API model should be a good reference, in simple, it calls your handle funtion with the parameter you provided when things are done.
unsolo
Posts: 155
Joined: Mon Apr 16, 2007 2:39 am
Location: OSLO Norway

Post by unsolo »

Spexms is open source GPL2 + exception anyone that feels that they can contribute feel free do join the project.
Don't do it alone.
Post Reply