pthreads

[hectic128]

I've finished up an implementation of the pthreads API for the PSP. The implementation is pretty complete, the only thing missing are things that don't really apply to the PSP platform. See the link below...

http://sourceforge.net/projects/pthreads-emb/

[jimparis]

Cool! Nice work.

[jean]

Not looked into it yet, but it sounds good! Nice job man!

[miniJB]

Nice work. Tests run great on psp FAT (after some very slight changes to make printf work and compile to pbp), though a few take a while ;). Very nice test set too!

I like the abstraction layer you use to support multiple platforms... and that its inherited pthreads-win32's exception handling. I was in the middle of a pthreads port based on Pth (but using the psp threading libs rather than setjmp/etc), so you sure saved me some work!

[pegasus2000]

I've finished up an implementation of the pthreads API for the PSP. The implementation is pretty complete, the only thing missing are things that don't really apply to the PSP platform. See the link below...

http://sourceforge.net/projects/pthreads-emb/

I have to ask you a thing.

I'm trying to port STLPort in nd environment. The previous
attempts have failed, because I haven't a ready pthread library
for PSP.

I want to ask you the permission to integrate the code of your
library in ndNanoCPP library. In this way, Nanodesktop will became able
also to support C++, and not only C.

Can I integrate your library ?

Thank you
Filippo Battaglia

[miniJB]

I have a suggestion regarding vfpu access..


I note that pthread.h defines

Code: Select all

      /*
       * pthread_{get,set}scope
       */
      PTHREAD_SCOPE_PROCESS         = 0,
      PTHREAD_SCOPE_SYSTEM          = 1,  /* Default */
      PTHREAD_SCOPE_PROCESS_VFPU    = 2,  /* PSP specific */

however in pthread_attr_setscope.c

Code: Select all

int
pthread_attr_setscope (pthread_attr_t * attr, int contentionscope)
{
#ifdef _POSIX_THREAD_PRIORITY_SCHEDULING
  switch (contentionscope)
    {
    case PTHREAD_SCOPE_SYSTEM:
      (*attr)->contentionscope = contentionscope;
      return 0;
    case PTHREAD_SCOPE_PROCESS:
      return ENOTSUP;
    default:
      return EINVAL;
    }
#else
  return ENOSYS;
#endif
}

only PTHREAD_SCOPE_SYSTEM seems honoured, and then only if _POSIX_THREAD_PRIORITY_SCHEDULING is defined.

Further, in create.c

Code: Select all

pthread_create (pthread_t * tid,
                const pthread_attr_t * attr,
                void *(*start) (void *), void *arg)
{
 //...
      stackSize = a->stacksize;
      tp->detachState = a->detachstate;
      priority = a->param.sched_priority;

 //...
}

does not examine the contentionscope (for any osal!).

and in pte_osal_generic.h

Code: Select all

pte_osResult pte_osThreadCreate(pte_osThreadEntryPoint entryPoint,
                                int stackSize,
                                int initialPriority,
                                void *argv,
                                pte_osThreadHandle* ppte_osThreadHandle);

no contentionscope is passed.

and in psp_osal.c,

Code: Select all

pte_osResult pte_osThreadCreate(pte_osThreadEntryPoint entryPoint,
                                int stackSize,
                                int initialPriority,
                                void *argv,
                                pte_osThreadHandle* ppte_osThreadHandle)
{

  //...
  pspAttr = 0;

  //  printf("%s %p %d %d %d\n",threadName, pspStubThreadEntry, initialPriority, stackSize, pspAttr);
  threadId = sceKernelCreateThread(threadName,
                                   pspStubThreadEntry,
                                   initialPriority,
                                   stackSize,
                                   pspAttr,
                                   NULL);
  //...
}

pspAttr is set to 0 without any check against the contentionscope.


Would it not be advantageous to add the contentionscope parameter to pte_osThreadCreate (since stacksize and priority are already here) and noop that for the osals that do not support it? We can then use this to set pspAttr.

This would allow the PSP to use PTHREAD_SCOPE_PROCESS for user threads and PTHREAD_SCOPE_SYSTEM for kernel threads, and we could also alter PTHREAD_SCOPE_PROCESS_VFPU to simply PTHREAD_SCOPE_VFPU, allowing us to use the familiar OR ops to specify VFPU access.

I understand this may not work as the scope normally does on pthread-compliant systems, and that error detection when setting contentionscope values may not be as robust, but it would be handy to specify vfpu threads...

edit:
Just realised that using PTHREAD_SCOPE_SYSTEM to make a kernel pthread when in a user thread obviously wouldnt work. I guess this should give an error at thread creation, but this has little bearing on exposing the vfpu...

[pegasus2000]

I've finished up an implementation of the pthreads API for the PSP. The implementation is pretty complete, the only thing missing are things that don't really apply to the PSP platform. See the link below...

http://sourceforge.net/projects/pthreads-emb/

I'm glab to tell that your library will be included in a new
component of the next Nanodesktop distribution called
NanoD (NanoC helper).

Naturally, in the documentation I'll do references to your
code and to your copyright.

Let's I know if you want some particular message or note
in nd documentation for the user, in way that I can fully
respect your license.

Bye
Filippo

[hectic128]

Let's I know if you want some particular message or note
in nd documentation for the user, in way that I can fully
respect your license.

Bye
Filippo

That's cool!

The only thing I would ask is to also put a link to the sourceforge site.

If you find any problems or make any improvements I'd appreciate if you could send me patches.

I haven't actually used it in a real application yet, so I'd be interested to hear how it works out for you.

[hectic128]

Would it not be advantageous to add the contentionscope parameter to pte_osThreadCreate (since stacksize and priority are already here) and noop that for the osals that do not support it? We can then use this to set pspAttr.

You're exactly right, and I like your solution.

This is one of those areas that I glossed over when I originally did it, and apparently never got around to fixing it...

Thanks for catching this.

[hectic128]

Nice work. Tests run great on psp FAT (after some very slight changes to make printf work and compile to pbp), though a few take a while ;). Very nice test set too!

I like the abstraction layer you use to support multiple platforms... and that its inherited pthreads-win32's exception handling. I was in the middle of a pthreads port based on Pth (but using the psp threading libs rather than setjmp/etc), so you sure saved me some work!

How is the exception handling stuff working? This was one area that I was always concerned about - it seemed like it should work OK, and the tests passed but it still seemed like there could be problems.

[miniJB]

How is the exception handling stuff working? This was one area that I was always concerned about - it seemed like it should work OK, and the tests passed but it still seemed like there could be problems.

I, like you, have so far only tested in under the unit tests that come with your pthreads-emb library. So far Ive only used your library to assist porting other software, and since (afaik) only the windows port of pthreads supports the exception handling, none of the software ive used pthreads-emb with required exception-handling.

[Wally]

oooo

So this means sheepshaver can be ported now :D

[miniJB]

Sheepshaver? I (cautiously) googled that, not knowing what it was... Thank goodness that despite the dodgy-sounding name, googles first hits are benign... Seems like an interesting PowerPC emulator...

If it needs pthreads, then yes this definitely helps port it.
Unfortunately, the graphics for the linux version seem to use fbdev, you will need to write a fbdev "driver" for the psp or alter sheepshaver to use the ge... Theres a nice little project in itself!

[paulotex]

Hi. I'm trying to compile an autotool project that needs pthreads and I was hoping that pthreads-emb could be a solution. The project uses g++ instead of gcc, and linking fails. I've reduced the problem to the conftest.cpp file:

Code: Select all

#include <pthread.h>
int
main &#40;&#41;
&#123;
pthread_t th; pthread_join&#40;th, 0&#41;;
                     pthread_attr_init&#40;0&#41;; pthread_cleanup_push&#40;0, 0&#41;;
                     pthread_create&#40;0,0,0,0&#41;; pthread_cleanup_pop&#40;0&#41;;
  ;
  return 0;
&#125;

If I use gcc, it compiles fine:

Code: Select all

$ psp-gcc -G0 -O2 -Wall -g -fno-strict-aliasing -G0 -L`psp-config -p`/lib -L`psp-config -P` -lc -lpspuser pth.c -lpthreads -lc -lpspuser -lpspsdk

However, if I use g++, things fail:

Code: Select all

psp-g++ -G0 -O2 -Wall -g -fno-strict-aliasing -G0 -I/Users/paulo/include  -L/Users/paulo/pspdev/psp/sdk/lib -lc -lpspuser pth.c -lpthreads -lc -lpspuser -lpspsdk
/var/folders/o4/o4pFMcJ3HoqF-VfCYAAZzk+++TI/-Tmp-//cchOjnUH.o: In function `main':
/Users/paulo/Documents/PSP/Apps/Bookr/Arrr/djvulibre-3.5.21-PSP/pth.c:5: undefined reference to `pthread_join(pte_handle_t, void**)'

I noticed that pthreads-emb was compiled with gcc instead of g++. Could that be the problem?

Thanks.

[TyRaNiD]

I would assume that the header does not extern C the function definitions so when compiled g++ is trying to find the mangled form which doesn't exist.

[hectic128]

I would assume that the header does not extern C the function definitions so when compiled g++ is trying to find the mangled form which doesn't exist.

Yup. Fix committed to SVN trunk.

Not sure how I missed that one...

[paulotex]

Yup. Fix committed to SVN trunk.

It works now, thanks. I had to add -lpspsdk, but I guess that is normal.

Two other minor issues though:

* I noticed that "make install" works partially for the PSP. pthreads.h is not installed so I copied it manually. Perhaps you can add that too?

* I had to change "#include <sched.h>" to "#include <sys/sched.h>" so that the project I was compiling could find it. I don't know if this is a fault of my project (missing the correct -I path) or if it is your source that should be changed.

Thanks for the great work.

[Rex_VF5]

I am quite a lamer with C/C++ and even more with auto configure/build tools on Linux. However I am attempting to compile some library that uses standard configure script. When it looks for pthreads it just doesn't find it altough I have compiled it and installed them to PSP SDK (make install). The output is like this:

Code: Select all

--------------------------- pthread stuff -------------------------------------
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... no
checking whether pthreads work with -pthreads... no
checking whether pthreads work with -mthreads... no
checking for the pthreads library -lpthread... no
checking whether pthreads work with --thread-safe... no
checking whether pthreads work with -mt... no
checking for pthread-config... no
configure: error: POSIX threads are required to build this program

Can anybody advice on how to fix this so I can proceed? Thank you very much.

[paulotex]

Hi Rex_VF5.

Have a look in config.log, it will tell you why it failed. I compiled djvu, and this lib needed "-lpspuser" to pass the configure tests, so I had to fix "configure.ac". Besides, I also had to point it to the correct pthreads lib. So my configure command was:

Code: Select all

CXXFLAGS="-I`psp-config -p`/include -I`psp-config -P`/include -G0 -fno-rtti" LDFLAGS="-L`psp-config -P`/lib -L`psp-config -p`/lib -lc -lpspuser" PTHREAD_LIBS="-lpthreads -lpspsdk" ./configure --host=psp --prefix=`psp-config -P` --disable-shared --enable-static --without-x --without-qt

Quite a mouthfull :) but YMMV...

[Rex_VF5]

Thank you for answer but this is going to be more complicated (due to my newbieness):

Code: Select all

configure:22142: checking for pthread_join in LIBS=-lpthreads -lpspsdk with CFLAGS=
configure:22172: psp-gcc -o conftest -g -O2 -Os -Wall   -L/opt/toolchains/psp//psp/lib -L/opt/toolchains/psp//psp/sdk/lib conftest.c -lpthreads -lpspsdk -lc -lpsplibc -lpspuser >&5
/opt/toolchains/psp/bin/../lib/gcc/psp/4.3.1/../../../../psp/bin/ld: cannot find -lpthreads
collect2: ld returned 1 exit status

So it says it cannot find -lpthreads. However I have successfully built (make) and installed (make install) pthreads-emb. so now I have libpthread-psp.a in /opt/toolchains/psp/psp/lib. My environment variable for PSP are:

Code: Select all

export PSPDEV="/opt/toolchains/psp/"
export PSPSDK="$PSPDEV/psp/sdk"
export PATH="$PATH:$PSPDEV/bin:$PSPSDK/bin"

I am obviously missing somethiink ;-)

[Rex_VF5]

Stupid me - there was a typo -lpthreads-psp

Now I get something like this:

Code: Select all

--------------------------- pthread stuff -------------------------------------
checking for pthread_join in LIBS=-lpthread-psp -lpspsdk with CFLAGS=... yes
checking for joinable pthread attribute... unknown
checking if more special flags are required for pthreads... no
----------------------- pthread_rwlock_t stuff --------------------------------
checking if pthread_rwlock_t is available... no, needs to fallback to pthread_mutex
configure: error: pthread_rwlock_t not available

The relevant part of the config.log says:

Code: Select all

configure:22608: checking if pthread_rwlock_t is available
configure:22637: /opt/toolchains/psp/bin/psp-gcc -c  -I/opt/toolchains/psp/psp/include/c++/4.3.1     -I/opt/toolchains/psp/psp/include     -I/opt/toolchains/psp/psp/sdk/include  -L
/opt/toolchains/psp/lib/gcc/psp/4.3.1 -static -Os -Wall  conftest.c >&5
conftest.c: In function 'main':
conftest.c:52: error: 'pthread_rwlock_t' undeclared (first use in this function)
conftest.c:52: error: (Each undeclared identifier is reported only once
conftest.c:52: error: for each function it appears in.)
conftest.c:52: error: 'x' undeclared (first use in this function)

Looking up my toolchain there is some pthread.h in /opt/toolchains/psp/psp/include and pthread_rwlock_t is declared in /opt/toolchains/psp/psp/include/sys/types.h that says:

Code: Select all

typedef __uint32_t pthread_rwlock_t;         /* POSIX RWLock Object */

However the pthread.h differs greatly from the one in pthreds_emb. When I overwrote the toolchain one with pthreads_emb one (and changed the include #include <sys/sched.h>) it started to work!!!

[Rex_VF5]

I have used this library to compile another library. When it tries to create a thread it crashes like this:

Code: Select all

host0:/> Exception - Address load/inst fetch
Thread ID - 0x04E49B7D
Th Name   - pthread0002__8a05110
Module ID - 0x01FF992D
Mod Name  - upnpcmd
EPC       - 0x0883A1C4
Cause     - 0x10000010
Exception - Address load/inst fetch
BadVAddr  - 0xFFFFFFFC
Thread ID - 0x04D27239
Status    - 0x00088613
Th Name   - user_main
zr:0x00000000 at:0xDEADBEEF v0:0xFFFFFFFC v1:0xFFFFFFFC
Module ID - 0x01FF992D
a0:0x08A05110 a1:0x08970000 a2:0xFFFFFFFC a3:0x00000010
Mod Name  - upnpcmd
t0:0x00000020 t1:0x0BBF4C04 t2:0x00000000 t3:0x08A05110
EPC       - 0x0883A1C4
t4:0x00000001 t5:0x0FFFFFFF t6:0xFFFFFFFF t7:0x0886A129
Cause     - 0x10000010
s0:0xDEADBEEF s1:0xDEADBEEF s2:0xDEADBEEF s3:0xDEADBEEF
BadVAddr  - 0xFFFFFFFC
s4:0xDEADBEEF s5:0xDEADBEEF s6:0xDEADBEEF s7:0xDEADBEEF
Status    - 0x20088613
t8:0x00000010 t9:0x00000057 k0:0x0BBF4F00 k1:0x00000000
zr:0x00000000 at:0xDEADBEEF v0:0xFFFFFFFC v1:0xFFFFFFFC
gp:0x08875F60 sp:0x0BBF4EB8 fp:0x0BBF4EC0 ra:0x08839690
a0:0x00000000 a1:0x08970000 a2:0xFFFFFFFC a3:0x0BBBFC10
0x0883A1C4: 0x8C430000 '..C.' - lw         $v1, 0($v0)
t0:0xFFFFFFFF t1:0x0BBBFC10 t2:0xDEADBEEF t3:0xDEADBEEF
t4:0xDEADBEEF t5:0xDEADBEEF t6:0xDEADBEEF t7:0xDEADBEEF
s0:0x00000000 s1:0x04E4FD75 s2:0x04E4FD75 s3:0x08A04F7C
s4:0x00000000 s5:0x00000013 s6:0xDEADBEEF s7:0xDEADBEEF
t8:0xDEADBEEF t9:0xDEADBEEF k0:0x0BBBFF00 k1:0x00000000
gp:0x08875F60 sp:0x0BBBFCA8 fp:0x0BBBFD98 ra:0x088399D4
0x0883A1C4: 0x8C430000 '..C.' - lw         $v1, 0($v0)

host0:/> disasm $epc
0x0883A1C4: 0x8C430000 '..C.' - lw         $v1, 0($v0)

It crashes on the final statement *tid = thread; of pthread_create in create.c. I have added some debugging outputs into the code like this:

Code: Select all

FAIL0:
  if (result != 0)
    {

      pte_threadDestroy (thread);
      tp = NULL;

      if (parms != NULL)
        {
          free (parms);
        }
    }
  else
    {
      if (tid != NULL)
pspDebugScreenPrintf("\ntid=%lX", tid);
pspDebugScreenPrintf("\nthread=%lX", thread);
        {
          *tid = thread;
        }
    }

  return (result);

}

and the result is like this:

Code: Select all

tid=BBBFDA0
thread=BBBFD70

Please anybody help me. I am really lost about what possibly do next.

[cosmito]

I've finished up an implementation of the pthreads API for the PSP.

Do you see as viable to take this PSP port as starting point for a PS2 port or would be simpler to port from the original instead?

[Rex_VF5]

I have been trying to figure this out whole weekend but didn't progress much. However I compiled pthread-psp-test and ran it. It gets stuck at Semaphore test #4. I thought maybe the FW was to blame - I was on 5.00 M33-3 so I downgraded to 3.90 M33-3 but the result is the same. Under 5.00 I tried to run the test through pslink but there was no exception. Maybe the exception is not related to the hang in Semaphore test #4...

Can anybody please advice on what to do next? I have uploaded my compiled pthread-psp-test here: http://d01.megashares.com/?d01=12e1682 so other people can try and tell me if it is something on my side... Any help is greatly welcome...

[mase]

I have include pthread.h.
But I get an error:

Code: Select all

‘pthread_t’ does not name a type