The new-style psptoolchain script.

Discuss the development of new homebrew software, tools and libraries.

Moderators: cheriff, TyRaNiD

ooPo
Site Admin
Posts: 2023
Joined: Sat Jan 17, 2004 9:56 am
Location: Canada
Contact:

The new-style psptoolchain script.

Post by ooPo »

After releasing the previous toolchain update on my site and watching some of the problems people had, I've updated it:

http://dev.oopo.net/files/psptoolchain-20070517.tar.bz2

There's been no changes to the patches, they're the same as you'd get with the current psptoolchain in the subversion repository.

Now for the why.

1) The toolchain script has grown annoying to manage.
2) There's a lot of random stuff added to the script instead of doing things like properly merging patches.
3) Checking dependencies before running is very important - adding more should be easy to do.

So, here's the new-style script:

Code: Select all

#!/bin/sh
# toolchain.sh by Dan Peori (danpeori@oopo.net)

 ## Create the build directory.
 mkdir -p `dirname $0`/build || { echo "ERROR: Could not create the build directory."; exit 1; }

 ## Enter the build directory.
 cd `dirname $0`/build || { echo "ERROR: Could not enter the build directory."; exit 1; }

 ## Run the depend scripts.
 for SCRIPT in `ls ../depends/*.sh | sort`; do $SCRIPT || { echo "$SCRIPT: Failed."; exit 1; } done

 ## Run the build scripts.
 for SCRIPT in `ls ../scripts/*.sh | sort`; do $SCRIPT || { echo "$SCRIPT: Failed."; exit 1; } done
All it does is create a build directory, run all the scripts in the 'depends' directory, and then all the scripts in the 'scripts' directory.

Adding a new dependency check is as simple as adding a new script in the 'depends' directory. Here's what we currently have:

Code: Select all

psptoolchain/depends/check-autoconf.sh
psptoolchain/depends/check-automake.sh
psptoolchain/depends/check-bison.sh
psptoolchain/depends/check-flex.sh
psptoolchain/depends/check-gcc.sh
psptoolchain/depends/check-make.sh
psptoolchain/depends/check-ncurses.sh
psptoolchain/depends/check-patch.sh
psptoolchain/depends/check-pspdev.sh
psptoolchain/depends/check-subversion.sh
psptoolchain/depends/check-texinfo.sh
psptoolchain/depends/check-wget.sh
Here's what one of these scripts look like:

Code: Select all

#!/bin/sh
# check-make.sh by Dan Peori (danpeori@oopo.net)

 ## Check for make.
 make -v 1> /dev/null || { echo "ERROR: Install make before continuing."; exit 1; }
Simple, yes? That's the point.

Build scripts are similar, but are numbered so that they get executed in the correct order:

Code: Select all

psptoolchain/scripts/001-binutils-2.16.1.sh
psptoolchain/scripts/002-gcc-4.1.0-stage1.sh
psptoolchain/scripts/003-pspsdk-stage1.sh
psptoolchain/scripts/004-newlib-1.15.0.sh
psptoolchain/scripts/005-gcc-4.1.0-stage2.sh
psptoolchain/scripts/006-pspsdk-stage2.sh
psptoolchain/scripts/007-gdb-6.4.sh
psptoolchain/scripts/008-insight-6.4.sh
Here's what one looks like on the inside:

Code: Select all

#!/bin/sh
# binutils-2.16.1.sh by Dan Peori (danpeori@oopo.net)

 ## Download the source code.
 if test ! -f "binutils-2.16.1.tar.bz2"; then
  wget ftp://ftp.gnu.org/pub/gnu/binutils/binutils-2.16.1.tar.bz2 || { exit 1; }
 fi

 ## Unpack the source code.
 rm -Rf binutils-2.16.1 && tar xfvj binutils-2.16.1.tar.bz2 || { exit 1; }

 ## Enter the source directory and patch the source code.
 cd binutils-2.16.1 && cat ../../patches/binutils-2.16.1-PSP.patch | patch -p1 || { exit 1; }

 ## Create and enter the build directory.
 mkdir build-psp && cd build-psp || { exit 1; }

 ## Configure the build.
 ../configure --prefix="$PSPDEV" --target="psp" --enable-install-libbfd || { exit 1; }

 ## Compile and install.
 make clean && make -j 2 && make install && make clean || { exit 1; }
If nobody objects, my plan in the near future is to create a 1.0 branch in subversion for the old script, and start clean with this script.

Questions? Comments?
hlide
Posts: 739
Joined: Sun Sep 10, 2006 2:31 am

Post by hlide »

that's a good way to modulate this script. I think there would be no objection.
hlide
Posts: 739
Joined: Sun Sep 10, 2006 2:31 am

Post by hlide »

well i failed with ncurse because I havn't /usr/lib/libncurses.so which is normal because i'm under cygwin and not under linux.

I think you toolchain need to be aware whether the host is windows or linux (or MacOX ?).
hlide
Posts: 739
Joined: Sun Sep 10, 2006 2:31 am

Post by hlide »

oh well, if a novice wants to install it, he/she would run into troubles :

Error : PSPDEV not set <== why not to propose one by default when not set ? usually /usr/local/pspdev as it is with the current psptoolchain.

I don't expect him/her to do a "export PSPDEV=/usr/local/pspdev" every time before running the toolchain.

I think you must fix those things before replacing the current psptoolchain with this one.

1) be sure it works like a charm under every (cygwin, linux, MacOS X)
2) don't let your scripts to be too petty. Whenever you have a variable not set, try to propose one by default so that people wanting just a working compiler to be able to compile a source they retrieve can do it without any knowledge about scripts
ooPo
Site Admin
Posts: 2023
Joined: Sat Jan 17, 2004 9:56 am
Location: Canada
Contact:

Post by ooPo »

well i failed with ncurse because I havn't /usr/lib/libncurses.so which is normal because i'm under cygwin and not under linux.
What does cygwin use for a termcap library? You need one in order to compile gdb. Would searching for libtermcap.so work instead?
I don't expect him/her to do a "export PSPDEV=/usr/local/pspdev" every time before running the toolchain.
I expect a user to read the documentation and do what's needed to build and use a toolchain:

Code: Select all

 ==================
  How do I use it?
 ==================

 1&#41; Set up your environment by installing the following software&#58;

  autoconf, automake, bison, flex, gcc, make, ncurses, patch, subversion, terminfo, wget

 2&#41; Add the following to your login script&#58;

  export PSPDEV=/usr/local/pspdev
  export PATH=$PATH&#58;$PSPDEV/bin

 3&#41; Run the toolchain script&#58;

  ./toolchain.sh
I do agree that the error messages could be more helpful, but having the script use defaults and try to assume things will have the user end up with a toolchain that still won't work properly until they do the setup they should have done in the first place.
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

Go for it! Looks good to me.
hlide
Posts: 739
Joined: Sun Sep 10, 2006 2:31 am

Post by hlide »

ooPo wrote:
well i failed with ncurse because I havn't /usr/lib/libncurses.so which is normal because i'm under cygwin and not under linux.
What does cygwin use for a termcap library? You need one in order to compile gdb. Would searching for libtermcap.so work instead?
i do have ncurse, but libncurses.so is linux-ish ! there is no .so in cygwin (but rather .dll). I suppose you may test libncurses.dll.a which is in the same directory instead.
ooPo wrote:
I don't expect him/her to do a "export PSPDEV=/usr/local/pspdev" every time before running the toolchain.
I expect a user to read the documentation and do what's needed to build and use a toolchain:

Code: Select all

 ==================
  How do I use it?
 ==================

 1&#41; Set up your environment by installing the following software&#58;

  autoconf, automake, bison, flex, gcc, make, ncurses, patch, subversion, terminfo, wget

 2&#41; Add the following to your login script&#58;

  export PSPDEV=/usr/local/pspdev
  export PATH=$PATH&#58;$PSPDEV/bin

 3&#41; Run the toolchain script&#58;

  ./toolchain.sh
I do agree that the error messages could be more helpful, but having the script use defaults and try to assume things will have the user end up with a toolchain that still won't work properly until they do the setup they should have done in the first place.
oh well i didn't read your readme.txt because i was expecting to use this psptoolchain the same way i did with the current one. So, yes, I should check if my login script has the right exports.
hlide
Posts: 739
Joined: Sun Sep 10, 2006 2:31 am

Post by hlide »

anyway, by just ignoring the ncurse check, i was able to run psptoolchaintool up to the end, so I suppose configure is able to retrieve ncurse even under cygwin.
ooPo
Site Admin
Posts: 2023
Joined: Sat Jan 17, 2004 9:56 am
Location: Canada
Contact:

Post by ooPo »

hlide wrote:i do have ncurse, but libncurses.so is linux-ish ! there is no .so in cygwin (but rather .dll). I suppose you may test libncurses.dll.a which is in the same directory instead.
You're absolutely right. Duh. :) That's an easy enough change to make.
ooPo
Site Admin
Posts: 2023
Joined: Sat Jan 17, 2004 9:56 am
Location: Canada
Contact:

Post by ooPo »

http://dev.oopo.net/files/psptoolchain-20070518.tar.bz2

Code: Select all

#!/bin/sh
# check-ncurses.sh by Dan Peori &#40;danpeori@oopo.net&#41;

 ## Check for a ncurses library.
 ls /usr/lib/libncurses.* 1> /dev/null || &#123; echo "ERROR&#58; Install ncurses before continuing."; exit 1; &#125;
This should work for everyone now, correct?
hlide
Posts: 739
Joined: Sun Sep 10, 2006 2:31 am

Post by hlide »

i tried psp-gdb, psp-gdbtui and psp-insight which seem to work perfectly (psp-gdbtui should use ncurse, right ?)

so i think you can go now
futaris
Posts: 45
Joined: Wed Dec 28, 2005 7:47 am

Post by futaris »

Don't suppose the script can be modified to download the .tar.gz and .tar.bz2's to a PSP_DL environment variable. Openembedded, buildroot, etc do something similar, so we can just download everything to one directory.
weltall
Posts: 310
Joined: Fri Feb 20, 2004 1:56 am
Contact:

Post by weltall »

sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

weltall wrote:sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)
Actually, the current stable gcc is now 4.2.0. :D
weltall
Posts: 310
Joined: Fri Feb 20, 2004 1:56 am
Contact:

Post by weltall »

J.F. wrote:
weltall wrote:sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)
Actually, the current stable gcc is now 4.2.0. :D
i mean regarding available gcc patches for psp :P
hlide
Posts: 739
Joined: Sun Sep 10, 2006 2:31 am

Post by hlide »

weltall wrote:
J.F. wrote:
weltall wrote:sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)
Actually, the current stable gcc is now 4.2.0. :D
i mean regarding available gcc patches for psp :P
those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.
ooPo
Site Admin
Posts: 2023
Joined: Sat Jan 17, 2004 9:56 am
Location: Canada
Contact:

Post by ooPo »

weltall wrote:sorry but where are the 4.0.2 patches? they are no more useful as 4.1.0 is now stable? the latest time I've used it I had a lot of problems with the resulting elfs doing strange things (more in optimized states like O3)
You can still grab the old toolchain and patches:

svn export svn://svn.pspdev.org/psp/tags/psptoolchain-1.0.0

Are there problems with 4.1.0? When I asked nobody seemed to know of any outstanding issues.
Luke
Posts: 5
Joined: Thu Sep 21, 2006 11:35 pm

Post by Luke »

Is this only an update to the toolchain-install-script or does this update the toolchain? If so, how do I do that? Can I simply run the new script?
weltall
Posts: 310
Joined: Fri Feb 20, 2004 1:56 am
Contact:

Post by weltall »

hlide wrote:
weltall wrote:
J.F. wrote: Actually, the current stable gcc is now 4.2.0. :D
i mean regarding available gcc patches for psp :P
those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.
do you mean that those problems i experienced with it were made by not defining that command line argument?
ooPo
Site Admin
Posts: 2023
Joined: Sat Jan 17, 2004 9:56 am
Location: Canada
Contact:

Post by ooPo »

Luke wrote:Is this only an update to the toolchain-install-script or does this update the toolchain? If so, how do I do that? Can I simply run the new script?
This is an update to the install script. If you already have a toolchain installed you don't need this - you're already up to date.

This new script just makes it easier to maintain updates in the future.
Luke
Posts: 5
Joined: Thu Sep 21, 2006 11:35 pm

Post by Luke »

Cool, thanks! And by the way: Great work! Really. PSP-Programming rocks! Thanks for making this possible for me!
hlide
Posts: 739
Joined: Sun Sep 10, 2006 2:31 am

Post by hlide »

weltall wrote:
hlide wrote:
those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.
do you mean that those problems i experienced with it were made by not defining that command line argument?
there is a topic somewhere covering the fact you can have wrong code because of strict aliasing and can be worse with -O3. Try to find it. This topic explains why and how to avoid those wrong code.

So yes I think your problems can be from the fact that you don't use this command line.
weltall
Posts: 310
Joined: Fri Feb 20, 2004 1:56 am
Contact:

Post by weltall »

hlide wrote:
weltall wrote:
hlide wrote:
those patches are absolutely necessary since they are specific to psp (especially MIPS architecture specific to PSP). Don't forget to add -fno-strict-aliasing in your makefile to avoid strange things.
do you mean that those problems i experienced with it were made by not defining that command line argument?
there is a topic somewhere covering the fact you can have wrong code because of strict aliasing and can be worse with -O3. Try to find it. This topic explains why and how to avoid those wrong code.

So yes I think your problems can be from the fact that you don't use this command line.
ok thanks when i have some time i will build the new toolchain and try again :)
Alek
Posts: 10
Joined: Fri Mar 09, 2007 3:26 am

Post by Alek »

Err... didn't say anything xD

Installing well, good job :)
lmame
Posts: 2
Joined: Sun May 27, 2007 12:45 pm
Location: July City
Contact:

Post by lmame »

Hi ^_^

First of all thank you for the toolchain and this new script, I'm quite new to psp dev world and for now I was just using oslib / pspsdk / cygwin under windows and when I saw your new script I decided to give it a try under Linux :)

I managed really simply to install and run things unit the last script until the last script is called: 008-insight-6.4.sh :
When the make -j 2 is run, I got this error (one line lost in an ocean of text):

Code: Select all

/bf/toolchain/binutils-2.17/tk/unix/../generic/tk.h&#58;96&#58;23&#58; X11/Xlib.h&#58; No such file or directory
So I had to install X11 and run it again ;)

Perhaps it would be call to check also if X11 exists in one of your script, I guess it would be cool for noobs like me ;)

Anyway, now it's working :)
Life is full of love and peace ^_^
hlide
Posts: 739
Joined: Sun Sep 10, 2006 2:31 am

Post by hlide »

ooPo :

gcc 4.2 is out, it has less regressions than gcc 4.1. It might be interesting to use it instead, provided that the current patches are still applyable.

gcc 4.2 : [ all : 252, serious : 129 ]
gcc 4.1 : [ all : 297, serious : 173 ]

EDIT : silly of me ! I wrote 2.x instead of 4.x !!!
Last edited by hlide on Mon May 28, 2007 8:43 pm, edited 1 time in total.
ufoz
Posts: 86
Joined: Thu Nov 10, 2005 2:36 am
Location: Tokyo
Contact:

Post by ufoz »

Surely you mean 4.2?
hlide
Posts: 739
Joined: Sun Sep 10, 2006 2:31 am

Post by hlide »

arfff yes it is 4.x not 2.x 8x !!!!
hlide
Posts: 739
Joined: Sun Sep 10, 2006 2:31 am

Post by hlide »

GRRRRRRRRRRRRRRRRR !!!!

the option "-mpreferred-stack-boundary=n" is not longer valid in this version !!!! still I modified some gcc files in gcc/config/mips directory so we can change the alignment requirement for stack !? does it means you use a different mips patch !?

now some of my sources are broken because of that.
jimparis
Posts: 1145
Joined: Fri Jun 10, 2005 4:21 am
Location: Boston

Post by jimparis »

Isn't relying on stack alignment a bit broken anyway? I thought even "-mpreferred-stack-boundary" is just an attempt not a guarantee. You could use memalign if alignment matters.
Post Reply