forums.ps2dev.org

As I'm sure most people who have been here quite a while can remember, a big motivation behind the creation of iopdrv and ps2sdk was to consolidate base system libraries into one package and clean up the environment variable hell we were in.

A while has passed and another "project" has popped up, ps2sdk-ports, which seems to have the same goal of grouping projects together in a singular location.

But I look around and I'm seeing a lot of projects which are requiring their own mess of evironment variables again, even ones which are in this new fangled ps2sdk-ports dir! Worse yet, I see at least once case where despite containing PS2 specific binaries it is using a environment varible with no prefix which looks as though it could easily be in use by a non-cross compiling setup as well (not to mention being inside ps2sdk-ports as well).

We also have another problem of some projects wanting to install themselves in the middle of other libraries.

What it really comes down to is a lack of organization. While I am not trying to advocate some sort of totalitarian state, I think we need to have a few rules to help keep things organized.

The exact rules (and indeed whether people even agree we should have them...) are up for debate, however here are some suggestions...

We should have a standard directory for all libraries or any other projects that are required by another project. $PS2DEV/ports is the directory name being tossed around but I will leave that up to debate.

My original idea for the structure of that directory was a separate directory for each lib, however pixel has brought up the point it should be a canonical directory, which is a good idea and I will let him explain the structure and answer any questions or field any arguments people may have.

PS2SDK, libraries contained in PS2SDK, or libraries that should be contained in PS2SDK would of course not be subject to this rule.

These projects should not provide an environment variable, and any projects which rely upon them should expect to file them at $PS2DEV/ports.

I'm sure there are lots of other issues to bring up and more specifics that could be made, but I'll just open this up for discussion and see what everyone thinks...

So, well, what I said to Drakonite is, whatever the directory would be, it should be in the canonical "lib/include/etc/share/..." form. That is, for example, zlib.h should be present in ports/include and libz.a should be present in ports/lib. Now SDL's includes is usually to be found in the SDL subdir, so it should become ports/include/SDL/SDL.h (just like because "it's usually done this way")

Just a word: the ps2sdk-ports should only be here to have a placeholder for some simple softwares to the PS2, meaning that it should only differ from the original soucetree by a makefile and maybe some minor ps2-specific tweaks, so that anybody should only be able to pull any source out of the cvs and simply do "make; make install" in order to have the port compiled and installed to the location everybody intends it to be.

As a side note, please see http://cvs.ps2dev.org/ps2sdk-ports/READ ... cvs-markup

Maybe $PS2DEV/ports is the wrong place to install the stuff. It needlessly entangles the ps2sdk-ports source tree and the new, environment-cleaning place to install libraries based on ps2sdk.

After all, what good is it if every lib that isn't a thin port from another platform gets its own environment variable still? That only leaves the problem half-solved.

I prefer $PS2SDK/ports, personally.

Neil Stevens wrote:Maybe $PS2DEV/ports is the wrong place to install the stuff. It needlessly entangles the ps2sdk-ports source tree and the new, environment-cleaning place to install libraries based on ps2sdk.

After all, what good is it if every lib that isn't a thin port from another platform gets its own environment variable still? That only leaves the problem half-solved.

Well, there are two mandatory environment variables: $PS2DEV and $PS2SDK. Actually, here is my environment: Moreover, what else would the /usr/local/ps2dev do ? :P Anyway, I think this is more Drakonite's point, not mine ;) Please discuss.

If you're going to call it ps2sdk-ports, logically you'd find them linked to ps2sdk somehow. Thus, $PS2SDK/ports. The default makefiles for ps2sdk can include links into this directory so things can work more-or-less automatically for the user.

$PS2DEV holds things like the ee/iop/dvp toolchains, other projects that are not ports like gsKit, or stuff that can't be put into ps2sdk for reasons of licensing like ps2eth.

What I mean is, whatever location is picked as the new install prefix for these port libraries, shouldn't be limited *just* to things that are ports of existing libraries. If it is, then things like gsKit will still need their *own* environment variables.

That's why I think it'd be a good idea to make the new install place be named something different from $PS2DEV/ports. Maybe $PS2DEV/usr or $PS2DEV/local or $PS2DEV/software. Anything that doesn't exclude new libraries like gsKit.

Oh, and don't take any of my posts as taking a side in the PS2DEV vs PS2SDK discussion.

Aah, sorry. Okay, so. We have a point from ooPo who votes for $PS2SDK/ports and one from Neil Stevens for $PS2DEV/{local,usr,softwares} (or any better idea rather than 'ports')

Well, imho, "local" or "usr" shouldn't be used: they are names which should be found elsewhere... now, if we say we are trying to build a really 'huge' environment, maybe we can feel like beeing allowed to use 'local' here :)

Heh, I go off on a week-long vacation (with only occasional net access) and a topic hot in my mind comes up.

I don't know how related this is, but the idea of even in PS2SDK that everything for includes, libs, and bins, are scattered across so many directories just makes the whole makefile issue so messy. I theory a person should be able to construct a simplistic command line for gcc with minimal flags and go to town on compilation.

Is this matter intended to be solved here ? Or am I the only one who thinks its a problem ?

As for this and other things, what the ps2sdk and the dev community needs is a "build architecture" board that can vote what to do when things like this come up.

Just my two cents.

Now, back to changing diapers...

Sorry ? The ps2sdk doesn't get so scattered once installed....

pixel wrote:Sorry ? The ps2sdk doesn't get so scattered once installed....

Code: Select all

./common
./common/include
./common/include/sys
./bin
./iop
./iop/include
./iop/irx
./iop/lib
./ee
./ee/include
./ee/lib
./ee/bin
./ee/startup

Thanks for making my point for me :) That structure has a vastly higher complexity to it than the development environments that exist on most full blown UNIX OS's. :)

I think I am going to work on an experimental version of the SDK that tries to avoid that kind of scattering. I am going to try to put everything for all PS2 architectures into a single directory structure:

I will install a single compiler toolchain to be used for both arches. Backwards compatibility to the ee-gcc/iop-gcc type prefixes can be maintained by aliases that specify the architecture wherever possible.

Makefiles can be simplified. Instead of having makefile.global[ee|iop].somethingor_other scattered at various subdirs, the command line can be simplified:

ps2-gcc -arch=<ee|iop> source.c
(I will try to have -I/.../ps2dsk/include and -L/.../ps2sdk/lib assumed as default)

In the source files, includes are easier to specify:
<common_type_include.h>
<sys/blah.h>
<ee/blah.h>
<iop/blah.h>
<sbv/blah.h>
<gskit/blah.h>
<some_package/blah.h>

Now...this is going to be alot of work, and who knows if I would complete this any more likely than anything else I started.

But what is my motivation ?

In short, in my mind, the SDK structure has greater complexity than most full-blown OS's. For example, just about any UNIX OS has the core system and dev related files rooted under /usr/[bin|include|lib|...]. Just type "make" and go, or for simple stuff, just type "gcc source.c" and go. My ideal situation is to get as close to that as possible for PS2SDK.

For me, the motivating factors started with includes, I read them alot but find myself going back in forth between several directories in one sitting.

When setting up makefiles, or modifying the sample makefiles, so much of the stuff the makefile looks for is scattered up multiple directories levels in .global makefiles, and a bazillion -L and -I gets set. So, the effort to startup anything new or modified existing gets annoying for me.

All in all, I believe the current structure of the SDK has a high complexity and time cost to startup and maintain. IMHO, for what its worth, since I haven't done much with it anyway as everyone well knows :)

Like I said, I will make this an experimental effort, and make releases for others to try. I wouldn't suggest it to try to replace the current SDK, since that is the one everyone else intends to use and has been developing for forever. And most importantly, most people probably don't mind the existing setup. So my experimental effort will be for my own benefit, as well as anyone else who is interested, but can also serve as a proof-of-concept if it can reach the goals.

Now, I am not suggesting the ideal I intend to strive for is so easy that there won't be conflicts along the way with certain files of similar/same names for EE and IOP arches, and resolving those in logical ways while staying true to the core premise of the effort. I merely outlined a goal and will meet those problems along the way and hopefully resolve them.

Anyhow, those are my thoughts. Comments anyone ? *bracing for backlash* ;)

Its not as complex as you make it out to be... the EE and IOP really are separate targets. Mixing them together may not be the best idea.

If you really really want to simplify it, how about a separate eesdk/iopsdk? Perhaps with two different toolchain scripts? Most users would only need the EE stuff anyway and could ignore the rest.

As for the complexity of the stuff you pass to gcc... isn't there a spec file of some sort that can be modified to put default options into?

ooPo wrote:Its not as complex as you make it out to be... the EE and IOP really are separate targets. Mixing them together may not be the best idea.

If you really really want to simplify it, how about a separate eesdk/iopsdk? Perhaps with two different toolchain scripts? Most users would only need the EE stuff anyway and could ignore the rest.

As for the complexity of the stuff you pass to gcc... isn't there a spec file of some sort that can be modified to put default options into?

The argument against that is that, why have two separate SDK's if there is no reason to ? :)

As for two separate SDK's, what if a project has both EE and IOP components ? Wouldn't it be great if a single Makefile can effortlessly handle both from the same environment and toolchain, just by flipping switches ? Granted you can go thru Makefile heroics when using separate SDK's, but...

I guess I don't buy the idea that two separate arch targets requires the amount of separation that exists in the SDK, given that some overlap exists, and the two architecturally work in tandam on the target machine. Granted, perhaps most people will use EE, but projects I have been wanting to do for awhile will exercise both EE and IOP, and surely I am not the only one ?

One more thing,

Anyhow, really more out of my own interest, I am proposing an ideal of what a build system SHOULD look like. I still want to emphasize that the current setup is more complex than most OS's, and that it doesn't at all need to be that way.

There are millions of ways of making thing work for sure, but I do have a sense that the current SDK doesn't have a true guiding structure, architecture, and vision behind it. Over time, I think it can become messy.

Again, since it works for most people, especially the people, who do the bulk of the PS2DEV development, I am willing to accept than I am just a crank making a fuss :)

But hopefully enuf people are interested to make a change to get a good consensual blueprint for what the build environment should look like. My own outline was for my own purposes if I had to go off on my own due to lack of interest on anyone elses part.

I'd like to see a single unified SDK that handled both, but I think separate SDKs are probably more realistic. As to projects which use both, just structure it like this:

/project/iop/
/project/ee/

Then have the root level makefile call the makefile in the iop and ee directories. Keeping the two separate should also make it easier for beginners to keep track of what is doing what.

I was trying to illustrate how silly the merging idea was by taking the opposite side of the debate. :)

We already have two separate sdks currently in ps2sdk, with a place for the overlap to be merged called 'common'.

The build system is still wonky, I agree. It could use a lot of cleaning up and/or friendly-upping. But really the whole of ps2sdk is generally well laid out already...

Okay, so.

There's something which seems to be missing in your minds right now. In every compiler toolchain, you have a "canonical structure". That is, "in the directory where the distribution resides", you have a bunch of directories like "bin", "include", "lib", "etc", "share", etc...

So, now: EE and IOP are for me two different "OS". We are having here two different kernels, two different collection of syscalls, two quite different ways to access the hardware, two radically different way of compiling, two different ABI, etc. So, for me, it seems completely normal that the both processors are like two different targets. For example, under my unix system, I have my native toolchain under /usr, and my mingw32 toolchain under /usr/i586-mingw32msvc.

gorim wrote: In short, in my mind, the SDK structure has greater complexity than most full-blown OS's. For example, just about any UNIX OS has the core system and dev related files rooted under /usr/[bin|include|lib|...]. Just type "make" and go, or for simple stuff, just type "gcc source.c" and go. My ideal situation is to get as close to that as possible for PS2SDK.

For me, the motivating factors started with includes, I read them alot but find myself going back in forth between several directories in one sitting.

When setting up makefiles, or modifying the sample makefiles, so much of the stuff the makefile looks for is scattered up multiple directories levels in .global makefiles, and a bazillion -L and -I gets set. So, the effort to startup anything new or modified existing gets annoying for me.

The actual problem you're pointing at here is NOT a directory tree structural problem. It's simply a lack of effort of me and probably other people to not define correctly the paths in the spec files of ee-gcc and iop-gcc. Have a look:
You can notice that the base path of the ee-gcc is /usr/local/ps2dev/ee and not /usr/local/ps2dev/ps2sdk/ee (derivated from the --prefix at compilation time). Same goes with iop-gcc which base path is /usr/local/ps2dev/iop and not /usr/local/ps2dev/ps2sdk/iop, still due to the --prefix

The workaround for that would be to fix the spec file at compilation time of gcc so that it adds -I $PREFIX/../ps2sdk/$TARGET/include to *cpp and -L $PREFIX/../ps2sdk/$TARGET/lib to *link (and maybe the common directory too). But that would mean that we completely fix the PS2SDK directory during the compilation of the toolchain, adding more rigidity to the environment. AND it would mean we are tying the compilers and the PS2SDK, which are two completely separate products right now. That would get rid of any env variable though, and indeed simplify makefiles. And since we are shipping both the compilers and the SDK, the tie would make sense. So my preference would go at 60% for such an addition in the spec files at compilation time.

But, once again, this is not related to the directory tree: IMHO it utterly makes sense to have two different distribution trees, since we have two different compilers with two different behaviors.

So this was only a side note, since I consider "makefile complexity" not to be a true argument for the debate, because it can be fixed by our lazy ass in the compiler :P I don't think the actual ps2sdk's structure should be changed, and I wouldn't vote for it anyway. Drakonite's point is rather about where to put optionnal packages that lots of people are starting to provide, because it would tend in A Big Mess (tm) soon if we are not trying to provide a (simple) unified policy for these new packages :) So, please recentrate the debate ;)

pixel wrote:Drakonite's point is rather about where to put optionnal packages that lots of people are starting to provide, because it would tend in A Big Mess (tm) soon if we are not trying to provide a (simple) unified policy for these new packages :) So, please recentrate the debate ;)

Well, my concerns were two, one of which dealt with structure, and a related issue along the same lines as Drakonites which dealt with additional packages. I honestly wasn't expecting people to be terribly impressed by the first matter I talked about.

Is recentrate a word ? If so, it is fine by me with doing merely that. ;)

P.S. I am finally glad I am not the only one making long posts around here. In fact, given the crap Drakonite gave me last night, it seems his initial posting was quite an extended thriller itself. ;)

Why can't we just all get along and just play games?!

ooPo wrote:Why can't we just all get along and just play games?!

Because you some of us need more games!

Drakonite wrote:Because you some of us need more games!

You some of us should send me more games! :)