Unofficial LaunchELF related ps2sdk changes

Post by **Herben** » Sun Feb 04, 2007 2:22 pm

As I've said before, ps2ftpd uses hacks to get the device list. I've committed a modified version of ps2ftpd which does a more intelligent method of getting the device list which should work with all versions of iomanX or simple ioman.

EP · Post by EP » Sun Feb 04, 2007 4:41 pm

Herben wrote:As I've said before, ps2ftpd uses hacks to get the device list. I've committed a modified version of ps2ftpd which does a more intelligent method of getting the device list which should work with all versions of iomanX or simple ioman.

Ps2ftpd is not the problem here though. I tried your ps2ftpd changes out and same result using the standard iomanX build.

Two issues I have experienced: The inability to access mass: from within uLE after loading other IOP modules. A problem with ps2netfs that causes all directories and files on mc to have get messed up names in the listing. Here is an example of the latter issue:

Code: Select all

fsclient -h ps2 dir mc0&#58;

 &#91;Contents of mc0&#58;&#93;

  -rwxrwxrwx         61 01-10-2007 16&#58;17&#58;44                 &#9617;&#9644;&#9786;
  -rw-rw-rw-          0 02-23-2006 12&#58;45&#58;32                 &#9617;&#9644;&#9786;
  -rwxrwxrwx          8 02-04-2007 15&#58;33&#58;33                 &#9617;&#9644;&#9786;
  -rwxrwxrwx          5 11-21-2004 12&#58;55&#58;03                 &#9617;&#9644;&#9786;
  -rwxrwxrwx          5 03-17-2004 12&#58;56&#58;00                 &#9617;&#9644;&#9786;
  -rwxrwxrwx         16 06-03-2004 05&#58;51&#58;43                 &#9617;&#9644;&#9786;
  -rwxrwxrwx          5 04-23-2004 11&#58;28&#58;53                 &#9617;&#9644;&#9786;
  -rwxrwxrwx          7 05-05-2004 05&#58;57&#58;57                 &#9617;&#9644;&#9786;
  -rwxrwxrwx          6 01-30-2007 05&#58;22&#58;17                 &#9617;&#9644;&#9786;

Also it's important to note that ps2ftpd functionality is loosely based on ps2netfs so whatever changes are needed for one are most likely needed by the other.;)

dlanor · Post by **dlanor** » Sun Feb 04, 2007 9:23 pm

Herben wrote:As I've said before, ps2ftpd uses hacks to get the device list. I've committed a modified version of ps2ftpd which does a more intelligent method of getting the device list which should work with all versions of iomanX or simple ioman.

Please note that most tests I reported on in my last post, using the iomanx module you sent me, were performed without having launched ps2ftpd at all. So any hacks or bugs which the ps2ftpd code may contain did not have any influence on those tests.

I did test ps2ftpd as well, just to check that it worked consistently with previous results, but I did not have that server running when I performed the main testing. And when I say 'not running' I do mean that this module was never loaded to IOP RAM in those test sessions.

Also note that the problem never was to make ps2ftpd work. It already does work, even when the bug has struck against normal uLE browsing.

The mass: bug I reported on occurs in simple browsing of mass: in uLE, whenever either of the HOST or HDD drivers have been loaded.

The host: bug I reported on occurs in all usage with the iomanx module you sent me, although the erroneous filenames shown depend on what other drivers have been loaded, as described in my previous post.

I believe that the only connection of these problems to ps2ftpd is that the same PS2Net command which launches ps2ftpd in uLE also loads the HDD driver before starting the server. It's apparently the loading of that driver (or any other) which triggers the mass: bug. Unfortunately this caused some early misunderstandings about exactly what it was that triggered this bug, making us believe that the FTP server was responsible, although that was not the case.

Best regards: dlanor

dlanor · Post by **dlanor** » Sun Feb 04, 2007 9:57 pm

EP wrote:A problem with ps2netfs that causes all directories and files on mc to have get messed up names in the listing.

That problem is quite likely related to the problem of the messed up names in host: listings, which I reported on after my tests with the experimental iomanx Herben sent me.

Also it's important to note that ps2ftpd functionality is loosely based on ps2netfs so whatever changes are needed for one are most likely needed by the other.;)

Good point, and we probably need to check out how this relates to the host module as well.

But that still doesn't give any clues about why the mass: driver fails as it does, even if the only bug 'triggering' performed is to load the HDD drivers.

After all, these are the standard HDD drivers of PS2SDK (our own mods shouldn't affect driver loading or init), while the mass: driver is Herben's own usbhdfsd... The whole idea of 'hacks' ruining the functionality of perfectly normal 'fioXxx' calls seems a bit odd, to say the least, all things considered.

I'm rather leaning towards the idea that the extensive redirection of all ioman function vectors, while still using the internal ioman functions (which we must), is in itself incompatible with some internal usage of those function vectors inside the ioman module.

Best regards: dlanor

Post by **Herben** » Sun Feb 04, 2007 11:39 pm

I wonder if the whole thing isn't related to redirecting ioman dread to iomanX dread. I'll have a look at the code later today, perhaps I'll need to make a legacy_dread function for ioman dread.

Let me explain briefly what I mean in regard to the hacks. ps2ftpd, ps2netfs, devfs and any others which do similar things to get the device list assume that iomanX has its device list at the start of the data section. This is not the case with my modified version and I feel that it's not really a smart thing to do. That's why I added the GetDeviceList export to iomanX instead. However, since those modules are designed to be used with or without iomanX, I have to do another type of hack to call GetDeviceList without declaring the import in imports.lst, otherwise loading of the module will fail if iomanX isn't present. I do feel that it's safer to do the hack that I use, which is to assume that if iomanX .text section + 4 == 0x03E00008(which is "jr $ra") that iomanX has GetDeviceList() located there which can then be called. It's much easier to maintain GetDeviceList as being at the start of the .text section than to keep the device list at the start of the .data section.

All of this is why I was saying in the "PS2SDK Enhancements?" thread that we need a standard method of doing these hacks. Some day I'll get around to making iopmgr into a proper module to do that and likely I'll also provide inline functions in the iopmgr.h so that you can use only what's needed without requiring the inclusion of iopmgr.irx in a project.

I'll let you guys know what I find out about dread()...

Post by **Herben** » Mon Feb 05, 2007 12:09 am

sure enough, I should have thought about that. I emailed another iomanX. I still need to add iomanX->ioman stat conversion so legacy ioman dreads can be used on iomanX devices but this should resolve your dread issues.

dlanor · Post by **dlanor** » Mon Feb 05, 2007 1:39 am

Herben wrote:I wonder if the whole thing isn't related to redirecting ioman dread to iomanX dread.

Well, it certainly does seem to be involved anyway, though it remains to be seen if that is the whole story.

The main difference of the two function prototypes is that the array elements for directory data contain fio_dirent_t structs for fioDread. while the array used for fileXioDread contains iox_dirent_t structs. Each struct of either sort contains a substruct, followed by the object name, but that name will be at different offsets for fioDread and fileXioDread, due to the different substructs used, which are fio_stat_t and iox_stat_t. The latter begins with data identical to the former, but then continues on with several additional elements (24 bytes larger in all).

So here we have a potential problem involving an offset to the object name, though other object data is compatibly stored. This could very well explain those problems that seem to affect only the object names in directory listings. But it is doubtful if that also explains the total loss of response from mass: in some of our tests. I really don't think so.

I'll have a look at the code later today, perhaps I'll need to make a legacy_dread function for ioman dread.

I think that's a good idea. Even if it doesn't cure all problems completely, it may allow more revealing tests.

Let me explain briefly what I mean in regard to the hacks. ps2ftpd, ps2netfs, devfs and any others which do similar things to get the device list assume that iomanX has its device list at the start of the data section. This is not the case with my modified version and I feel that it's not really a smart thing to do.

I agree that it's not a smart thing to rely on such things for new designs, but it may still be best to try moving that data for backwards compatibility to existing programs and their embedded modules.

This is a very serious concern for uLE, since most programs we launch will inherit the modules used by uLE, and if those programs want to load some IOP modules of their own (not conflicting ones of course), then they may need the 'old' iomanX methods.

----- snip ----- re: your GetDeviceList function
I agree that it's better to have a standardized access function than directly accessing the data section, but backwards compatibility remains an issue to consider.

However, I don't see this as connected to our main problem at all, since the device list access methods don't seem affected by the state of the FULL_IOMAN option, and undefining that switch does cure our problem.

It's much easier to maintain GetDeviceList as being at the start of the .text section than to keep the device list at the start of the .data section.

Perhaps, but it might be worth the effort of doing both, to maintain maximum backwards compatibility, while still allowing new applications to use a more 'standardized' access method.

I'll let you guys know what I find out about dread()...

Please do. I think that experiments with its redirection could reveal more clues, even if they don't provide a total cure for everything.

Btw:
I just had another look into the iomanX sources and noted that AddDrv and DelDrv are performed in entirely different ways depending on the state of FULL_IOMAN. What struck me as most surprising is that you never call the underlying functions of ioman if the FULL_IOMAN flag is set, relying entirely on the assumption that only the copy of the device list maintained by iomanX will ever be used.

This is naturally correct for devices designed for iomanX standards, so that all programs accessing them will be using fileXioXxx functions, but I find it dubious for devices designed for ioman standards, which will probably be accessed through fioXxx functions.

Any function of the ioman module that refers to a subfunction through its own device list, for any purpose whatsoever, may then miss a device that was added only to the list maintained by iomanX. And that could explain some of the symptoms we see. It even explains why that FTP server is unaffected, as it doesn't call such functions through either ioman or iomanX, but uses direct driver calls via pointers from the driver structs it found through iomanX. (So its device list is always complete.)

I have a strong hunch that improved legacy functions for AddDrv and DelDrv could fix this.

Best regards: dlanor

dlanor · Post by **dlanor** » Mon Feb 05, 2007 1:53 am

Herben wrote:sure enough, I should have thought about that. I emailed another iomanX.

I have it now and will proceed with its testing directly.

I still need to add iomanX->ioman stat conversion so legacy ioman dreads can be used on iomanX devices

I see, but the lack of such support will not affect my tests, as uLE itself never mixes these standards either way.

but this should resolve your dread issues.

Good! That should at least eliminate the weird host: listings I got earlier. And then we can concentrate on the 'missing mass:' bug, unless that too is now cured (though I think that's a bit much to hope for).

Best regards: dlanor

Post by **Herben** » Mon Feb 05, 2007 2:10 am

actually, I'm pretty sure the mass problem is directly related to this.

dlanor · Post by **dlanor** » Mon Feb 05, 2007 3:09 am

Herben wrote:actually, I'm pretty sure the mass problem is directly related to this.

Maybe it is, but the new iomanX module you sent did not eliminate any of the problems, although it did modify the symptoms.

The mass: device is still 'lost' for uLE browsing whenever HDD or HOST drivers are loaded, but this shows itself in an even worse way now. Previously it merely resulted in an empty directory listing. It was then still possible to 'back out' and browse some other device instead. But with the new iomanX module you sent, such an attempt to browse mass: crashes the console.

Browsing mass: without HDD or HOST drivers loaded works fine though, just as it always did.

The error with host: has also worsened, such that it now crashes whenever attempting to request a directory listing through fioDread.

Note that this buggy behaviour still doesn't affect the raw file transfer functions. I can still see the pseudo directory list representing the "elflist.txt" contents, as that doesn't use fioDread, but any attempt to browse those paths results in the console crashing, since that is when fioDread is used.

I'm afraid something must have gone badly wrong with your patching of fioDread... :(

Best regards: dlanor

EP · Post by EP » Mon Feb 05, 2007 8:52 am

@Herben if you want to see some of this first hand, here's one way of doing it:

1. Build ps2link with the current iomanX then execute it.
2. Run ps2client or inLink and load up the following IOP modules: ps2netfs.irx, usbd.irx, and usbhdfsd.irx.
3. You should see the following with 'fsclient -h ps2 devlist'

Code: Select all

 &#91;Active Devices&#93;

  rom    - Onboard rom device.
  cdrom  - Standard cd/dvd device. &#40;cdrom&#58;&#41;
  dev9x  - Blah blah blah.

 &#91;3 Devices Found&#93;

4. Now load the ps2ftpd.irx module and you'll see that mass is really there even though fsclient can't see it through ps2netfs.

Note: There is an issue currently with accessing memory cards in PS2Link v1.51, as the mc driver is lost or not loaded, but I'm not going to get into that here. Anyway, the main thing is that you should be able to see how ps2netfs seems to miss mass0: where as ps2ftpd sees it just fine. This is probably what is going awry with uLE and how it handles mass: access. This should hopefully help you see what's happening with iomanX a little better.:)

Post by **jbit** » Mon Feb 05, 2007 9:05 am

Note: There is an issue currently with accessing memory cards in PS2Link v1.51, as the mc driver is lost or not loaded, but I'm not going to get into that here.

New ps2links no longer load sio2man and mcman since they conflict with xsio2man and xmcman. (Which causes absolute hell if you want to use a multitap, for example).
If you load sio2man and mcman memory card access works fine (I've used it here many times :)
It is assumed if an application needs these it will load them itself. (Since you shouldn't really rely on certain modules being loaded anyway)

EP · Post by EP » Mon Feb 05, 2007 11:19 am

jbit wrote:New ps2links no longer load sio2man and mcman since they conflict with xsio2man and xmcman. (Which causes absolute hell if you want to use a multitap, for example).
If you load sio2man and mcman memory card access works fine (I've used it here many times :)
It is assumed if an application needs these it will load them itself. (Since you shouldn't really rely on certain modules being loaded anyway)

Oh, ok. I loaded them up through ps2client and everything was fine.:) May take a while to get use to but I think a simple batch file should do the trick for my usual scenarios.

Post by **Herben** » Tue Feb 06, 2007 4:09 pm

ok, it's going to be a while before I have more time to work on this..

fyi, I believe you can add SIOMAN and MCMAN to the EXTRA.CNF file for ps2link to have it load them automatically. I've never done this, myself.

dlanor · Post by **dlanor** » Wed Feb 07, 2007 12:05 am

Herben wrote:ok, it's going to be a while before I have more time to work on this..

I see. In that case we'll just have to go with the solution of disabling the "FULL_IOMAN" switch in the iomanX module used by uLE, at least for the time being. Please let us know when you have time to work on this stuff again.

fyi, I believe you can add SIOMAN and MCMAN to the EXTRA.CNF file for ps2link to have it load them automatically. I've never done this, myself.

Thanks for the info. We'll try that in future testing then.

Best regards: dlanor