Is it possible to survive SifIopReset?

606u · Post by **606u** » Tue Aug 17, 2004 8:33 pm

Or the long question: is there a way to write/load an IOP module that would continue to run even after SifIopReset? Or maybe there is a way to "capture" SifIopReset call and to mimic its behaviour or even call the real one and load IOP again afterwards...?

Guest · Post by **Guest** » Wed Aug 18, 2004 12:00 am

Well there are many approaches one can take that are a bit involved,
and to various degrees of success. It really all depends on what you
are trying to achieve. If you provide a little more details it will be much
more helpful in deciding how to handle this situation.

Gorim

606u · Post by **606u** » Wed Aug 18, 2004 7:07 am

I saw fakehost IOP that is located in the CVS so I think of a similar one that would install a fake cdrom driver (oposed to fake host: driver) translating file system calls to the USB drive. And I think that it would be possible to deceive PS2 emulators floating around to use flash drive instead of CD-ROM (I don't have a mod, so I have troubles booting those on my console).

However, homebrew is usually making IOP reset at the begining, hence IOP installed would be killed in an instant.

Now... knowing the long story, are you still going to help? :-)

PS: I do not flatter myself of thinking I could possibly make something like put-the-dirty-word-here.

Post by **ooPo** » Wed Aug 18, 2004 10:19 am

Here's an interesting question. What exactly are you hoping to do?

If you're just interested in seeing if its possible, may I suggest doing something like a fake memorycard driver instead? People have been always poking around for how to use those crazy usb sticks as a place to save games...

606u · Post by **606u** » Wed Aug 18, 2004 6:16 pm

That is a good idea. However, AFAIK USB mass storage driver is currently read-only.

Ok, then, I'm going to try to set up a new general... "bridge", based on the fakehost, that would allow you to bridge one file system to another, depending on the run-time arguments.

Post by **Drakonite** » Wed Aug 18, 2004 7:02 pm

606u wrote:That is a good idea. However, AFAIK USB mass storage driver is currently read-only.

IMO one of the best things you could work on right now is helping to fix that.

Heh, and god forbid someone works on an enduser app or homebrew game..

606u · Post by **606u** » Wed Aug 18, 2004 11:48 pm

I'll consider helping for USB mass storage write support, however, that is not my strong field. I have too many black holes about USB stuff and not being able to use a debugger is really pain in the a**.

However, I've managed to build a preliminary file-system bridge which is available here.

It is not a big thing - you could map memory card to the CD-ROM or vice-versa, or create an alias for an existing device (like accessing `blah:BEDATA-SYSTEM/BOOT.ELF' instead of `mc0:BEDATA-SYSTEM/BOOT.ELF').

I have some questions, though:
1. For some reason, I'm unable to pass _start arguments (via argv) to the IOP - is it because I'm using gcc-3.2.2?
2. Why driver's init function is called twice (and uninit only once)?
3. Should I attempt to close any files on uninit, and is it possible uninit to be called while there are still files opened?
4. My ioman.h declares open, close, read, write and lseek only; where are the other functions (like directory operations and etc.)?
5. Is there an "errno" I should care about in case of an error?
6. Should I worry about concurrency (that is use synchronization objects, like semahores) when working with shared data?
7. Why AddDrv returns ok, but file system driver does not work when registered against IOMANX (it works fine with IOMAN)?
8. What is the approximate stack size available? How much is the safe size of stack buffers?

That's all so far, I think. There are some more changes that I should make - for example it looks like cdrom driver requires `\'-slashes and ';1'-suffixes (that should be set/removed when remapping file names) and I have not even tried HDD support. Maybe in the next few days...

Regards, b.

Post by **Herben** » Thu Aug 19, 2004 7:28 am

Your FS "init" function getting called twice is due to craziness in the design of IOMANX. I gave t0mb0la a fix for this, but I guess it never made it into the CVS.. Basically what happens is that IOMANX calls the init function then passes control to normal IOMAN which ALSO calls the init function...

606u · Post by **606u** » Sat Aug 21, 2004 7:39 pm

Just to remind you there are a lot of questions unanswered here and to ask a new one: if I decide to work on writing support for FAT file systems, what is the typical amount of memory permitted to file system drivers (that is dynamically allocated, to use as a cache, etc.)? FAT12 and FAT16 are probably fine, but, FAT32 could grow quite large (with bigger disk sizes). Although it will probably work, it is going to be quite slow and with a lot of redundant disk operations.

Guest · Post by **Guest** » Sat Aug 21, 2004 8:13 pm

606u wrote:Just to remind you there are a lot of questions unanswered here and to ask a new one: if I decide to work on writing support for FAT file systems, what is the typical amount of memory permitted to file system drivers (that is dynamically allocated, to use as a cache, etc.)? FAT12 and FAT16 are probably fine, but, FAT32 could grow quite large (with bigger disk sizes). Although it will probably work, it is going to be quite slow and with a lot of redundant disk operations.

I don't have much specific answers, just my basic thoughts. Others might
have more to offer.

1. For things that use FAT or FAT16, there is probably no real issue. If
you are talking floppy disks or USB flash storage, then the media will be
so slow that doing lots of caching won't be terribly useful. You *do* have
a maximum of 2MB in the IOP, but as you already realize, you can't use
it all for caching.

2. FAT32, well, you may just decide to live with the slowness
given the constraints of the system.

3. You might start with the approach of doing FAT first, and see what
lessons you learn along the way of developing it. Then once you have
your feet wet and learned alot along the way, you might be better
prepared to tackle the issues of FAT16 and/or32.

My 2 cents, no real useful information, just ideas on approach.

Gorim

Post by **pixel** » Sat Aug 21, 2004 8:46 pm

Fat12/16/32 are only different in the cluster and pointer sizes. That's only meant to handle different drive sizes. That is, 12 bits, 16 bits and 32 bits for a FAT entry, which can lead to 4096, 65536, or 2^32 clusters reachable on the disk.

Anyway: if vfat filesystem is not really fast, nor reliable, it's quite easy to handle. (compared to ntfs or ext2 that is...)

606u · Post by **606u** » Mon Aug 23, 2004 2:45 am

Read-only is almost done. Anyone got a concept how the access to the file-systems sould be done? What I mean is, that with a mass-storage device you could have the following hierarchy:
1st level: list of devices (could have one device, more than one device, several LUNs if that is how USB is organized);
2nd level: list of partitions for a particular device (it is not necessarily a flash memory; I've an external USB box with a hard-disk and 10 test FAT partitions already);
3rd level: a mounted partition for a particular device.

As I see it, maybe it would be best if one could enumerate USB mass-storage devices (I cannot help here - that is low-level stuff), select a device, enumerate device partitions, select a partition, mount a partition and use mounted partition via IOMAN(X) (in the same way as mc0: or cdrom0:).

For example, something like (pseudocode):

Code: Select all

dopen &#40;"usb_mass&#58;/"&#41;;
dread &#40;...&#41;; // <= get next device
dclose &#40;&#41;;

mount &#40;"usb_mass&#58;/0" to "usb_dev1"&#41;;
dopen &#40;"usb_dev1&#58;/"&#41;;
dread &#40;...&#41;; // <= get next partition
dclose &#40;&#41;;

mount &#40;"usb_dev1&#58;/1" to "C"&#41;;
open &#40;"C&#58;\blah.txt", O_RDONLY&#41;; // ha-ha-ha
read &#40;...&#41;;
close &#40;&#41;;

unmount &#40;"C"&#41;;

unmount &#40;"usb_dev1"&#41;;

How does it sound? It is a little bit clumsy, but it is more-flexible...

Maybe it should be integrated with the devfs (I'm not that familiar with PS2's devfs)...?

Any ideas are welcome.

Regards, b.

ole · Post by **ole** » Wed Aug 25, 2004 7:42 pm

usb_mass driver is currently read only beacause I didn't have guts (to accidentaly screw up your data) to do it. I can add support for writing sectors on the device, but doing correct write support for FATXX is much more challenging task (to do it from scratch). Another way is to port some existing (and already tested) vfat driver and replace the current one.

About the write speed:
I think that there would be some slown down in write opeartions comparing the read operations. The reason is that FAT cluster chain have to be alocated (and eventally written) when each single cluster is written. The cluster allocation involves finding the suitable place in the FAT table (that could be larger than 2Mb) and updating the FAT sector. Another issue of the usb write operations is that anyone can disconnect the device during write operation and that could lead to data loss or even filesystem corruption.

As for the mounting:
The hierarchy you wrote is correct:
1) device
2) LUN on device
3) partition on LUN (primary partition, extended partition)
No level of this hierarchy is currently handled (only one device is handled, first LUN on the device is used - LUN number 0, and finally first suitable partition of the LUN is used). It's a question how important is to do it correctly since 90% of the orinary users have only one device with single partition... (maybe I'm wrong :-).

On the other hand I think that mounting (assigning the numbers) should be done automaticaly when device is plugged in. Manual mounting in program is imho too complicated (confusing) for ordinary user. I don't know how many numbers are available (is it possible to address mass32:/ for example ?) for device addressing. Originally the driver was written for ps2lib which AFAIK couldn't recognize the drive numbers. I've done the simplest (less painfull) conversion to ps2sdk, but the support for drive numbers is still missing in the driver (and supported in ps2sdk).

606u · Post by **606u** » Thu Aug 26, 2004 7:36 am

I do not underrate what you've done. However, it is easier for me to work from scratch. At least for the FAT file system. So far I've got read-only access including multiple devices and partitions support, directory browsing and file reading. That is working on a Windows box using RAW dumps (in a similar way like you) and using RAW device access API (CreateFile ("\\.\PhysicalDrive1", ...) ).

I've some writing support already - file deletion ;-) which is not a big deal, but now you can intentionately screw your data :-) and scandisk is happy about it. But you're right - it is not a trivial task because there are too many limitations - mostly about the tiny amount of memory that could be used. I use not very clever caching engine working with a pool of 128 sectors (little more than 64KBytes). The performance issues are also caused by the fact, that in order disk I/O to be faster, one should read or write several sectors at a time - 16 or 32 is fine. However, partitioning the already small enough cache to 32-sector parts would cause quite misses and waits.

PS: IMO disconnecting the device during write operation could cause corruption on any OS if FAT is being used.

ole · Post by **ole** » Fri Aug 27, 2004 5:31 pm

IMO disconnecting the device during write operation could cause corruption on any OS if FAT is being used.

You are right :) but my point is that nobody is (most likely) going to unplug hdd when running for example. So prgrammer can enable write cache etc. If programmer knows that unplugging may occur at any time the driver have to be more robust (all fats should be updated immediatelly etc...)

Anaway, feel free to take whatever you need from the current driver to make a better one and let us know about the write support progress.

606u · Post by **606u** » Mon Aug 30, 2004 6:48 am

Ok... the current snapshot could be found on http://606u.dir.bg/ps2/fat/ps2_fat-20040829.tgz . I advice you not to run it unless you know what you're doing, because the preliminary write support is there.

Snapshot *IS GOING TO WRITE* on Windows physical device 2. Since I have two physical drives, that is the third one (they are zero-based) - usually USB flash drive on an external USB HDD box. You've been warned :-)

Here is the current features list:
- supports file dumps and physical devices under Win32; device access for Linux is a trivial task; however, it does not ran it on IOP, yet;
- enumeration of FAT volumes on a physical device;
- mounting and unmounting of volumes;
- enumeration of files and directories;
- reading files contents;
- creates files and directories;
- extends and modifies files;
- removes files.

Don't expect that all this is going to work flawlessly. File writing is still far too slow and code is not fully thread-safe.

Edit: Forgot to mention, that in order to see the changes on a USB drive you should first unplug it and plug it again. Also, remember, that this one does a low-level access bypassing Windows' drivers, therefore making changes from both places without replugging the drive first will most-likely cause corruption.