Investigation of the Unicode version of sceIoDread

[Tong]

I am working on a file browser that lists the files in the memory stick. I implement using sceIoDopen followed by sceIoDread. I hope to support showing filenames of different languages, including Chinese, Korean and some other western languages apart from English.

Unfortunately, sceIoDread has serious problems:

1. It returns filenames encoded in Shift-JIS (SJIS) which is used for the Japanese only. The function fails to handle filenames with characters that can't be mapped by SJIS. This issue was discussed at http://forums.ps2dev.org/viewtopic.php? ... sceiodread

2. The broken filename obtained from sceIoDread CANNOT be used to open file with sceIoOpen. (Is it the reason for being unable to load ISOs with non-ascii/non-japanese filenames in CFW?)


There are solutions to this problem:

1. Write a custom fat driver to read Unicode filenames. This approach is used by pmplayer-advance (fat.c). However, it is slow and has low compatibility with other homebrews.

2. Use the idea from alexp (http://forums.ps2dev.org/viewtopic.php?t=8590) who use sceIoDevctl and sceIoIoctl to read Unicode filenames. But this solution is valid only for old firmware, I have confirmed it can't work on recent ones (4.xx, 5.xx).

I am crazy about this issue as I see the vsh CAN read and display these non-ascii / non-japanese filenames properly. I am sure they use sceIoDread too. I have done some sample tests to prove that Unicode version of sceIoDread really exists: (test platform: CFW 5.00 M33-6)

1. Put a image named "Hello 你好嗎" to the PICTURE folder of the memory stick. It is obvious that its name is displayed properly in vsh.


2. Open PSPLINK 3.0OE, do

Code: Select all

reset vsh
cd ms0:/PICTURE
ls

3. Load start flash0:/vsh/module/vshmain.prx to open vsh.

4. Do ls again in PSPLINK, note that the filename of the image is different!



I do some memory dump and found that the filename in step 2 is "Hello ■
好■.bmp" in SJIS. However, the one in step 4 is "Hello 你好嗎.bmp" in UTF-8!

I suspect that the vsh_module(vshmain.prx) has done something to the fatms driver to enable Unicode sceIoDread.

Can anyone help me investigate how it can do so?
Secondly, what we can do in our homebrew games to enable Unicode sceIoDread?

Btw, I also found that vsh directly pass the utf-8 filenames to sceIoOpen and sceIoDopen, are these IO functions support utf-8 too?

thanks

[TyRaNiD]

What does psplink do if you list a UTF8 filename on host? Although you might not be able to do it on Windows (you probably need Linux for that kinda trick as I make not effort to encode the filenames appropriately). That would at least narrow down to whether it is something in the guts of sceIoDread itself or in the actual ms driver.

If it is in the driver itself then about the only thing I can suggest is to look at what IOCTL/DEVCTL calls are made to it by hooking those in psplink and dumping the output, see if you can find one which is being called to set the encoding.

[Tong]

Thanks for your reply.

What does psplink do if you list a UTF8 filename on host? Although you might not be able to do it on Windows (you probably need Linux for that kinda trick as I make not effort to encode the filenames appropriately). That would at least narrow down to whether it is something in the guts of sceIoDread itself or in the actual ms driver.

Actaully, I apihooks sceIoDread and then use the SAVEMEM command of psplink to get the SceIoDirent structure returned by the function. I open the mem file with firefox and adjust the encoding settings to see what happen. This is not convenient but at least it works fine.

If it is in the driver itself then about the only thing I can suggest is to look at what IOCTL/DEVCTL calls are made to it by hooking those in psplink and dumping the output, see if you can find one which is being called to set the encoding.

Yes, I have thought of that too. But I found that sceIoDread changes to unicode BEFORE any calls to sceIoDevctl or sceIoIoctl. Perhaps these 2 functions are not related to this issue.