genuine PSP font with intraFont & PGF binary format

Discuss the development of new homebrew software, tools and libraries.

Moderators: cheriff, TyRaNiD

BenHur
Posts: 28
Joined: Sat Oct 20, 2007 5:26 pm

genuine PSP font with intraFont & PGF binary format

Post by BenHur »

Here's a simple solution to use (all of) the genuine PSP fonts:

Image

It features the stuff you see above plus: variable size glyph cache, pre-caching, fast speed (as little as 35ms initial loadtime per font; up to 100,000 chars/sec), low memory requirements (as little as 66kB per font) and no dependencies (e.g. freetype).

Download: http://www.psp-programming.com/benhur

The archive contains samples for GU and graphics.h - just look at them to see how to use intraFont and read the readme.

(If you used pgeFont by InsertWittyName (http://insomniac.0x89.org) before: you'll feel right at home since intraFont uses parts of pgeFont - but intraFont doesn't need the freetype library!)

Let me know what you think and have fun!
BenHur
Last edited by BenHur on Fri Nov 23, 2007 1:07 am, edited 1 time in total.
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

Great! This totally rocks for most homebrew. Unless you need a custom font, this completely blows FreeType out of the water. :)
hibbyware
Posts: 78
Joined: Wed Mar 28, 2007 10:29 am

Post by hibbyware »

I don't post anywhere much but this is worth one of my rare posts,

This looks really good and i will try to use it in my next project,

Keep up the good work,
Team-3GO
Posts: 17
Joined: Wed Nov 21, 2007 7:40 am
Location: PSP-Development

Post by Team-3GO »

Very nice!
"Ignorance is the night of the mind, but a night without moon and star."
moonlight
Posts: 567
Joined: Wed Oct 26, 2005 7:46 pm

Post by moonlight »

This looks awesome.
freezeburger
Posts: 3
Joined: Wed Nov 21, 2007 9:49 am

Post by freezeburger »

Sorry maybe my question is not related to this thread...., I hope Benhur can help me with this :D
I am thinking about making homebrew that works like notepad but it can input japanese character so far i know there is no any writer homebrew that support japanese, I think that I can not using native OSK to input very large of character, so I am thinking to make my own OSK for my writer homebrew. In inputing Japanese you must be able to change from hiragana to Kanji otherwise your input all will be in hiragana. I have already search from flash0 file, i suspect that PSP use flash0/dic/***.dic for transformation from hiragana to kanji, what i want to know is what format is ***.dic file? is really psp using ***.dic for japanese character transformation?

Regards,
freezeburger
User avatar
Raphael
Posts: 646
Joined: Tue Jan 17, 2006 4:54 pm
Location: Germany
Contact:

Post by Raphael »

Very nice work BenHur indeed :)

I'm working on getting this into the SDK. It was time for a pspFont library.

Question to the SDK maintainers (and others if they want to give their opinion anyway ;P):
Should pspFont library be low-level and only provide the ability to render the glyphs or strings into an image buffer (no dependency on sceGu) or have an inbuilt back-end for the rasterization too (like intraFont currently does)?

I'm more towards the first solution atm, as IMO this should be independent of the graphics setup of an application. Also it's easy to build a back-end on top at any time.

EDIT: Having pspFont only rasterize the glyphs into an image buffer would however kill the gain from a swizzled font texture and make the texture cache suffer from many texture changes, however each string would only require two vertices.
Alternatively, you could have pspFont just build a vertex buffer for the string that the user is responsible for drawing, but then again that's already close enough to a render back-end that you could just do that step further too.
Meh, now I'm not that much biased towards the first solution any more :/
Last edited by Raphael on Wed Nov 21, 2007 11:00 am, edited 1 time in total.
<Don't push the river, it flows.>
http://wordpress.fx-world.org - my devblog
http://wiki.fx-world.org - VFPU documentation wiki

Alexander Berl
User avatar
Raphael
Posts: 646
Joined: Tue Jan 17, 2006 4:54 pm
Location: Germany
Contact:

Post by Raphael »

freezeburger wrote:Sorry maybe my question is not related to this thread...., I hope Benhur can help me with this :D
I am thinking about making homebrew that works like notepad but it can input japanese character so far i know there is no any writer homebrew that support japanese, I think that I can not using native OSK to input very large of character, so I am thinking to make my own OSK for my writer homebrew. In inputing Japanese you must be able to change from hiragana to Kanji otherwise your input all will be in hiragana. I have already search from flash0 file, i suspect that PSP use flash0/dic/***.dic for transformation from hiragana to kanji, what i want to know is what format is ***.dic file? is really psp using ***.dic for japanese character transformation?

Regards,
freezeburger
Take a look here: http://forums.ps2dev.org/viewtopic.php?p=60742#60742
<Don't push the river, it flows.>
http://wordpress.fx-world.org - my devblog
http://wiki.fx-world.org - VFPU documentation wiki

Alexander Berl
StrmnNrmn
Posts: 46
Joined: Wed Feb 14, 2007 11:32 pm
Location: London, UK
Contact:

Post by StrmnNrmn »

This is really great - thanks.

There is a bug in the call to sceGuDrawArray though. It's passing in twice as many verts as it should do for GU_SPRITES:

Code: Select all

sceGuDrawArray&#40;GU_SPRITES, GU_TEXTURE_32BITF|GU_COLOR_8888|GU_VERTEX_32BITF|GU_TRANSFORM_2D, &#40;n_glyphs+n_sglyphs&#41;<<2, 0, v&#41;;
It should be:

Code: Select all

sceGuDrawArray&#40;GU_SPRITES, GU_TEXTURE_32BITF|GU_COLOR_8888|GU_VERTEX_32BITF|GU_TRANSFORM_2D, &#40;n_glyphs+n_sglyphs&#41;<<1, 0, v&#41;;
i.e. num_verts*2, not *4.
Had me scratching my head for quite some time...
freezeburger
Posts: 3
Joined: Wed Nov 21, 2007 9:49 am

Post by freezeburger »

Raphael wrote:
freezeburger wrote:Sorry maybe my question is not related to this thread...., I hope Benhur can help me with this :D
I am thinking about making homebrew that works like notepad but it can input japanese character so far i know there is no any writer homebrew that support japanese, I think that I can not using native OSK to input very large of character, so I am thinking to make my own OSK for my writer homebrew. In inputing Japanese you must be able to change from hiragana to Kanji otherwise your input all will be in hiragana. I have already search from flash0 file, i suspect that PSP use flash0/dic/***.dic for transformation from hiragana to kanji, what i want to know is what format is ***.dic file? is really psp using ***.dic for japanese character transformation?

Regards,
freezeburger
Take a look here: http://forums.ps2dev.org/viewtopic.php?p=60742#60742
thanks for answer, but I think that writer homebrew can not use native OSK for writing..... because it would be too many characters that must be put into osk textbox I think, correct me if i am wrong....
StrmnNrmn
Posts: 46
Joined: Wed Feb 14, 2007 11:32 pm
Location: London, UK
Contact:

Post by StrmnNrmn »

Raphael wrote:I'm working on getting this into the SDK. It was time for a pspFont library.
I think this would be fantastic.
Raphael wrote:Question to the SDK maintainers (and others if they want to give their opinion anyway ;P):
Should pspFont library be low-level and only provide the ability to render the glyphs or strings into an image buffer (no dependency on sceGu) or have an inbuilt back-end for the rasterization too (like intraFont currently does)?

I'm more towards the first solution atm, as IMO this should be independent of the graphics setup of an application. Also it's easy to build a back-end on top at any time.
For what it's worth, I'd prefer something along the lines of the first solution too. It would be very handy to have a low-level library that could be used to render a specified set of glyphs to an image buffer, and provide details on the relevant kerning etc in some other structure. That would let people who have more complex text rendering requirements handle rendering themselves. It would also give people more control over e.g. the location of the allocated texture memory.

It would be relatively simple to provide another layer above this to provide the same kind of rasterising functionality as intraFont currently provides, which would be ideal for most homebrew and for quickly getting projects up and running.
User avatar
Raphael
Posts: 646
Joined: Tue Jan 17, 2006 4:54 pm
Location: Germany
Contact:

Post by Raphael »

freezeburger wrote: thanks for answer, but I think that writer homebrew can not use native OSK for writing..... because it would be too many characters that must be put into osk textbox I think, correct me if i am wrong....
You can set the maximum length of the input, however, you don't need to let the user input the whole text in one OSK session. You could make it so he only writes single words with it, which you could force by checking if the user input a space and in that case insert the current word into the text buffer and reset the OSK input string.
<Don't push the river, it flows.>
http://wordpress.fx-world.org - my devblog
http://wiki.fx-world.org - VFPU documentation wiki

Alexander Berl
freezeburger
Posts: 3
Joined: Wed Nov 21, 2007 9:49 am

Post by freezeburger »

Raphael wrote:
freezeburger wrote: thanks for answer, but I think that writer homebrew can not use native OSK for writing..... because it would be too many characters that must be put into osk textbox I think, correct me if i am wrong....
You can set the maximum length of the input, however, you don't need to let the user input the whole text in one OSK session. You could make it so he only writes single words with it, which you could force by checking if the user input a space and in that case insert the current word into the text buffer and reset the OSK input string.
Thank you for enligthen me :D
Team-3GO
Posts: 17
Joined: Wed Nov 21, 2007 7:40 am
Location: PSP-Development

Post by Team-3GO »

the samples work fine on fw 1.50 but will crash at 84% on 3.52 M33 under booth kernels. maybe its caused by some mods. will try it on a clean system again asap.
"Ignorance is the night of the mind, but a night without moon and star."
ufoz
Posts: 86
Joined: Thu Nov 10, 2005 2:36 am
Location: Tokyo
Contact:

Post by ufoz »

Raphael wrote:
freezeburger wrote: thanks for answer, but I think that writer homebrew can not use native OSK for writing..... because it would be too many characters that must be put into osk textbox I think, correct me if i am wrong....
You can set the maximum length of the input, however, you don't need to let the user input the whole text in one OSK session. You could make it so he only writes single words with it, which you could force by checking if the user input a space and in that case insert the current word into the text buffer and reset the OSK input string.
There are generally no spaces in Japanese text. And typing word-for-word would be... horrible, imo.

But typing anything longer than a one-liner on an IME with so few buttons seems like a real pain in the ass in the first place
StrmnNrmn
Posts: 46
Joined: Wed Feb 14, 2007 11:32 pm
Location: London, UK
Contact:

Post by StrmnNrmn »

Another small bug:

Code: Select all

float intraFontMeasureText&#40;intraFont *font, const char *text&#41; &#123; 
    if &#40;!font&#41; return 0.0f; 
	
	int i, length = 0;
	float x = 0.0f;
	
	while &#40;&#40;text&#91;length&#93; > 0&#41; && &#40;text&#91;length&#93; != '\n'&#41;&#41; length++; //strlen until end of string or newline
	
	for&#40;i = 0; i < strlen&#40;text&#41;; i++&#41; &#123; 
Should be:

Code: Select all

for&#40;i = 0; i < length; i++&#41; &#123; 
Speaking of string lengths, it would be very handy if the rendering/measuring APIs took an explicit 'length' argument. This is very handy when formatting text (e.g. word wrapping text into a rectangle). For instance, I've made these changes locally:

Code: Select all

float intraFontPrintUCS2Ex&#40;intraFont *font, float x, float y, const unsigned short *text, int length&#41;;
float intraFontPrintEx&#40;intraFont *font, float x, float y, const char *text, int length&#41;;
float intraFontMeasureTextEx&#40;intraFont *font, const char *text, int length&#41;;
float intraFontMeasureTextUCS2Ex&#40;intraFont *font, const unsigned short *text, int length&#41;; 
I can post a patch here if anyone is interested.
BenHur
Posts: 28
Joined: Sat Oct 20, 2007 5:26 pm

Post by BenHur »

StrmnNrmn wrote:There is a bug in the call to sceGuDrawArray though. It's passing in twice as many verts as it should do for GU_SPRITES:

Code: Select all

sceGuDrawArray&#40;GU_SPRITES, GU_TEXTURE_32BITF|GU_COLOR_8888|GU_VERTEX_32BITF|GU_TRANSFORM_2D, &#40;n_glyphs+n_sglyphs&#41;<<2, 0, v&#41;;
It should be:

Code: Select all

sceGuDrawArray&#40;GU_SPRITES, GU_TEXTURE_32BITF|GU_COLOR_8888|GU_VERTEX_32BITF|GU_TRANSFORM_2D, &#40;n_glyphs+n_sglyphs&#41;<<1, 0, v&#41;;
I spent two hours trying to track down that bug (causes graphical glitches in rare cases), but just couldn't find it. It just didn't occur to me to look there... Thank you for checking the code and reporting this bug!

Your second bug (strlen(text) vs. length) is of course correct, too. Assuming a decent software developer writes 7 bugs per 1000 lines of code, there's now at least 5 bugs left :-)

Cheers, BenHur
BenHur
Posts: 28
Joined: Sat Oct 20, 2007 5:26 pm

Post by BenHur »

Team-3GO wrote:the samples work fine on fw 1.50 but will crash at 84% on 3.52 M33 under booth kernels. maybe its caused by some mods. will try it on a clean system again asap.
Since my PSP is a 3.52 M33-2, I would assume it's a moded font that causes the problem. ttf2pgf was written with little knowledge of the pgf file format. Since intraFont extracts a lot more info from the pgf files, some incorrectly set values might crash the system. I guess I have to add more error-checking and refuse to load those fonts. Is the flash0:/font/jpn0.pgf by chance an empty dummy file? If not, send it to me (email at the bottom of my page) so I can analyze it.

Cheers, BenHur
StrmnNrmn
Posts: 46
Joined: Wed Feb 14, 2007 11:32 pm
Location: London, UK
Contact:

Post by StrmnNrmn »

BenHur wrote:I spent two hours trying to track down that bug (causes graphical glitches in rare cases), but just couldn't find it. It just didn't occur to me to look there... Thank you for checking the code and reporting this bug!
No problem at all - I'm pretty sure I'm going to switch Daedalus's text rendering over to use this library, so it's the least I can do to repay you :)
plebihan
Posts: 10
Joined: Sun Jul 16, 2006 11:29 pm

Post by plebihan »

Hi All,

is there a way to use those librarys in an SDl Env. My Homebrew already uses SDL Graphic env and i don't think that using plenty a graphics env is a good thing.

Thanx for your answers and best regards

plebihan
Team-3GO
Posts: 17
Joined: Wed Nov 21, 2007 7:40 am
Location: PSP-Development

Post by Team-3GO »

BenHur wrote:
Team-3GO wrote:the samples work fine on fw 1.50 but will crash at 84% on 3.52 M33 under booth kernels. maybe its caused by some mods. will try it on a clean system again asap.
Since my PSP is a 3.52 M33-2, I would assume it's a moded font that causes the problem. ttf2pgf was written with little knowledge of the pgf file format. Since intraFont extracts a lot more info from the pgf files, some incorrectly set values might crash the system. I guess I have to add more error-checking and refuse to load those fonts. Is the flash0:/font/jpn0.pgf by chance an empty dummy file? If not, send it to me (email at the bottom of my page) so I can analyze it.

Cheers, BenHur
Right, figured this out earlier today while checking the font folder.
So some checks would help to solve the problem tought.
"Ignorance is the night of the mind, but a night without moon and star."
BenHur
Posts: 28
Joined: Sat Oct 20, 2007 5:26 pm

intraFont 0.21 (bugfix version) and PGF binary format doc

Post by BenHur »

I released version 0.21 of intraFont which fixes the bugs discovered by StrmnNrmn (possible graphical glitches, function could return a wrong value). You can download it from
http://www.psp-programming.com/benhur

For those interested in the binary format of PGF files: at the same place you'll find a document describing what's inside a PGF file. This document is at version 0.9934, i.e. I believe to have identified the meaning of 99.34% of all bytes. (To improve this number is hard, since there are only about 20 original PGF files out there.)

Cheers, BenHur
CpuWhiz
Posts: 42
Joined: Mon Jun 04, 2007 1:30 am

Post by CpuWhiz »

I am finally getting around to trying out your library. My project is using GU_PSM_8888 textures. When I tried to add in your library the text displays fine but everything else is rendering wrong. Before rendering my other content, I made sure to set sceGuTexMode, sceGuTexFilter, sceGuTexWrap, and sceGuTexFunc, but it's still not working right. What am I missing? Can I not mix indexed and non-indexed textures?
BenHur
Posts: 28
Joined: Sat Oct 20, 2007 5:26 pm

Post by BenHur »

CpuWhiz wrote:What am I missing? Can I not mix indexed and non-indexed textures?
That's my first guess, too. BUT, I'm by no means a GU-expert: the intraFont rendering stuff is pretty much identical to that of pgeFont. So you shouldn't rely on my opinion, but hopefully get a reply from someone who knows/wrote this stuff (Raphael, IWN, ...?)
StrmnNrmn
Posts: 46
Joined: Wed Feb 14, 2007 11:32 pm
Location: London, UK
Contact:

Post by StrmnNrmn »

CpuWhiz wrote:I am finally getting around to trying out your library. My project is using GU_PSM_8888 textures. When I tried to add in your library the text displays fine but everything else is rendering wrong. Before rendering my other content, I made sure to set sceGuTexMode, sceGuTexFilter, sceGuTexWrap, and sceGuTexFunc, but it's still not working right. What am I missing? Can I not mix indexed and non-indexed textures?
Are you sure you're setting sceGuTexMode etc every time that you render text, rather than just once?

When you render, intraFont sets the following state:

Code: Select all

sceGuClutMode&#40;GU_PSM_8888, 0, 255, 0&#41;;
sceGuClutLoad&#40;2, clut&#41;;

sceGuEnable&#40;GU_TEXTURE_2D&#41;;
sceGuTexMode&#40;GU_PSM_T4, 0, 0, &#40;&#40;font->options & INTRAFONT_CACHE_ALL&#41; == INTRAFONT_CACHE_ALL&#41; ? 1 &#58; 0&#41;;
sceGuTexImage&#40;0, font->texWidth, font->texWidth, font->texWidth, font->texture&#41;;
sceGuTexFunc&#40;GU_TFX_MODULATE, GU_TCC_RGBA&#41;;
sceGuTexEnvColor&#40;0x0&#41;;
sceGuTexOffset&#40;0.0f, 0.0f&#41;;
sceGuTexWrap&#40;GU_CLAMP, GU_CLAMP&#41;;
sceGuTexFilter&#40;GU_LINEAR, GU_LINEAR&#41;;
sceGuEnable&#40;GU_DEPTH_TEST&#41;;
It should just be a case of going through each of these and ensuring you're resetting them to your program's defaults after rendering any text.
StrmnNrmn
Posts: 46
Joined: Wed Feb 14, 2007 11:32 pm
Location: London, UK
Contact:

Post by StrmnNrmn »

BenHur wrote:
CpuWhiz wrote:What am I missing? Can I not mix indexed and non-indexed textures?
That's my first guess, too. BUT, I'm by no means a GU-expert: the intraFont rendering stuff is pretty much identical to that of pgeFont. So you shouldn't rely on my opinion, but hopefully get a reply from someone who knows/wrote this stuff (Raphael, IWN, ...?)
It's definitely fine mixing indexed and non-indexed textures - check out the pspsdk samples.

FWIW Daedalus quite happily mixes a number of different texture formats. I just call sceGuTextMode and sceGuTexImage to install each texture.
BenHur
Posts: 28
Joined: Sat Oct 20, 2007 5:26 pm

Post by BenHur »

StrmnNrmn wrote:It's definitely fine mixing indexed and non-indexed textures - check out the pspsdk samples.

FWIW Daedalus quite happily mixes a number of different texture formats. I just call sceGuTextMode and sceGuTexImage to install each texture.
Thanks for the info. A small note on the bug you reported earlier: If you havn't already changed it, don't forget to also change the memory allocation from

Code: Select all

v = sceGuGetMemory&#40;&#40;sizeof&#40;fontVertex&#41;<<2&#41; * &#40;n_glyphs+n_sglyphs&#41;&#41;;
to

Code: Select all

v = sceGuGetMemory&#40;&#40;sizeof&#40;fontVertex&#41;<<1&#41; * &#40;n_glyphs+n_sglyphs&#41;&#41;;
It cuts down the memory required.

Cheers, BenHur
CpuWhiz
Posts: 42
Joined: Mon Jun 04, 2007 1:30 am

Post by CpuWhiz »

As I already mentioned, I was setting the texture mode and I had already looked in the intraFont source for what needed to be set. It turns out my problem was as follows: I was copying the textures I needed to render once into vram and then just using the copy from vram to render each frame. Seems there is something going on with the indexed mode overwriting my textures. Copying my texture to vram every frame works but is not ideal. I render my other textures first and then the text, and that would be why this is working, because it's not overwriting my textures until after they have been rendered that frame.

Is there any way around this or do I just have to render text last and copy my textures every frame?
J.F.
Posts: 2906
Joined: Sun Feb 22, 2004 11:41 am

Post by J.F. »

CpuWhiz wrote:As I already mentioned, I was setting the texture mode and I had already looked in the intraFont source for what needed to be set. It turns out my problem was as follows: I was copying the textures I needed to render once into vram and then just using the copy from vram to render each frame. Seems there is something going on with the indexed mode overwriting my textures. Copying my texture to vram every frame works but is not ideal. I render my other textures first and then the text, and that would be why this is working, because it's not overwriting my textures until after they have been rendered that frame.

Is there any way around this or do I just have to render text last and copy my textures every frame?
From the GU example:

Code: Select all

	sceGuDrawBuffer&#40;GU_PSM_8888, &#40;void*&#41;0, BUF_WIDTH&#41;;
	sceGuDispBuffer&#40;SCR_WIDTH, SCR_HEIGHT, &#40;void*&#41;0x88000, BUF_WIDTH&#41;;
	sceGuDepthBuffer&#40;&#40;void*&#41;0x110000, BUF_WIDTH&#41;;
These set the address of the buffer to draw to, the buffer to display, and the depth test buffer, respectively. When doing your drawing with the Gu, you have to set these. Check that the addresses you set for your "normal" drawing and the addresses you set for text drawing are the same and do no overlap where you are saving the textures.

Note: those "addresses" are really just offsets into the video EDRAM.
CpuWhiz
Posts: 42
Joined: Mon Jun 04, 2007 1:30 am

Post by CpuWhiz »

Allow me to be more specific. I have depth testing disabled. I assume if I leave it disabled I don't need to allocate memory for it?

Code: Select all

src/psp/renderer.cpp&#58;50&#58; Draw buffer at 0x04000000 &#40;557056 bytes&#41;
src/psp/renderer.cpp&#58;51&#58; Display buffer at 0x04088000 &#40;557056 bytes&#41;
src/scene_battle.cpp&#58;107&#58; Font cache of size 256x256 at 0x04110000 &#40;32768 bytes&#41;
src/psp/renderer.cpp&#58;116&#58; Allocated texture of size 64x64 at 0x04120000 &#40;16384 bytes&#41;
src/psp/renderer.cpp&#58;116&#58; Allocated texture of size 128x128 at 0x04124000 &#40;65536 bytes&#41;
src/psp/renderer.cpp&#58;116&#58; Allocated texture of size 128x128 at 0x04134000 &#40;65536 bytes&#41;
src/psp/renderer.cpp&#58;116&#58; Allocated texture of size 128x128 at 0x04144000 &#40;65536 bytes&#41;
src/psp/renderer.cpp&#58;116&#58; Allocated texture of size 64x64 at 0x04154000 &#40;16384 bytes&#41;
Current VRAM Usage&#58; 09/64 allocs | 1409024/2097152 bytes &#40;67.19%&#41;
I am keeping track of where everything goes and using sceGuCopyImage and then sceGuTexImage with the vram address. As you can see above I changed the source a little to copy the font cache texture into vram also. Is there anything wrong with the above?
Post Reply