Calling for ideas about builing image thumbnails

[Tong]

Dear all, this is the first time for me to post a topic in this forum.

I am currently working on a PSP photo viewer in which users can see the previews of their images (PNG and JPG) before they open them, just like the one in Sony's XMB. I propose the following algorithms to show the previews:

1.Read the WHOLE picture into RAM and select some pixels form it to make the thumbnail;

2.Read the image row by row but only take some pixels to make the thumbnail.

Unfortunately, the above methods are found to be very inefficient and resource-consuming especially for large images. Does anyone have better ideas, or know any functions in libpng and libjpeg that are useful in building thumbnails? Thanks.

[rapso]

some jpg are progressive encoded, so they have already a thumbnail.
once you made the tumbnail, you could save it in the same place and next time just load it (most pictureviewer I know have this kind of caching), so it won't be that bad.

btw. I'd do methode 2 to generate the images, as some decompressed jpg might requiere more memory than the psp has.

[Tong]

I have just implemented method 2 using libpng to generate thumbnail images with sides 64px, and here is the codes.

Code: Select all

//do some preparations here...

float SkipX=(float)ImgWidth/64,
	SkipY=(float)ImgHeight/64,
	TargetX=0, TargetY=0;
int StepX=0, StepY=0, ThumbX=0, ThumbY=0;
	
for&#40;y = 0; y <ImgHeight; y++&#41; 
&#123;	
	png_read_row&#40;png_ptr, &#40;u8*&#41;tempLine, png_bytep_NULL&#41;; //read a row
        
	if &#40;y==StepY&#41;//if the row is the one we want
	&#123;
		//select some pixels from the row
		for &#40;TargetX=0, ThumbX=0; TargetX<&#40;float&#41;ImgWidth; TargetX+=SkipX, ThumbX++&#41;
		&#123;
			StepX=RoundOff&#40;TargetX&#41;;
			ThumbData&#91;ThumbY*64+ThumbX&#93;=tempLine&#91;StepX&#93;;
		&#125;
		//next row
		ThumbY++;
		TargetY+=SkipY;
		StepY=RoundOff&#40;TargetY&#41;;
	&#125;
&#125;

What I observe is that the time taken for it to generate a thumbnail is almost the SAME(some may longer) as reading the whole image. It is not surprising as I think each generation needs to read through the whole image. I wonder how SONY's photo viewer in XMB reads thumbnails, it can just finish all in a split second!!

[StrmnNrmn]

What I observe is that the time taken for it to generate a thumbnail is almost the SAME(some may longer) as reading the whole image. It is not surprising as I think each generation needs to read through the whole image.

You code is reading through the entire image, so it's always going to take at least as long. It sounds like you really need to cache the thumbnails you generate.

In terms of minimising the time spent generating the thumbnail data, you could use the graphics hardware just to blit from the source image down to your desired thumbnail size? It you do this, you'll also get bilinear filtering for free (or close to free) and it'll look much better than the point sampling method you're currently using. It'll take a bit of fiddling with render targets, but I'm sure it'll be far faster than anything you could achieve with the cpu.

If you want to keep your UI responsive while generating the thumbnails you could also look at shifting thumbnail generation to a separate thread?

[Brunni]

You can use the GPU to reduce your image size but it requires to load it entirely, and thus it's slow. You'd have to do it in a parallel thread I guess. Example (with OSLib):

Code: Select all

//Creates a 64x64 image from an file
OSL_IMAGE *loadThumbnail(const char *filename)		{
	OSL_IMAGE *src, *dst, *oldDrawBuf = oslGetDrawBuffer();
	int oldBilinear = osl_bilinearFilterEnabled, oldDither = osl_ditheringEnabled;

	src = oslLoadImageFile(filename, OSL_IN_RAM, OSL_PF_8888);
	dst = oslCreateImage(64, 64, OSL_IN_VRAM, OSL_PF_5650);
	if (src && dst)			{
		oslSetDrawBuffer(dst);
		oslSetBilinearFilter(1);
		oslSetDithering(1);
		src->x = src->y = 0;
		src->stretchX = 64;
		src->stretchY = 64;
		oslDrawImage(src);
		oslSetDrawBuffer(oldDrawBuf);
		oslSetBilinearFilter(oldBilinear);
		oslSetDithering(oldDither);
	}
	if (src)
		oslDeleteImage(src);

	return dst;
}

Edit: sorry, my answer is not really helpful seeing what StrmnNrmn posted :-'

[rapso]

I wonder how SONY's photo viewer in XMB reads thumbnails, it can just finish all in a split second!!

libpng and libjpg are not really fast. e.g. the intelJpgLib (no, there is no version for psp ;) ), could read about 20 320*200 jpg images per second on a Pentium200MMX (if I remeber correctly). so, if you want it faster, you'd have to improve the speed of the libs.
Afaik, some picture-viewer (lieg irfanview) cache the next image in advance. so if you step to the next image, you already have it.
you'd not even need threading.

currently you do something like:

Code: Select all

wait for key
if key==next then 
  load next image
  show next image

with a little modification you could do something like

Code: Select all

wait for key
if key==next then 
  show next image
  load next->next image
  memorize next->next as next image

so you'd first show the cached image and then, when the user is viewing it, you could already load the next image. if the user wants to see the next, he'd still wait some time if it's not cached yet, but probably still less waiting then in your current version.

[noxa]

My day job deals with nothing but photos, so I've got some experience here :)
My suggestion is to go with epeg (http://www.enlightenment.org/Libraries/Epeg/) (or hack it up and change it to your liking like I did). It is incredibly fast at what it does, and combined with a few tricks (such as sampling at powers of two larger than what you want and then scaling down) you can get thumbnails that look just as good as ones taken from the real image.

Edit: epeg uses libjpeg. I haven't seen a PSP port, but it is POSIX-based and when I compiled it on OS X I didn't have to make any changes.

[rapso]

Edit: epeg uses libjpeg. I haven't seen a PSP port, but it is POSIX-based and when I compiled it on OS X I didn't have to make any changes.

hmm... if it rely on libjpeg, then the only improvement is the downsampling speed, but I guess the decompression with huffman and DCT ist the really time consuming stuff.

My day job deals with nothing but photos, so I've got some experience here :)

sounds like you'd be watching playboy all day long ;););)

[noxa]

hmm... if it rely on libjpeg, then the only improvement is the downsampling speed, but I guess the decompression with huffman and DCT ist the really time consuming stuff.

Wrong. It only reads the lines from the image it needs for the desired size image. As long as the underlying file APIs are smart enough (and from what I've seen of newlib, they should be), a lot of file IO can be saved. In my particular use of epeg, I was loading large (3k x 3k) jpegs on crappy macs with slow hdds, and saw a large performance increase (talking of orders of magnitude in terms of time required to load/decode images). Going from loading 3000 lines 3000 pixels long to 128 lines 3000 pixels long is a big win. Not only in file IO, but in memory - on a device like the PSP it would be impossible to load an image of that size - there is not enough RAM to hold the decoded contents. epeg does a good job keeping memory usage during decoding low (and with some modification it could be made much better).

This is exactly the kind of stuff that was mentioned above, only epeg works and deals with all the nasty stuff for you.

sounds like you'd be watching playboy all day long ;););)

Funny.....?

[Tinnus]

How about that...

Code: Select all

for&#40;y = 0; y <ImgHeight; y++&#41; 
&#123;	
	png_read_row&#40;png_ptr, &#40;u8*&#41;tempLine, png_bytep_NULL&#41;; //read a row
        
	if &#40;y==StepY&#41;//if the row is the one we want
	&#123;
		//select some pixels from the row

Code: Select all

for&#40;y = 0; y <ImgHeight; y++&#41; 
&#123;	
	if &#40;y==StepY&#41;//if the row is the one we want
	&#123;
		png_read_row&#40;png_ptr, &#40;u8*&#41;tempLine, png_bytep_NULL&#41;; //read a row
		
		//select some pixels from the row

edit: and maybe decompress the temp line thing to VRAM, it should be a little faster than RAM too. Anyway, to get the best speed possible your best bet would probably be to hack the read_row code in libpng.

[Tong]

Code: Select all

for&#40;y = 0; y <ImgHeight; y++&#41;
&#123;   
   if &#40;y==StepY&#41;//if the row is the one we want
   &#123;
      png_read_row&#40;png_ptr, &#40;u8*&#41;tempLine, png_bytep_NULL&#41;; //read a row
      
      //select some pixels from the row

Tinnus, you have made a big mistake! This routine does not work as sequential read is used in scanning rows. In another word, if you want to scan the last row, you have to scan all those rows before it.

[Tong]

A big thanks to all who had give ideas to me especially noxa and rapso. I have spent a couple of days hacking into libepeg as recommended by noxa and I am now making use of it. Well, this library is really fantastic as noxa said, it can generate thumbnails at a terribly high speed!

As there no libepeg for psp up to now, let me share how I compile and use it with those who are interested at this useful library.

1.Download and install libjpeg (necessary, libepeg relies on it).
2.Download libepeg form http://enlightenment.freedesktop.org/fe ... 007.tar.gz
3.Unpack the package and go into src/lib directory.
4.Open “epeg_private.h” and remove line 16 “#include “config.h”” as it is useless to us.
5.“make” it using the following Makefile:

Code: Select all

PSPSDK=$(shell psp-config --pspsdk-path)
PSPDIR=$(shell psp-config --psp-prefix)

TARGET_LIB = libepeg.a
OBJS = epeg_main.o

INCDIR = $(PSPSDK)/../include

CFLAGS = -O2 -G0 -Wall
CXXFLAGS = $(CFLAGS) -fno-exceptions -fno-rtti
ASFLAGS = $(CFLAGS)

LIBS = -ljpeg
include $(PSPSDK)/lib/build.mak

install: $(TARGET_LIB)
	@echo "Installing libepeg into $(PSPDIR)"
	@mkdir -p $(PSPDIR)/include $(PSPDIR)/lib
	@cp Epeg.h $(PSPDIR)/include
	@cp libepeg.a  $(PSPDIR)/lib
	@echo "Done"

6.If successful, you should find Epeg.h and libepeg.a in your psp dir. You can use it now.

[Tong]

Here is a simple example for outing a 128x96 thumbnail.

Code: Select all

#include <stdio.h>
#include <stdlib.h>
#include <pspkernel.h>
#include <pspdebug.h>
#include <Epeg.h>

PSP_MODULE_INFO&#40;"epeg", 0, 1, 1&#41;;

#define printf pspDebugScreenPrintf

int main&#40;&#41;
&#123;
	pspDebugScreenInit&#40;&#41;;
	//paste codes for setting up callbacks here
	
	Epeg_Image *MyImage;
	printf&#40;"Opening and reading info.\n"&#41;;
	if &#40;&#40;MyImage = epeg_file_open&#40;"Input.jpg"&#41;&#41;==NULL&#41;
	&#123;
		printf&#40;"Cannot find Input.jpg, exiting..."&#41;;
		sceKernelDelayThread&#40;2000000&#41;;
		sceKernelExitGame&#40;&#41;;
	&#125;
	
	int width, height;
	epeg_size_get&#40;MyImage, &width, &height&#41;;
	printf&#40;"Original dimemsion&#58; %dx%d\n", width, height&#41;;
	
	epeg_decode_size_set&#40;MyImage, 128, 96&#41;; //key point&#58; size of thumbnail
	epeg_quality_set &#40;MyImage, 85&#41;;
	epeg_thumbnail_comments_enable &#40;MyImage, 1&#41;;
	epeg_comment_set &#40;MyImage, "thumbnail"&#41;;
	
	printf&#40;"Encoding and saving thumbnail...\n"&#41;;	
	epeg_file_output_set &#40;MyImage, "Output.jpg"&#41;;
	epeg_encode &#40;MyImage&#41;; //this creates the file 
	epeg_close &#40;MyImage&#41;; //free all allocated resources
	
	printf&#40;"Done. Exiting..."&#41;;
	sceKernelDelayThread&#40;1000000&#41;;
	sceKernelExitGame&#40;&#41;;
    return 0;    
&#125;

And its Makefile.

Code: Select all

TARGET = Epeg
OBJS = main.o

CFLAGS = -O2 -G0 -Wall
CXXFLAGS = $(CFLAGS) -fno-exceptions -fno-rtti
ASFLAGS = $(CFLAGS)

LIBS = -lepeg -ljpeg 

EXTRA_TARGETS = EBOOT.PBP
PSP_EBOOT_TITLE = libepeg sample

PSPSDK=$(shell psp-config --pspsdk-path)
include $(PSPSDK)/lib/build.mak

See the scary speed? Note: if we read the entire file into RAM and open it using Epeg_Image *epeg_memory_open (unsigned char *data, int size), the thumbnailing speed can be further increased.

[noxa]

Glad it worked for you, and thanks for posting the results. For anyone looking in the future, this will be the thread to see :)

[Tong]

noxa, do you know any similar libraries for PNG thumbnailing?