access on protected ram & software-mmu

gringo · Post by **gringo** » Sat Nov 24, 2007 9:16 am

Hey there

Since I got nothing on the on this found here, I'll ask. I am about to think writing a piece of softstuff for the psp. But before I go to the workbench I want to clarify a few things..
I found the int sceKernelSetDdrMemoryProtection function in the sdk and obviously it protects a memory area. What happens when a user-module wants to allocate memory that lies in that scope. I think the psp-kernel-thing doesn't have a MMU, so I guess the user-module will fail allocation with an error, instead of the kernel giving the module a free place somewhere else.
I guess there is no virtual memory address space...
Actually I want to know: Is there anyway to implement a kernel-space-module that assigns the memory to the user-space-programs - so to say a software-mmu.

Post by **TyRaNiD** » Sat Nov 24, 2007 6:06 pm

I am not sure you quite understand, by default in user mode you get all the RAM you are allowed to have just there for you to use. You can get an extra 4 Megs by enabling the Volatile memory (sceKernelLockVolatileMemory I think is the function) but that isn't designed for long term use.

Any other memory is kernel memory which is already in use in some fashion, and on the slim the latest M33 gives you the ability to use another 26 megs of ram in one contiguous block.

So I am not sure what you would need a software-mmu for :)

gringo · Post by **gringo** » Sun Nov 25, 2007 12:27 am

Thanks for your reply

I am not sure you quite understand, by default in user mode you get all the RAM you are allowed to have just there for you to use.

Well, that's what I was worried about. So, I interpret what you said as: "My user-mode program can use the memory it wants and the kernel can't do nothing against it", right.
Well I had lil hope, that the kernel would be so nice and allocate a different area, if the claimed area is already used - but that was more like a wish ;)

So I am not sure what you would need a software-mmu for :)

Don't hit me - but I had the idea of a multi-tasking engine that integrates seamless into the system. Thus, I would have needed memory protection..
I could cache all states and memory onto mem-stick - but that sucks bad and I wouldn't want to even begin programming something like that.

moonlight · Post by **moonlight** » Sun Nov 25, 2007 2:36 am

The extra 4 MB is not a kernel space. It's a volatile space. It is used by the sleep mode, and it is also used by the user/vsh mode dialogs and modules that utility.prx inits. (kernel mode can load user modules in that space by specifying partition id 5).

User mode is free of using that area when they use the VolatileMemoryLock, but they have to watch the sleep mode issue.

gringo · Post by **gringo** » Sun Nov 25, 2007 4:19 am

Hmm ... ok - thanks for the additional information. But without insulting you - this is not really what I wanted to know.
Thanks anyway ;)

Don't hit me - but I had the idea of a multi-tasking engine that integrates seamless into the system. Thus, I would have needed memory protection..
I could cache all states and memory onto mem-stick - but that sucks bad and I wouldn't want to even begin programming something like that.

J.F. · Post by **J.F.** » Sun Nov 25, 2007 4:24 am

gringo wrote: Don't hit me - but I had the idea of a multi-tasking engine that integrates seamless into the system. Thus, I would have needed memory protection..
I could cache all states and memory onto mem-stick - but that sucks bad and I wouldn't want to even begin programming something like that.

Multi-tasking and memory protection are two completely different things. Multi-tasking does not need memory protection, and vice versa. Those tend to be linked by users as features in "modern" OSes, but they really aren't related at all. Similarly, memory protection and virtual memory aren't related either. Good example - Macs had virtual memory LONG before they ever had any kind of memory protection.

gringo · Post by **gringo** » Sun Nov 25, 2007 8:15 am

Multi-tasking and memory protection are two completely different things.

You mean, like men and women ;) Or like Macs and PSPs (sorry I couldn't stop myself from saying that :||

At the moment I simply cannot think of a way to multitask between apps on the psp without memory-protection (or whatever you might call it ...)
If you can think of way, so please tell me.
The psp has no memory management function in it's OS, so I would guess my question could be "answerable" and straightforward.

J.F. · Post by **J.F.** » Sun Nov 25, 2007 10:47 am

gringo wrote:
Multi-tasking and memory protection are two completely different things.
You mean, like men and women ;) Or like Macs and PSPs (sorry I couldn't stop myself from saying that :||

At the moment I simply cannot think of a way to multitask between apps on the psp without memory-protection (or whatever you might call it ...)
If you can think of way, so please tell me.
The psp has no memory management function in it's OS, so I would guess my question could be "answerable" and straightforward.

All personal computers multitasked without memory protection for quite some time. The Amiga was probably the most advanced, doing preemptive multitasking while everyone else was still cooperative.

The main issue is being able to load PSP apps at dynamically determined locations. Relocating the code and data of a program is not a simple task, and can be made even more difficult by the file format used. PSP uses a form of ELF files, so you should do some research into how you load ELF format files in general.

To make PSP programs multitask, you first task (no pun intended) is to make a loader that can load an EBOOT (the ELF file inside it at least) into whatever memory you allocate for it. Loading multiple apps is then a matter of just allocated different blocks of memory for them to be loaded into.

As far as multitasking goes, look at various examples of how other OSes multitask. A small one (always good to look at small SIMPLE OSes first) to check out is Contiki. You might also check out AROS. AROS is actually a good candidate for a shell to run on the PSP to get a nice multitasking desktop for apps to run in. AROS does preemptive multitasking without memory protection or even an MMU on a number of platforms.

Jim · Post by **Jim** » Sun Nov 25, 2007 10:59 am

Xinu might be worth a look too.

Jim

gringo · Post by **gringo** » Sun Nov 25, 2007 11:07 am

Thanks J.F. and Jim
This information is very useful. I now have a more general idea how it could work, I will work on it and I hope I can get back if I have questions later on.

Post by **TyRaNiD** » Sun Nov 25, 2007 12:29 pm

As so what you are actually asking for is an allocator which isn't really the same thing :)

There you are in luck, the kernel already hands out allocation blocks. Normally you don't see it but the executable, stack and heap are all allocated out of memory using a simple block allocator.

If all you want to do is multitask then just load and stop prx modules. As long as their heaps are small then it should work as expected, however you would need to implement a sort of user mode mediator to share graphical resources and such. In the end it is less of an OS and more just some sort of shell app with module support.

gringo · Post by **gringo** » Sun Nov 25, 2007 9:57 pm

Yea, Tyranid you understood. That's exactly what I want :D
A memory allocater.

There you are in luck, the kernel already hands out allocation blocks. Normally you don't see it but the executable, stack and heap are all allocated out of memory using a simple block allocator.

Yes I saw these functions in the sdk. My practical problem is kind of silly.
An app stores somethin to a absolute address (someone built a assembler routine that wants that), say 0x09000000 a secound app uses exact the same address to store somethin else. Can the kernel handle that with its allocation/reallocation mechanism?? Can it?

In the end it is less of an OS and more just some sort of shell app with module support.

That's ok. I never said I want to code a OS ;)
But I don't want to start the apps in an app in order to switch between them. But that's only finetuning..
It would be nice if you could get back to XMB to start them. If you press on 'home' (2seks or such) then you the the running tasks.

tacoSunday · Post by **tacoSunday** » Sun Nov 25, 2007 11:08 pm

An app stores somethin to a absolute address (someone built a assembler routine that wants that), say 0x09000000 a secound app uses exact the same address to store somethin else. Can the kernel handle that with its allocation/reallocation mechanism?? Can it?

My understanding of the issue is that it's not the allocator that handles the issue of code relocation, but it is the module loading function. The ELF format stores the jumps and pointers as an offset from a specific point(the starting address of the allocated block). When the load module function called it adds the offset to the base address as it writes the module to the memory block. If you hardcode a specific address in a module, say 0x09000000, then code relocation will not change it.

As Tyranid stated virtual memory and multitasking are two different things.

Hope this helps.

gringo · Post by **gringo** » Mon Nov 26, 2007 2:09 am

Thank tacoSunday for your response
As J.F. said, I first should read into ELF-loading a lil bit, then I will probably understand better how I can solve my stuff. What you said about it confirmed what I have guessed about loading.

If you hardcode a specific address in a module, say 0x09000000, then code relocation will not change it.

Assumption1: <Yes, if it doesn't change that, so there would be a conflict if another module has hardcoded this address in.>

if (Assumption1)
<
So I have to patch the loader/reloader that he gives these hardcoded addresses a place on free ram. This would be even harder if the app dynamically assigns hardcoded addresses, and thus not visible at loading time, right?

The taskmanager should be 99%compatible to all apps and not only to apps that have no hardcoded addresses in use.
>

if !(Assumption1)
<
please tell me ;)
>

Post by **TyRaNiD** » Mon Nov 26, 2007 3:26 am

No apps should really be using fixed addresses for normal ram access, they should be accessing either heap address, internal data section addresses or stack addresses all of which can change as long as you use a relocatable executable.

That doesn't hold though if you are accessing vram, or the GE subsystem or any other non-sharable resource. The PSP was not designed to have many arbitrary apps working at the same time sharing the same IO, it was designed for a single app to take control over the whole system.

If what you really want (by the sound of it) is to be able to multitask arbitrary homebrew under which you have no control then you are fucked.

gringo · Post by **gringo** » Mon Nov 26, 2007 6:52 am

No apps should really be using fixed addresses for normal ram access, they should be accessing either heap address, internal data section addresses or stack addresses all of which can change as long as you use a relocatable executable.

..I have to read into it elf-loading. ...So not all elf-binarys are relocatable exectuables?

The PSP was not designed to have many arbitrary apps working at the same time sharing the same IO, ...

I don't want to run them at the same time. Just want to fast switch between them.
And the sharable ressources are a problem but I think this can be solved (for the simpler apps ;)

Post by **TyRaNiD** » Mon Nov 26, 2007 7:35 am

..I have to read into it elf-loading. ...So not all elf-binarys are relocatable exectuables?

No in fact the primary executable is designed to be relocatable on the PSP

I don't want to run them at the same time. Just want to fast switch between them.
And the sharable ressources are a problem but I think this can be solved (for the simpler apps ;)

Well you are still screwed, and lets face it I don't see much point in it :) Complain to the homebrew authors that you want the ability to store state which can be easily recovered, or implement an extendable shell where you can build little applets which can be easily switched. And you will only be able to solve sharable resources for the most trivial of applications, it really wont be worth it.