apascan - Handy tool for Linux

Chewi · Post by **Chewi** » Mon Dec 31, 2007 5:03 am

I've written a program for Linux that scans APA partitioned hard drives and uses Linux's device-mapper to create block devices for each partition. This means you can plug the hard drive into your PC and work directly with any Linux partitions you may have on the drive. No more fudging around with sector offsets! You can even mount installed games as ISOs, though I'm not sure whether that's really very useful.

This program could also potentially be used to boot the PS2 into Linux from an APA partition. It runs in userspace but an initrd/initramfs could do the necessary work before the system boots. I have Linux 2.6 in mind here but MontaVista's APA code is adequate for 2.4. I did try and update that code for 2.6 so that everything could be done in kernelspace but the non-contiguous partitions made it too messy.

Anyway, it's actually not 100% finished yet, it doesn't seem to get the offsets for the additional partition segments quite right but it's nearly there and you can still see it in action.

Code: Select all

root@rhapsody # ./apascan /dev/hdb
apascan v0.1 - Copyright &#40;c&#41; 2008 James Le Cuirot <chewi@aura-online.co.uk>
Distributed under the GNU General Public License v2 with no warranty.

Scanning /dev/hdb...
Found partition "__mbr" with 1 parts.
Mapping as "hdb-apa-__mbr".
Found partition "PP.HDL.Gunbird Special Edition" with 3 parts.
Mapping as "hdb-apa-PP.HDL.Gunbird_Special_Edition".
2 partitions mapped.
root@rhapsody # ls /dev/mapper
control  hdb-apa-__mbr  hdb-apa-PP.HDL.Gunbird_Special_Edition
root@rhapsody # mount /dev/mapper/hdb-apa-PP.HDL.Gunbird_Special_Edition iso
root@rhapsody # ls iso
bootmenu.bin  inaf.bin  modules  sles_530.21  system.cnf  wallpapr.bin

I'll upload it to my Gentoo PS2 wiki when it's ready.

Mega Man · Post by **Mega Man** » Tue Jan 01, 2008 2:16 am

This is very interesting. Please post a link when your are finished.

As I understand you, I can also use it to install (copy) Linux on the hard disc without a PS2 (as long I have created the partition on the PS2).

Chewi · Post by **Chewi** » Wed Jan 02, 2008 2:48 am

That is partially correct, you could use ps2fdisk on your machine and save yourself the bother of creating the partitions on the PS2. :)

It's ready now. It seems to work well, the only problem I've had is that the DMS HDD Format Tool appears to create partition tables that not even hdl_dump, let alone apascan, can read. uLaunchELF can read them though. Maybe this isn't really a problem but I expected it to work and I want to know what's up with it.

http://gps2.aura-online.co.uk/trac/gent ... ki/apascan

RashivaX · Post by **RashivaX** » Sat Mar 15, 2008 2:53 pm

I currently have a 320 GB HD installed in my PS2 and am wanting to use a bit of the space for media, such as music, etc... I created two extra partitions with DMS, but network transfers are incredibly slow. If I want to upload mass amounts of media, it will take a hair ripping amount of time.

I need to be able to mount the partitions on one of my PC's... one, my primary, is running Ubuntu Linux... so this works nicely for apascan :) but, again, the problem: DMS formatted... so, if not the ability to handle these partitions through apascan, and this gets into different territory, is there another app I can use to create partitions which would be fully compatible/readable on both the PC and the PS2? You mention ps2fdisk... not used this, but could I use this without losing current data--- just dealing with unallocated space?

BTW, this is another question probably better in another thread, but I will ask it anyway, since it's related... I have created one of the DMS partitions as 60GB... DMS does not show the size correctly... makes me slightly nervous... especially when thinking about how DMS is doing something somewhat mysterious, since you were having the issue with the partitions showing... so, am fearing other data might be overwritten.

Anyway, help on the main point at least, is much appreciated... thanks...

RashivaX · Post by **RashivaX** » Sat Mar 15, 2008 7:06 pm

Having another problem now.. that is with compiling apascan.

Am using Ubuntu 7.10 amd64.

First got this error about libdevmapper:

Code: Select all

checking libdevmapper.h usability... no
checking libdevmapper.h presence... no
checking for libdevmapper.h... no
configure&#58; error&#58; libdevmapper not found. Get it from http&#58;//sourceware.org/dm.

trying to install libdevmapper gave this:

Code: Select all

&#58;~/DL/apascan-0.1$ sudo apt-get install libdevmapper
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting libdevmapper1.02.1 instead of libdevmapper
libdevmapper1.02.1 is already the newest version.&#91;

I then installed libdevmapper-dev and also did a:

Code: Select all

sudo modprobe dm-crypt

Which I am uncertain about... saw this in a thread on ubuntuforums in regards to an issue someone was having with crypto--- my knowledge is VERY limited here :( But, thought I should mention all I can...

This seems to have gotten through the ./configure fine.. but when I go to make, I get:

Code: Select all

gcc -DHAVE_CONFIG_H -I.     -g -O2 -MT apascan.o -MD -MP -MF .deps/apascan.Tpo -c -o apascan.o apascan.c
In file included from apascan.c&#58;26&#58;
/usr/include/sys/types.h&#58;46&#58; error&#58; conflicting types for ‘loff_t’
/usr/include/linux/types.h&#58;30&#58; error&#58; previous declaration of ‘loff_t’ was here
/usr/include/sys/types.h&#58;62&#58; error&#58; conflicting types for ‘dev_t’
/usr/include/linux/types.h&#58;13&#58; error&#58; previous declaration of ‘dev_t’ was here
In file included from /usr/include/sys/types.h&#58;133,
                 from apascan.c&#58;26&#58;
/usr/include/time.h&#58;105&#58; error&#58; conflicting types for ‘timer_t’
/usr/include/linux/types.h&#58;22&#58; error&#58; previous declaration of ‘timer_t’ was here
In file included from apascan.c&#58;26&#58;
/usr/include/sys/types.h&#58;198&#58; error&#58; conflicting types for ‘int64_t’
/usr/include/linux/types.h&#58;98&#58; error&#58; previous declaration of ‘int64_t’ was here
/usr/include/sys/types.h&#58;204&#58; error&#58; conflicting types for ‘u_int64_t’
/usr/include/linux/types.h&#58;97&#58; error&#58; previous declaration of ‘u_int64_t’ was here
In file included from /usr/include/sys/types.h&#58;220,
                 from apascan.c&#58;26&#58;
/usr/include/sys/select.h&#58;78&#58; error&#58; conflicting types for ‘fd_set’
/usr/include/linux/types.h&#58;12&#58; error&#58; previous declaration of ‘fd_set’ was here
In file included from apascan.c&#58;26&#58;
/usr/include/sys/types.h&#58;235&#58; error&#58; conflicting types for ‘blkcnt_t’
/usr/include/linux/types.h&#58;124&#58; error&#58; previous declaration of ‘blkcnt_t’ was here
In file included from /usr/include/inttypes.h&#58;28,
                 from /usr/include/libdevmapper.h&#58;19,
                 from apascan.c&#58;35&#58;
/usr/include/stdint.h&#58;56&#58; error&#58; conflicting types for ‘uint64_t’
/usr/include/linux/types.h&#58;96&#58; error&#58; previous declaration of ‘uint64_t’ was here
make&#91;1&#93;&#58; *** &#91;apascan.o&#93; Error 1

Any ideas?

Chewi · Post by **Chewi** » Sat Mar 15, 2008 9:04 pm

First of all, as it says in the README, apascan won't handle drives larger than 127GB. The official APA format doesn't support this and larger sizes were only achieved through the 48-bit LBA hack and the ToxicOS APAEXT format. I'd like to support them but I don't have any drives larger than 120GB.

You were right to install libdevmapper and libdevmapper-dev. I don't think dm-crypt is necessary and you wouldn't need to modprobe anything until after it's built anyway.

The header conflict you're getting may be a Ubuntu-specific issue and it appears to be due to a clash between x86 and amd64 headers. I don't know how Ubuntu handles multilib but Gentoo does it by replacing the usual /usr/include/sys/types.h with the following and does the same for many other header files.

Code: Select all

/* Autogenerated by create_ml_includes&#40;&#41; in multilib.eclass */

#ifdef __i386__
# include <gentoo-multilib/x86/sys/types.h>
#endif /* __i386__ */

#ifdef __x86_64__
# include <gentoo-multilib/amd64/sys/types.h>
#endif /* __x86_64__ */

That picks out the appropriate file for the architecture you're building for. I don't have access to a Ubuntu amd64 system, only a Debian amd64 one. I haven't tried building on it (it's not really my machine) but /usr/include/sys/types.h has the usual contents so I'm not sure how they handle multilib. If I were to guess, I'd say that yours is the amd64 version, as it should be, and it's actually the headers in /usr/include/linux that are wrong. Maybe you installed the wrong kernel headers somehow?

RashivaX · Post by **RashivaX** » Thu Mar 27, 2008 2:08 am

Should have replied back before...just remembered, but not much that can be done since there is not support currently for drives larger than 127GB.

My Kernel is generic. linux-headers-2.6.22.14-generic. I haven't had this amd64 for very long and did a fresh install of Ubuntu 7.10 amd64 bit version... this is what was installed. Could it be that it is generic that it is causing problems?

Again, knowledge is limited. Only been a dedicated Linux user for about a year and no good books, just learn as I go along from forums and trial and error :)

I would give you access to this machine to compile on if not for distance... eh well...

Chewi · Post by **Chewi** » Thu Mar 27, 2008 2:45 am

That package appears to be the correct one. I did a little digging and found a similar issue. Try moving the "#include <sys/types.h>" line to the top of the list of includes in apascan.c. Let me know if this works.

RashivaX · Post by **RashivaX** » Thu Mar 27, 2008 3:08 am

Ok, done... appears to have compiled and installed properly. I get the output text for apascan :) But, I cannot test it further ATM... I do have a 60GB drive that I could try within a day or so.

This probably won't make much sense to me anyway, but why was that the problem?

Chewi · Post by **Chewi** » Thu Mar 27, 2008 3:30 am

The suggestion came from this bug. I've just had a look through the files to see what's going on.

asm/byteorder.h includes linux/byteorder/little_endian.h and that includes linux/types.h. This defines types such as loff_t but only when __KERNEL_STRICT_NAMES is false. At this point, it is false so they are defined. loff_t is then redefined as a conflicting type later on in sys/types.h.

sys/types.h includes features.h and this sets __KERNEL_STRICT_NAMES to true. By including sys/types.h first and having this set to true, loff_t does not get redefined in linux/types.h later on.

Why this happened to you but not me and whether this was really my fault or not is a mystery! Thanks for helping me fix this. I'll put out a new version soon.

cosmito · Post by **cosmito** » Sun Aug 16, 2009 1:55 am

Hi Chewi,

It seems gps2.aura-online.co.uk is not online... Definitely off? Any chance of uploading the apascan to somewhere else?

Chewi · Post by **Chewi** » Wed Sep 23, 2009 11:43 pm

Apologies for not replying, I didn't see this message until now. Since I've largely given up on PS2 development and have moved from SVN to GIT, I wasn't in a hurry to get that site back up. I have now created a repository for apascan on github instead.

http://github.com/chewi/apascan

forums.ps2dev.org

apascan - Handy tool for Linux

apascan - Handy tool for Linux

Any progress on DMS HDD formatted partitions?

apascan compile issues