Network startup delay

boman666 · Post by **boman666** » Tue Feb 17, 2004 2:26 am

Hello,

It takes ages (7-8 sec) after ps2smap.irx is started before I'm able to connect to the PS2. Anyone knowing what's causing the delay and what to do about it?

Post by **blackdroid** » Tue Feb 17, 2004 5:43 am

autonegotiation most likely. if you have a switch ( which is my guess ) replace it with a xover cable. or if you can set the port on the switch to something explicit like 100/full duplex. some switches have builtin arpflood mechanism and wont broadcast arp until a certain delay has expired.

boman666 · Post by **boman666** » Wed Feb 18, 2004 10:14 pm

The PS2 is directly connected to my PC so I guess the switch shouldn't interfere. However, the driver for the NIC was set to full autonegotiation, I changed it to force 100mbit/s full duplex but it had no impact on the delay. Any other suggestions? I'll try to connect the PS2 to the other NIC, if the prb is related to that particular NIC. Anyways, thanks for the input!

Post by **blackdroid** » Wed Feb 18, 2004 11:48 pm

you could always try with 10/full aswell..

Post by **Drakonite** » Thu Feb 19, 2004 9:46 am

I have the exact same problem.

I'm using a cross over cable and have tried every possible settng for speed/duplex mode and no matter what I do I have a significant pause when starting, and transfer speeds are slower than they should be.

I have not yet figured out what the problem is. It appears I may have finally found someone who has the same problem I do ;)

What OS and NIC are you using?

boman666 · Post by **boman666** » Thu Feb 19, 2004 10:03 am

Win XP Pro with a nVidia nForce2 NIC. By the way I have a SPH-50004 PS2. I read somewhere on ps2-linux that the newer models, which I guess include SCPH-50004, require a updated smap.c. I don't know if it's related to this particular prb but it might be worth looking into.

Post by **mrbrown** » Fri Feb 20, 2004 12:22 am

boman666 wrote:Win XP Pro with a nVidia nForce2 NIC. By the way I have a SPH-50004 PS2. I read somewhere on ps2-linux that the newer models, which I guess include SCPH-50004, require a updated smap.c. I don't know if it's related to this particular prb but it might be worth looking into.

smap.c is for ps2linux, not ps2link.

boman666 · Post by **boman666** » Fri Feb 20, 2004 8:11 am

Yeah, I know. But, AFAIK, many of the functions in ps2smap.irx is based on smap.c that comes with the linuxkit. Maybe some incompatibility with newer HW has snuck into ps2smap.irx that way. Or am I totally off base here?

Post by **blackdroid** » Fri Feb 20, 2004 8:18 am

just a thought, what kind of transfer speed do you get ?

Post by **mrbrown** » Fri Feb 20, 2004 8:21 am

No, I don't think you're off base, but the IOP has a different method of hardware access to SMAP than the EE does. I think the main ps2linux problem is that they weren't accessing SMAP (and the HDD) with account for the EE's hardware "view" of the SBUS, which caused problems when reading from/writing to SMAP memory-mapped registers. There also might be some synchronization issues in the way interrupts are triggered on the EE.

That isn't to say that there aren't bigger problems in smap.c that could have something to do with the IOP ps2smap.irx. However the "workaround" Sony does in smap.c is definitely not a real fix, and not related to your problem.

BTW, there is a WIP SMAP driver in ps2drv under iop/ps2smap/. It is a completely different approach than the ps2smap we use today, and I believe it handles autonegotiation a wee-bit differently (don't know if it would help though). Unfortunately it's not ready for primetime.

boman666 · Post by **boman666** » Wed Mar 10, 2004 6:44 am

It seems as the delay was caused by connecting the PS2 to my PC with a cross over cable. After connecting the PS2 to a hub or switch, the delay went away.

I'm still plagued by less than stellar transferspeed though, about 6 mbit/s. I'd expect ps2ip to perform better than that or is it whishful thinking on my behalf?

Post by **mrbrown** » Wed Mar 10, 2004 7:29 am

768KB/s is a very nice transfer rate. Last I recall, the PS2 Linux Kit gave me 900KB/s.

boman666 · Post by **boman666** » Wed Mar 10, 2004 7:06 pm

Do you really think a 6% utilization is good? I'd expect it to be higher than that. I guess lwip is designed to be conservative with the resources at the expense of performance. I just didn't expect it to be that conservative. BTW: I got 2.5-3 MByte/s with the linuxkit.

Post by **mrbrown** » Wed Mar 10, 2004 9:42 pm

On the IOP, yes I think it's good. At the time I got 900KB/s under Linux, I was only on a 10Mbit network. I have a 100Mbit setup now, but I have'nt bothered to check what I can get.

Keep in mind that SMAP is tied to the IOP, a 36Mhz processor doing much more than just servicing SMAP interrupts. Sure, ps2ip could be made a bit faster, but 768KB is more than enough to get things done. A few versions ago, ps2ip could only get about 300-400KB/s, but we have Oobles to thank for speeding it up to where it is now.

boman666 · Post by **boman666** » Thu Mar 11, 2004 2:36 am

Ok, now I know that my numbers are in line with what to expect.

While we still are on the subject of network performance. I'm experiencing sluggish performance when I do remote debugging with the gdb-stub. The bandwith offered by ps2ip should be more than adequate to communicate with gdb. Since the data transmitted usually consist of many small packets, I've started to wonder if there is an latency issue with ps2ip. So before I start to do any latency tests, does anyone have any info to offer on the subject?

boman666 · Post by **boman666** » Thu Mar 11, 2004 5:09 am

I've just discovered that there is a huge discrepancy between upload- and download-speed. The upload-speed is only ~6 KBytes/s. Can anyone confirm my findings?

Post by **Oobles** » Thu Mar 11, 2004 8:21 am

I can't confirm your findings, however when I was doing my performance testing I was more interested in PC to PS2 speed for ps2link, etc than PS2 to PC speed. So what you say could be true.

ps2ip has not had a lot of work done on it in the last year and there are still some problems. The version most people are currently using is tagged PS2IP_062. Around May last year I did some work to try and increase performance however this seemed to introduce some more bugs which is why people arn't using it. I think the later version would be better for performance though.

Last week I committed the LWIP 0.7.1 code base as a branch in CVS. The plan was to go back and study the changes made since the PS2IP_062 release and see if the problems are due to LWIP or introduced.

On a side note.. The license on PS2IP is non-commercial. I have already stated to a number of people that I'd like to make PS2IP an AFL license and move it into PS2SDK. But currently we don't have anyone working on PS2SDK, so it's all a slow process.

I recognise there is still some problems with PS2IP, however I don't have time to work on PS2IP at the moment. I'd be happy to let hand over my knowledge to anyone who wants to have a go at updating it.

Major issues with ps2ip include:
- The select() call doesn't work because it needs a timed wait.
(this is also why the ps2dns - domain name resolver won't work)
- The latest LWIP 0.7.1 code needs to be modified slightly to work with ps2ip.
- The changes after PS2IP_062 need to be examined to see if any bugs were introduced.

Also related to ps2ip is the ps2smap driver:
- Look at how double buffering can be reduced. ( Actually, not sure if this is still an issue. )

Anyway.. that's the state of play with ps2ip. Hope it helps your understanding.

Oobles.

boman666 · Post by **boman666** » Wed Mar 17, 2004 1:40 pm

I've had a look at it now and I've found what I believe are the problems.

First, there is a bug, or bad behavior, in smap_send. When there isn't enough space in the TX-buffer, something which happens quite easily since the buffer is only 4KB, the packet is dropped silently. Since the message-thread in ps2ip thinks the packet was sent successfully, it waits for an ack that will never be received. After about 1 sec there is a timeout and the unacked packets will be transfered first in the unsent-queue and a re-transmit is performed. Only a couple of packets will be sent before the TX-buffer is flooded again and the timeout procedure will start all over.

Secondly, the IOP RTOS seems to have problem to quickly respond to events and signals when there are non-idle threads running, which makes the inter-thread based message-system that ps2ip and ps2smap uses, non-optimal. Typically, the smap-thread doesn't receive the event from the smap interrupthandler until the ps2ip-thread becomes idle. I tried increasing the priority of the smap-thread but that didn't have any noticable impact on the latency. There might be a way of getting the thread-switching to run smoother but with my limited knowledge of the RTOS, I didn't manage to achive that.

I've changed the behavior of smap_send and added an asynchronous, interrupt driven, send-queue and the upload speed has increased to about 75 KB/s, which is still far from what should be possible to achive.

Without redesigning the interface between ps2ip and smap, I can think of one obvious optimization to make and that is to copy the data from/to memory with DMA. However, I'm doubtful it'll have any major impact on the performance. AFAICT, what is really hurting the transferspeed is the latency. With only a 4KB TX-buffer it takes ~330 us to send the buffer @ 100MBit/s and according to my profiling the RTOS isn't up to the task, with the current design.

Post by **mrbrown** » Wed Mar 17, 2004 3:16 pm

There is an alternative smap driver in the ps2drv project that you may want to take a look at. It's not in a completely usable state, but I was attempting to make better use if IOP threading (truth be told, most of it was based on Sony's original SMAP driver, but at some point I had to hack at it to get it to work with ps2ip).

You're probably right as far as the RTOS not being up to the task. It's very difficult to guage exactly how bogged down the IOP will become in situations like this, since response time to threads depends on what drivers you have loaded. If you have padman loaded, you're guranteed to get interrupted every vblank, for example.

DMA will provide a speedup, but dev9 DMA is shared with atad, so as you start using the hard drive, I'd expect performance to drop. Unfortunately, the single dev9 interrupt is also shared between SMAP and the hard drive. Anyway, I hope you can get some ideas from ps2smap in ps2drv, and please keep posting with your progress and any questions you may have.

boman666 · Post by **boman666** » Thu Mar 18, 2004 6:01 am

Thanks for the input. I plan to redesign the interface between ps2ip and smap and let the interrupthandler post the message directly in the ps2ip message-queue. That should eliminate one context-switch aswell as the need for the smap-thread. Since I'll need to modify ps2ip, I'll probably dl LWIP 0.7.1 and work with that codebase instead. I'll let you know how it progress.

Post by **Oobles** » Thu Mar 18, 2004 8:17 am

As I mentioned. There is a branch in CVS for lwip 0.7.1. This is just a direct copy of the 0.7.1 lwip source with no modifications. The head of CVS has a number of modifications (particularly in the api directory) that are required.

The main change being a method to call the tcpip slow and fast timers. In 0.6.2 that most people use this is done with a callback from the video vblank handler. The head has a imo better method with a seperate thread adding a message to the tcpip queue. These changes need to be reimplemented in 0.7.1. One thing to be careful of with ps2ip is that its thread handling is not the best. This is iirc the reason why the timer thread is sending a message to the tcpip queue to handle the fast and slow timers.. because calling the functions directly may have some bad effects with context switching. Actually, now I think of it... if IOP only context switches when a thread actively gives up control, the placing the message on a queue is not needed???

In terms of smap. Is it possible to modify the buffer that smap writes it's data to be a lwip buffer directly. I believe currently, smap writes to a buffer which the driver then copies into a lwip buffer. It would be much faster to allocate the lwip buffer.. point smap at it.. get the interrupt that data has arrived and pass the buffer directly to lwip.

Nice work so far. I'll be happy to setup CVS write access for you so you can make your changes available. It might encourage me to find some time to help out.

Oobles.

Post by **mrbrown** » Thu Mar 18, 2004 10:06 am

Oobles wrote:Actually, now I think of it... if IOP only context switches when a thread actively gives up control, the placing the message on a queue is not needed???

Nope, the IOP will also preempt the running thread if an interrupt handler causes a context switch to a thread with a higher priority than the interrupted thread.

Oobles wrote:In terms of smap. Is it possible to modify the buffer that smap writes it's data to be a lwip buffer directly. I believe currently, smap writes to a buffer which the driver then copies into a lwip buffer. It would be much faster to allocate the lwip buffer.. point smap at it.. get the interrupt that data has arrived and pass the buffer directly to lwip.

The SMAP driver in ps2drv already does this.

mharris · Post by **mharris** » Sat Mar 20, 2004 3:56 pm

boman666 wrote:First, there is a bug, or bad behavior, in smap_send. When there isn't enough space in the TX-buffer, something which happens quite easily since the buffer is only 4KB, the packet is dropped silently. Since the message-thread in ps2ip thinks the packet was sent successfully, it waits for an ack that will never be received. After about 1 sec there is a timeout and the unacked packets will be transfered first in the unsent-queue and a re-transmit is performed. Only a couple of packets will be sent before the TX-buffer is flooded again and the timeout procedure will start all over.

I've forgotten most of the TCP low-level stuff, so I may be way off here...

Couldn't something be done by adjusting the TCP sliding window size -- i.e., make it small, so the remote host doesn't send more than two or three packets without receiving an ACK? This would hopefully eliminate the buffer overflows and get rid of the resend timeouts.

Post by **blackdroid** » Sat Mar 20, 2004 9:48 pm

it sure sounds like ps2ip is advertising being able to cope with more than it can, making sure the window size is proper should eliminate dropped tcp packets.

boman666 · Post by **boman666** » Thu Mar 25, 2004 12:18 pm

mharris wrote:Couldn't something be done by adjusting the TCP sliding window size -- i.e., make it small, so the remote host doesn't send more than two or three packets without receiving an ACK? This would hopefully eliminate the buffer overflows and get rid of the resend timeouts.

I'm by no means a TCP-expert, actually this is my first encounter with TCP/IP- and network programming, but I think you're missunderstanding my message. The problem is related to the TX-buffer when sending data, not the RX-buffer when receiving data.

I guess it's feasible to eliminate the packet dropping by specifying the send window or some of the buffers small enough, for instance the send-buffer, but not without compromising the transferspeed.

mharris · Post by **mharris** » Thu Mar 25, 2004 1:27 pm

boman666 wrote:... I think you're missunderstanding my message. The problem is related to the TX-buffer when sending data, not the RX-buffer when receiving data.

Oops, sorry, you're right; I was assuming there was a receive buffer overrun. I'll read more carefully next time...

boman666 · Post by **boman666** » Thu Mar 25, 2004 2:35 pm

Ok, here's an update to what I've been upto since my last post.

I've moved ps2ip and smap in ps2eth into ps2drv since I got the impression that's the direction the homebrew development is heading. There already existed a ps2smap in ps2drv/iop so I named the one I've been working with to ps2smap2.

I've updated lwip to 0.7.1, even though I didn't use the 0.7.1 branch from CVS. My experience with CVS is very limited and I don't know how to list/view the different branches available. Oobles, wrote it was a direct copy so I got hold of another copy on the internet instead.

Besides the minor changes I had to make to get ps2ip to work with 0.7.1, I've made some changes to sys_arch.c aswell. My goal was to get lwip to behave better, for instance when a connection is lost. I think the timeout handling now is implemented as it was intented, though I haven't had the time to test if the behavior has improved. There were some minor bugs in the post-/fetchmessage functions which I removed.

All the manipulation of the lwip-objects are done in the context of the tcpip-thread which eliminates the need for additional synchronization objects.

I've also tried to reduce the memory usage of ps2ip, by changing some of the defines in lwipopts.h. I don't know how well this will work in general, but it works fine for me so far.

The smap-thread is gone and the interrupthandler posts the inputmsg directly in the tcpip-msgbox.

The receivebuffer is gone and the received data is stored directly in a pbuf, which is passed on to ps2ip. Similar to how ps2smap works.

I also omit the copying of the tx-data to the intermediary tx-buffer, if it's possible ie. the data is located in one contiguous block that is properly aligned.

With those changes the downloadspeed is ~675kb/s for a 18MB file copied to the ps2 hd. The uploadspeed is ~800kb/s when the same file is uploaded from the hd. I get ~415kb/s dl- / ~6kb/s ul-speed when using old ps2ip/ps2smap.

I still don't use DMA when copying to/from the EMAC, because there probably is some other bottleneck limiting the performance. This assumption is based on the fact that no speed improvement was made when I removed the unnecessary copying to/from the rx-buffer. I'd like to take the time to sit down and do some thorough profiling, but that's not possible due to time constraints on my behalf. Maybe I'll give it another go in the future.

If the HB community is interested in the changes I've made, I don't mind sharing them with you.

Post by **mrbrown** » Thu Mar 25, 2004 4:40 pm

Wow, incredible! I'd bet it's even faster when not using the HDD at all! There's one thing I have to mention though. While Oobles has stated his intention of relicensing ps2ip under the AFL, the ps2smap in ps2eth is covered under the GPL, and I've stated in the past that I can't accept GPL'd code in ps2drv. So as long as ps2smap stays in ps2eth, that's fine.

Do you have a URL to your changes? Would you like for one of us to import them into CVS?

boman666 · Post by **boman666** » Fri Mar 26, 2004 10:00 am

Yes, it would save me a lot of hassle if one of you guys could could import it to CVS. Unfortunately, I don't have a URL, how do we solve that?

Post by **Drakonite** » Fri Mar 26, 2004 11:35 am

boman666 wrote:Yes, it would save me a lot of hassle if one of you guys could could import it to CVS. Unfortunately, I don't have a URL, how do we solve that?

If you can arrange a way to post it we can accomidate you ;)

If it'd be easiet for you, as long as it's under 10mb (so it will fit in my inbox) you can put it all in a zip/rar/bzip/whatever archive and email it to me at "makeshift _ ps2dev @ 123mail . org" (remove the spaces) and I'll make sure it gets taken care of.