receive hangs; was TCP send() performance and eventual hangs

[zigg]

Just for fun, I built a libnds that clears the IRQ_TIMER3 line in REG_IE before bringing REG_IME up, then restores it at the end.

Code: Select all

--- libnds-src-1.3.6/source/common/fifosystem.c.orig	2009-06-22 16:18:40.000000000 -0400
+++ libnds-src-1.3.6/source/common/fifosystem.c	2009-07-07 09:14:43.000000000 -0400
@@ -426,7 +426,9 @@
 int processing=0;
 
 static void fifoInternalRecvInterrupt() {
-	REG_IE &= ~IRQ_FIFO_NOT_EMPTY;
+	/* save IRQ_TIMER3 and mask before reenabling interrupts */
+	int timer3_save = REG_IE & IRQ_TIMER3;
+	REG_IE &= ~(IRQ_FIFO_NOT_EMPTY | IRQ_TIMER3);
 	REG_IME=1;
 
 	u32 data, block=FIFO_BUFFER_TERMINATE;
@@ -544,6 +546,9 @@
 		REG_IME = 0;
 		processing = 0;
 	}
+
+	/* restore IRQ_TIMER3 */
+	REG_IE |= timer3_save;
 }
 
 static void fifoInternalSendInterrupt() {

I found I was needing to do significantly more retransmits due to lost ACKs (again, for those just tuning in, this is TFTP, which uses UDP, not TCP; ACKs are part of the TFTP protocol itself.) Since my timeout for this was 2s, I went back to select(), went for a 100ms timeout instead. Cleaned up the warnings while I was in there.

I'm getting throughput of over 40 KiB/s now, and that can be improved with a better TFTP blocksize and less printf'ing, I'm sure. I've transmitted 15 payloads thus far; I'm going to go set up a long-running test here in a minute to see if it will actually keep running or not... once I find my AC adapters and such.

The latest .nds (linked to my custom libnds, based on 1.3.6) is in this zip, along with my current code:

http://www.zigg.com/tmp/tftps-ds-test2.zip

[zigg]

Has been running for 30 minutes with 77 successfully transmitted 1 MiB payloads thus far.

elhobbs might be on to something.

[zigg]

60 minutes, 152 MiB transferred. I ended it myself. I think this is good enough for me to resume work on my code.

If there's anything I can do to help work out a proper fix, though, that'd be great.

I imagine the missed ACK increase I saw was due to an increase in missed TIMER3 interrupts. I imagine it would be more productive to figure out why they are resulting in corruption rather than just mask them entirely.

Is there a way to see the stack when it hangs? See what was running when the fifoIRI interrupt fired, then the TIMER3 update fired, assuming that's what's really going on? I only have a DS Lite, an R4, and an M3, so I'm not sure how to connect a debugger.

[elhobbs]

Wifi_Update on the arm9 does modify WifiData->rxbufIn while interrupts are enabled - which I suspect could lead to processing the same packet twice (maybe?).

Code: Select all

void Wifi_Timer(int num_ms) {
   Wifi_Update();
   sgIP_Timer(num_ms);
}

does Wifi_Timer need to call Wifi_Update anymore. I think this was needed before fifo was in place when it still used a shared memory region for IPC which I would imagine would lead to delays in processing incoming packets. would removing Wifi_Update from the the timer call lead to problems?

what about disabling timer3 interrupts during Wifi_Update?

or possibly an in_wifi_update volatile global to flag when it is already being processed?

[zigg]

Wifi_Update on the arm9 does modify WifiData->rxbufIn while interrupts are enabled

Aha...

Timer_50ms is an interrupt handler (and is so called with IME off), then the call chain goes Wifi_Timer(50), Wifi_Update. So IME should be off for this.

But Wifi_Update is also called by Wifi_Sync, by way of wifiValue32Handler, which is a fifo value32 handler. Which fifoIRI calls right after turning IME on, looks like.

(Which is pretty much what you said, I guess.)

I'll rebuild dswifi with at the very least a mask for IRQ_TIMER3 in Wifi_Update and see what's up.

EDIT: Actually, heck, I'll try it without Wifi_Update in Timer_50ms first. Good idea perhaps? EDIT #2: Well, now it's not associating, so I guess not

[elhobbs]

it looks like the state change code at the top of Wifi_Update needs to be called in the timer.

edit: fifo messages from the arm7 would probably be a better fit though.

[zigg]

IRQ_TIMER3 masking inside Wifi_Update is working, btw. Performance is about the same.

it looks like the state change code at the top of Wifi_Update needs to be called in the timer.

The code in the #ifdef WIFI_USE_TCP_SGIP?

Are you saying we could possibly call just that code inside the timer, and call the rest (maybe even minus that block) inside the fifo handler?

[elhobbs]

I think the packet receiving block

Code: Select all

	while(WifiData->rxbufIn!=WifiData->rxbufOut) {

should be skipped during the timer interrupt. I do not think anything needs to change for the sync updates - meaning still run the state change code all the time. though - as I said in the edit to my previous post I think a fifo message might be a better fit for the state changes.

[zigg]

I think the packet receiving block

Code: Select all

	while(WifiData->rxbufIn!=WifiData->rxbufOut) {

should be skipped during the timer interrupt. I do not think anything needs to change for the sync updates - meaning still run the state change code all the time. though - as I said in the edit to my previous post I think a fifo message might be a better fit for the state changes.

Rewriting that code to use a fifo is beyond me.

I should be able to rig up the behavior you mention otherwise though. It doesn't look like I can read REG_IF, though. So I'd have to either split Wifi_Update up or change its signature to take a flag indicating whether it should process packets or not.

EDIT: Although, just looking at it now, I don't know if that's a good idea. sgIP_Hub_AddHardwareInterface, called in the first group there, seems to be modifying HWInterfaces, a global array.

[elhobbs]

a flag parameter looks like the easiest approach.