Page 3 of 5
Re: TCP send() performance and eventual hangs
Posted: Tue Jul 07, 2009 2:40 pm
by zigg
Just for fun, I built a libnds that clears the IRQ_TIMER3 line in REG_IE before bringing REG_IME up, then restores it at the end.
Code: Select all
--- libnds-src-1.3.6/source/common/fifosystem.c.orig 2009-06-22 16:18:40.000000000 -0400
+++ libnds-src-1.3.6/source/common/fifosystem.c 2009-07-07 09:14:43.000000000 -0400
@@ -426,7 +426,9 @@
int processing=0;
static void fifoInternalRecvInterrupt() {
- REG_IE &= ~IRQ_FIFO_NOT_EMPTY;
+ /* save IRQ_TIMER3 and mask before reenabling interrupts */
+ int timer3_save = REG_IE & IRQ_TIMER3;
+ REG_IE &= ~(IRQ_FIFO_NOT_EMPTY | IRQ_TIMER3);
REG_IME=1;
u32 data, block=FIFO_BUFFER_TERMINATE;
@@ -544,6 +546,9 @@
REG_IME = 0;
processing = 0;
}
+
+ /* restore IRQ_TIMER3 */
+ REG_IE |= timer3_save;
}
static void fifoInternalSendInterrupt() {
I found I was needing to do significantly more retransmits due to lost ACKs (again, for those just tuning in, this is TFTP, which uses UDP, not TCP; ACKs are part of the TFTP protocol itself.) Since my timeout for this was 2s, I went back to select(), went for a 100ms timeout instead. Cleaned up the warnings while I was in there.
I'm getting throughput of over 40 KiB/s now, and that can be improved with a better TFTP blocksize and less printf'ing, I'm sure. I've transmitted 15 payloads thus far; I'm going to go set up a long-running test here in a minute to see if it will actually keep running or not... once I find my AC adapters and such.
The latest .nds (linked to my custom libnds, based on 1.3.6) is in this zip, along with my current code:
http://www.zigg.com/tmp/tftps-ds-test2.zip
Re: TCP send() performance and eventual hangs
Posted: Tue Jul 07, 2009 3:15 pm
by zigg
Has been running for 30 minutes with 77 successfully transmitted 1 MiB payloads thus far.
elhobbs might be on to something.
Re: TCP send() performance and eventual hangs
Posted: Tue Jul 07, 2009 3:49 pm
by zigg
60 minutes, 152 MiB transferred. I ended it myself. I think this is good enough for me to resume work on my code.
If there's anything I can do to help work out a proper fix, though, that'd be great.
I imagine the missed ACK increase I saw was due to an increase in missed TIMER3 interrupts. I imagine it would be more productive to figure out why they are resulting in corruption rather than just mask them entirely.
Is there a way to see the stack when it hangs? See what was running when the fifoIRI interrupt fired, then the TIMER3 update fired, assuming that's what's really going on? I only have a DS Lite, an R4, and an M3, so I'm not sure how to connect a debugger.
Re: TCP send() performance and eventual hangs
Posted: Tue Jul 07, 2009 4:50 pm
by elhobbs
Wifi_Update on the arm9 does modify WifiData->rxbufIn while interrupts are enabled - which I suspect could lead to processing the same packet twice (maybe?).
Code: Select all
void Wifi_Timer(int num_ms) {
Wifi_Update();
sgIP_Timer(num_ms);
}
does Wifi_Timer need to call Wifi_Update anymore. I think this was needed before fifo was in place when it still used a shared memory region for IPC which I would imagine would lead to delays in processing incoming packets. would removing Wifi_Update from the the timer call lead to problems?
what about disabling timer3 interrupts during Wifi_Update?
or possibly an in_wifi_update volatile global to flag when it is already being processed?
Re: TCP send() performance and eventual hangs
Posted: Tue Jul 07, 2009 5:06 pm
by zigg
elhobbs wrote:Wifi_Update on the arm9 does modify WifiData->rxbufIn while interrupts are enabled
Aha...
Timer_50ms is an interrupt handler (and is so called with IME off), then the call chain goes Wifi_Timer(50), Wifi_Update. So IME should be off for this.
But Wifi_Update is also called by Wifi_Sync, by way of wifiValue32Handler, which is a fifo value32 handler. Which fifoIRI
calls right after turning IME on, looks like.
(Which is pretty much what you said, I guess.)
I'll rebuild dswifi with at the very least a mask for IRQ_TIMER3 in Wifi_Update and see what's up.
EDIT: Actually, heck, I'll try it without Wifi_Update in Timer_50ms first. Good idea perhaps?
EDIT #2: Well, now it's not associating, so I guess not
Re: receive hangs; was TCP send() performance and eventual hangs
Posted: Tue Jul 07, 2009 5:42 pm
by elhobbs
it looks like the state change code at the top of Wifi_Update needs to be called in the timer.
edit: fifo messages from the arm7 would probably be a better fit though.
Re: receive hangs; was TCP send() performance and eventual hangs
Posted: Tue Jul 07, 2009 5:50 pm
by zigg
IRQ_TIMER3 masking inside Wifi_Update is working, btw. Performance is about the same.
elhobbs wrote:it looks like the state change code at the top of Wifi_Update needs to be called in the timer.
The code in the #ifdef WIFI_USE_TCP_SGIP?
Are you saying we could possibly call just that code inside the timer, and call the rest (maybe even minus that block) inside the fifo handler?
Re: receive hangs; was TCP send() performance and eventual hangs
Posted: Tue Jul 07, 2009 5:57 pm
by elhobbs
I think the packet receiving block
Code: Select all
while(WifiData->rxbufIn!=WifiData->rxbufOut) {
should be skipped during the timer interrupt. I do not think anything needs to change for the sync updates - meaning still run the state change code all the time. though - as I said in the edit to my previous post I think a fifo message might be a better fit for the state changes.
Re: receive hangs; was TCP send() performance and eventual hangs
Posted: Tue Jul 07, 2009 6:28 pm
by zigg
elhobbs wrote:I think the packet receiving block
Code: Select all
while(WifiData->rxbufIn!=WifiData->rxbufOut) {
should be skipped during the timer interrupt. I do not think anything needs to change for the sync updates - meaning still run the state change code all the time. though - as I said in the edit to my previous post I think a fifo message might be a better fit for the state changes.
Rewriting that code to use a fifo is beyond me.
I should be able to rig up the behavior you mention otherwise though. It doesn't look like I can read REG_IF, though. So I'd have to either split Wifi_Update up or change its signature to take a flag indicating whether it should process packets or not.
EDIT: Although, just looking at it now, I don't know if that's a good idea. sgIP_Hub_AddHardwareInterface, called in the first group there, seems to be modifying HWInterfaces, a global array.
Re: receive hangs; was TCP send() performance and eventual hangs
Posted: Tue Jul 07, 2009 6:46 pm
by elhobbs
a flag parameter looks like the easiest approach.