Panic Mode On (20) Server problems

Message boards : Number crunching : Panic Mode On (20) Server problems
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 8 · 9 · 10 · 11 · 12 · 13 · 14 . . . 15 · Next

AuthorMessage
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 918341 - Posted: 16 Jul 2009, 2:16:50 UTC - in response to Message 918297.  

I made a small change that tripled our upload bandwidth. The change is to quickly drop excess connections rather than let them wait in the TCP queue.

<image removed>

We were doing this last week, but apparently Matt turned it off. IDK why. Matt is on vaca now.

Clearly it helps. Thanks for linking to Cricket so we can watch how it changes.
ID: 918341 · Report as offensive
Profile TCP JESUS
Avatar

Send message
Joined: 19 Jan 03
Posts: 205
Credit: 1,248,845
RAC: 0
Canada
Message 918342 - Posted: 16 Jul 2009, 2:16:50 UTC - in response to Message 918335.  

I've not bothered to count all the wu's I have for uploading, but my total is down to about 230mb of space seti@home is using across my machines. I don't want to think about what all my machines at work have stored on them lol.


I was just looking through the message log and got excited....lol

On this machine, on a 22" monitor @ 1680x1050 res, I have noticed that I am averaging 1 FINISHED result per complete vertical feed of text on the screen.....lol.

It's gonna be a while, but atleast it's finally moving.
I am TCP JESUS...The Carpenter Phenom Jesus....and HAMMERING is what I do best!
formerly known as...MC Hammer.
ID: 918342 · Report as offensive
Profile Pappa
Volunteer tester
Avatar

Send message
Joined: 9 Jan 00
Posts: 2562
Credit: 12,301,681
RAC: 0
United States
Message 918350 - Posted: 16 Jul 2009, 3:21:44 UTC - in response to Message 918342.  
Last modified: 16 Jul 2009, 3:24:16 UTC

The interesting part is when it gets below critical mass and the requests work. Over the years have also found it interesting is how Boinc is better at "guessing" when to retry and be successful.


I've not bothered to count all the wu's I have for uploading, but my total is down to about 230mb of space seti@home is using across my machines. I don't want to think about what all my machines at work have stored on them lol.


I was just looking through the message log and got excited....lol

On this machine, on a 22" monitor @ 1680x1050 res, I have noticed that I am averaging 1 FINISHED result per complete vertical feed of text on the screen.....lol.

It's gonna be a while, but atleast it's finally moving.

Please consider a Donation to the Seti Project.

ID: 918350 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 918360 - Posted: 16 Jul 2009, 4:09:28 UTC

Well I see there are a lot of uploads going through (cricket, and server status page shows a large number of "received in last hour"), so I know uploads are going through. Every time I try, I get an instant "http error" and the exponential backoff occurs. I'll just leave networking suspended for more time.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 918360 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13731
Credit: 208,696,464
RAC: 304
Australia
Message 918382 - Posted: 16 Jul 2009, 5:15:49 UTC - in response to Message 918297.  

I made a small change that tripled our upload bandwidth. The change is to quickly drop excess connections rather than let them wait in the TCP queue.

Ah, that would explain why some of my uploads are going through, and others are timing out after a second or less.
Grant
Darwin NT
ID: 918382 · Report as offensive
Profile TCP JESUS
Avatar

Send message
Joined: 19 Jan 03
Posts: 205
Credit: 1,248,845
RAC: 0
Canada
Message 918385 - Posted: 16 Jul 2009, 6:09:01 UTC

AAhhhh......Satisfaction.

850 WU's Up...and 1500 WU's back down and I am all ready to go once again.

Finally satisfied.
I am TCP JESUS...The Carpenter Phenom Jesus....and HAMMERING is what I do best!
formerly known as...MC Hammer.
ID: 918385 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65738
Credit: 55,293,173
RAC: 49
United States
Message 918386 - Posted: 16 Jul 2009, 6:28:39 UTC - in response to Message 918297.  

Yeah really love It Eric. :o Hope You get the fiber You need soon, As this place needs an enema(something to make the place runnnnn).
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 918386 · Report as offensive
Profile [B^S] madmac
Volunteer tester
Avatar

Send message
Joined: 9 Feb 04
Posts: 1175
Credit: 4,754,897
RAC: 0
United Kingdom
Message 918388 - Posted: 16 Jul 2009, 6:37:08 UTC

When I switched on the comp this morning one went up and the others http errors so I will now just wait until the others go up and hopefully have some downloads. My other Boinc projects will just have to carry on working.
ID: 918388 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 918396 - Posted: 16 Jul 2009, 8:07:56 UTC

Well this turned out kind of not how I would have expected. People got enough of their uploads sent in and started hammering the servers for downloads, so now there's no bandwidth for the uploads to get through.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 918396 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 918397 - Posted: 16 Jul 2009, 8:16:02 UTC

Uploads are getting through, just slowly as there are still a lot of machines trying to upload finished work.

ID: 918397 · Report as offensive
Profile tullio
Volunteer tester

Send message
Joined: 9 Apr 04
Posts: 8797
Credit: 2,930,782
RAC: 1
Italy
Message 918398 - Posted: 16 Jul 2009, 8:16:26 UTC
Last modified: 16 Jul 2009, 8:33:16 UTC

A wise Venetian proverb says "el tacon xe pezo del buso" which means the mend is worse that the gash.
Tullio
ID: 918398 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13731
Credit: 208,696,464
RAC: 304
Australia
Message 918400 - Posted: 16 Jul 2009, 8:50:22 UTC - in response to Message 918398.  


Enough of my uploads went through for it to start asking for more work, but the cache still isn't full as i hit the daily WU limit. And i notice a lot of the WU are shorties again.
Although it looks like AP has almost done it's last disk, so in a few hours that'll cause the downloads to drop off significantly once that dries up again.
Grant
Darwin NT
ID: 918400 · Report as offensive
Profile [AF>Libristes] erik
Volunteer tester
Avatar

Send message
Joined: 30 Jul 07
Posts: 19
Credit: 4,016,114
RAC: 0
France
Message 918405 - Posted: 16 Jul 2009, 10:18:15 UTC - in response to Message 918240.  

[quote]erik

Just let it run (if you normally do overnight). It will recover with time.


It's now fine. Uploads are normal for me now.

Thanks for your response.
just a poet
ID: 918405 · Report as offensive
Profile Keith T.
Volunteer tester
Avatar

Send message
Joined: 23 Aug 99
Posts: 962
Credit: 537,293
RAC: 9
United Kingdom
Message 918440 - Posted: 16 Jul 2009, 13:33:53 UTC
Last modified: 16 Jul 2009, 13:57:39 UTC

I have a slightly different upload problem. All but one of my stuck uploads have cleared, except for http://setiathome.berkeley.edu/result.php?resultid=1302869985.

BoincView log shows that task completed 14/07/09 23:35 and I have completed and uploaded 7 other tasks since. For some reason this older task is still stuck even though more recent ones have cleared.

I think a similar problem has happened before. It may be linked to ISP, I use T-Mobile "mobile broadband". I will try an upload later when I have an alternate internet access through a public WiFi connetion.

Keith

[Edit]
Now 8 sucesses after this one stuck result.
ID: 918440 · Report as offensive
Profile [B^S] madmac
Volunteer tester
Avatar

Send message
Joined: 9 Feb 04
Posts: 1175
Credit: 4,754,897
RAC: 0
United Kingdom
Message 918473 - Posted: 16 Jul 2009, 16:05:26 UTC

I have had 3/4 of my wus uploaded hopefully I will start getting some downloads soon as I have a few of my other boinc project to do
ID: 918473 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 918479 - Posted: 16 Jul 2009, 16:34:00 UTC

A large percentage of my uploads have gone though. One box has gotten them all pushed through, but the others keep getting HTTP errors.

Looking at the cricket graphs, it is obvious where the AP splitters ran out of tapes and the upload line immediately goes up.

It seems as if we're at the point, presently, where there are no sockets available in Apache. No telling how long this will last.

I suppose the good news is that luck of the draw pushed most of my shorties through, so I was able to report those before re-issues were reported.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 918479 · Report as offensive
Profile Jack Zhang
Volunteer tester
Avatar

Send message
Joined: 2 Jul 06
Posts: 206
Credit: 6,142,449
RAC: 0
Canada
Message 918503 - Posted: 16 Jul 2009, 18:46:38 UTC - in response to Message 918479.  

Can't get any work. Looks like the splitters aren't fast enough, no work units being sent.
What if Fiction was Fact and Fact was Fiction and vice versa?
ID: 918503 · Report as offensive
Profile Evil Decepticon
Avatar

Send message
Joined: 12 Jul 09
Posts: 23
Credit: 15,847
RAC: 0
Spain
Message 918506 - Posted: 16 Jul 2009, 18:58:04 UTC - in response to Message 914643.  

I had the same problem, but now the server works correctly
ID: 918506 · Report as offensive
Profile TCP JESUS
Avatar

Send message
Joined: 19 Jan 03
Posts: 205
Credit: 1,248,845
RAC: 0
Canada
Message 918514 - Posted: 16 Jul 2009, 19:20:13 UTC

I have only gotten about 20-25 new MB WU's downloaded in the last hour or so, since I turned the scheduler back on after last night's downloads maxing out @ 1500 WU's.

Hopefully things pickup soon as everyone's cache get's filled up again after the 'speedbump' over the previous couple of days.
I am TCP JESUS...The Carpenter Phenom Jesus....and HAMMERING is what I do best!
formerly known as...MC Hammer.
ID: 918514 · Report as offensive
Profile Shannock9
Avatar

Send message
Joined: 4 Jul 99
Posts: 1396
Credit: 634,964
RAC: 0
United Kingdom
Message 918533 - Posted: 16 Jul 2009, 20:22:47 UTC
Last modified: 16 Jul 2009, 20:23:57 UTC

Within the last 30 mins all my WU's uploaded and I got new WU (all MB, no AP).

I only have CPU's here, no CUDA, because I only buy laptops (for non BOINC reasons)

ID: 918533 · Report as offensive
Previous · 1 . . . 8 · 9 · 10 · 11 · 12 · 13 · 14 . . . 15 · Next

Message boards : Number crunching : Panic Mode On (20) Server problems


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.