Panic Mode On (72) Server problems?

Message boards : Number crunching : Panic Mode On (72) Server problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 5 · 6 · 7 · 8 · 9 · Next

AuthorMessage
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 1211348 - Posted: 29 Mar 2012, 0:47:50 UTC - in response to Message 1211309.  

I'm sure there's plenty of astropulses going out but my estimates have gone from 280hrs with the first V6, the second was 264hrs and with the third that just finished downloading 241hrs. Phew, it's gonna be a while for things to get back to normal (cache wise) for me :)

Agreed. Mine started at 209 hours and after doing six of them, they are now down to 192. 10-day cache is 3 running + 3 waiting to run. The ETAs should start dropping a bit faster after 10 are completed though. I know they did on this new machine back in January for v505. First dozen or so APs were all 200+ and then the next one I was issued after the 10th one validated was down in the 40 range and took about 20 or so to come down the rest of the way to ~13.

The difference now is that it'll take longer to get the completed count up to 10 since only validated tasks which ran full length and had less than 10% blanking go into the average. Based on our observations of v505 work showing about 1 in 4 having higher blanking plus the early exits, at 14 validated there would be about a 50/50 chance of having 10 in the average. But the protections ought to mean after 10 completed the estimates will be fairly close.

For someone running only AP, DCF will have gone down while getting there, but in a way it's fortunate the initial estimates are so high. Whenever the runtime is less than 1/10 the estimate, DCF only goes down a little.
                                                                  Joe
ID: 1211348 · Report as offensive
B-Man
Volunteer tester

Send message
Joined: 11 Feb 01
Posts: 253
Credit: 147,366
RAC: 0
United States
Message 1211349 - Posted: 29 Mar 2012, 0:51:21 UTC - in response to Message 1211331.  

Unfortunately, my vacation is not going too well yet.

Found a flat tire on the car and it was determined that a new tire was needed instead of fixing. That took most of the day and $118.

I plan on leaving in the morning now.

Better luck in the morning. Have fun on your vacation.
ID: 1211349 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1211392 - Posted: 29 Mar 2012, 3:37:52 UTC - in response to Message 1211348.  

The difference now is that it'll take longer to get the completed count up to 10 since only validated tasks which ran full length and had less than 10% blanking go into the average. Based on our observations of v505 work showing about 1 in 4 having higher blanking plus the early exits, at 14 validated there would be about a 50/50 chance of having 10 in the average. But the protections ought to mean after 10 completed the estimates will be fairly close.

For someone running only AP, DCF will have gone down while getting there, but in a way it's fortunate the initial estimates are so high. Whenever the runtime is less than 1/10 the estimate, DCF only goes down a little.

And so far what I've seen in my handy spreadsheet is that of the 11 APs that I have crunched and returned with r557, one was a B3_P1 with 100% blanking, and only three of the other 10 had less than 10% blanked.

It does of course depend on the data itself, but a very early and large margin-of-error estimate is about 1 in 3 have <10% based on my data.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1211392 · Report as offensive
Profile Slavac
Volunteer tester
Avatar

Send message
Joined: 27 Apr 11
Posts: 1932
Credit: 17,952,639
RAC: 0
United States
Message 1211397 - Posted: 29 Mar 2012, 3:43:19 UTC - in response to Message 1211392.  

Getting quite a lot of shorties though I don't mind, I enjoy crunching them.


Executive Director GPU Users Group Inc. -
brad@gpuug.org
ID: 1211397 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1211433 - Posted: 29 Mar 2012, 6:15:25 UTC - in response to Message 1211397.  

Getting quite a lot of shorties though I don't mind, I enjoy crunching them.

The kitties luv shorties too. Just as long as the servers can keep up and keep the limited kibble bowls full.
The Frozen 920 has about 1560 GPU tasks in cache...right now about 1150 of them are shorties. It would take about 9-1/2 hours for it to crunch all of the shorties if the servers went down.
Hence my wish for an increase or lifting of the current limits.

"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1211433 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1211442 - Posted: 29 Mar 2012, 7:08:03 UTC - in response to Message 1211433.  


Uploads were a bit iffy for a while there, but they seem to be going through now. Eventually.
Grant
Darwin NT
ID: 1211442 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1211459 - Posted: 29 Mar 2012, 8:29:11 UTC - in response to Message 1211442.  


Uploads were a bit iffy for a while there, but they seem to be going through now. Eventually.

Make that uploads are still very iffy.
Looking at the network traffic, inbound traffic is looking a bit jagged. Things aren't good in upload land.
Grant
Darwin NT
ID: 1211459 · Report as offensive
Profile Brother Frank

Send message
Joined: 10 Dec 11
Posts: 26
Credit: 15,142,410
RAC: 0
United States
Message 1211564 - Posted: 29 Mar 2012, 15:47:59 UTC - in response to Message 1210940.  

I've been seeing this too along with many long backoff's on my two best crunchers. Then I retry for both downloads and uploads and it will all get through. I thought things were returning to near normal on Tuesday evening but then it go much worse again on Wednesday. This maintenance period has been a nightmare for my uploads. Even with my slowest machine I have had to manually update and retry a great deal. I joined just a few months ago, but this has not happened before for me except for that really bad period around the 2nd, 3rd, and 4th weeks in Feb.

Brother Frank
ID: 1211564 · Report as offensive
Profile cliff
Avatar

Send message
Joined: 16 Dec 07
Posts: 625
Credit: 3,590,440
RAC: 0
United Kingdom
Message 1211572 - Posted: 29 Mar 2012, 16:00:18 UTC - in response to Message 1211459.  
Last modified: 29 Mar 2012, 16:00:45 UTC

Things aint to good in d/l land either, every single WU download has had to be retried a min of 3 times..

Not a great problem during my waking hours, but on waking to find both rigs in project backoffs etc is getting a tad wearing:-)

I now find my waking hours are somewhat altered, up at between 03:00 and 04:00hrs, back to kip at 06:00, awake again at 09:30hrs.. Kip again about 23:59hrs.. and so on..

I request a public warning be posted:- SETI@HOME can be ADDICTIVE :-)

Regards,
Cliff,
Been there, Done that, Still no damm T shirt!
ID: 1211572 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1211580 - Posted: 29 Mar 2012, 16:27:34 UTC - in response to Message 1211572.  

Things aint to good in d/l land either, every single WU download has had to be retried a min of 3 times..

Not a great problem during my waking hours, but on waking to find both rigs in project backoffs etc is getting a tad wearing:-)

I now find my waking hours are somewhat altered, up at between 03:00 and 04:00hrs, back to kip at 06:00, awake again at 09:30hrs.. Kip again about 23:59hrs.. and so on..

I request a public warning be posted:- SETI@HOME can be ADDICTIVE :-)

Regards,

LOL...you are talking to the KING of Seti induced bad sleep habits.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1211580 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 1211632 - Posted: 29 Mar 2012, 18:53:52 UTC - in response to Message 1211392.  

The difference now is that it'll take longer to get the completed count up to 10 since only validated tasks which ran full length and had less than 10% blanking go into the average. Based on our observations of v505 work showing about 1 in 4 having higher blanking plus the early exits, at 14 validated there would be about a 50/50 chance of having 10 in the average. But the protections ought to mean after 10 completed the estimates will be fairly close.
...

And so far what I've seen in my handy spreadsheet is that of the 11 APs that I have crunched and returned with r557, one was a B3_P1 with 100% blanking, and only three of the other 10 had less than 10% blanked.

It does of course depend on the data itself, but a very early and large margin-of-error estimate is about 1 in 3 have <10% based on my data.

Only one of your first 5 validated tasks went into the average, I hope that's just a Murphy's Law effect. As of last Monday, over 110000 AP v6 tasks from hosts running the stock Windows build had gone into the averages, but that discussion with Dr. korpela was on another subject and he didn't indicate how many total tasks were involved.

That one which did go into your average produced an APR very close to that you had for many AP v505 tasks, a good sign.
                                                                  Joe
ID: 1211632 · Report as offensive
Profile S@NL - XP_Freak

Send message
Joined: 10 Jul 99
Posts: 99
Credit: 6,248,265
RAC: 0
Netherlands
Message 1211646 - Posted: 29 Mar 2012, 19:21:16 UTC - in response to Message 1211572.  

Things aint to good in d/l land either, every single WU download has had to be retried a min of 3 times..


At the moment I can't download a single WU.

Goodbye Seti Classic
ID: 1211646 · Report as offensive
B-Man
Volunteer tester

Send message
Joined: 11 Feb 01
Posts: 253
Credit: 147,366
RAC: 0
United States
Message 1211772 - Posted: 30 Mar 2012, 4:31:25 UTC

Downloads and uploads are slow but looking at the pipe it not strange in any way.
ID: 1211772 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1211784 - Posted: 30 Mar 2012, 5:15:08 UTC - in response to Message 1211649.  

Same issue as many many times before. Direct connection to the download servers, and the download is totally borked. Finding a good proxy, preferrably on the U.S west coast, and downloading goes like a rocket.

WTF?

Since i edited my hosts file to connect only to the one server i haven't had any real issues downloading.
For the last day or 2 it's been a bit iffy- quite a bit slower than it has been & sometimes takes a couple of tries before it finally starts downloading.
Grant
Darwin NT
ID: 1211784 · Report as offensive
Topher

Send message
Joined: 23 May 99
Posts: 36
Credit: 12,655,451
RAC: 0
Australia
Message 1211787 - Posted: 30 Mar 2012, 5:33:52 UTC - in response to Message 1211784.  

Same issue as many many times before. Direct connection to the download servers, and the download is totally borked. Finding a good proxy, preferrably on the U.S west coast, and downloading goes like a rocket.

WTF?

Since i edited my hosts file to connect only to the one server i haven't had any real issues downloading.
For the last day or 2 it's been a bit iffy- quite a bit slower than it has been & sometimes takes a couple of tries before it finally starts downloading.

Same here, there are a few hiccups now and then but its mostly pretty smooth. Out of interest, which IP Address are you using? .13 or .18?
ID: 1211787 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1211790 - Posted: 30 Mar 2012, 5:45:07 UTC - in response to Message 1211787.  

Same issue as many many times before. Direct connection to the download servers, and the download is totally borked. Finding a good proxy, preferrably on the U.S west coast, and downloading goes like a rocket.

WTF?

Since i edited my hosts file to connect only to the one server i haven't had any real issues downloading.
For the last day or 2 it's been a bit iffy- quite a bit slower than it has been & sometimes takes a couple of tries before it finally starts downloading.

Same here, there are a few hiccups now and then but its mostly pretty smooth. Out of interest, which IP Address are you using? .13 or .18?

.13
Grant
Darwin NT
ID: 1211790 · Report as offensive
Topher

Send message
Joined: 23 May 99
Posts: 36
Credit: 12,655,451
RAC: 0
Australia
Message 1211810 - Posted: 30 Mar 2012, 6:54:39 UTC - in response to Message 1211790.  

Same issue as many many times before. Direct connection to the download servers, and the download is totally borked. Finding a good proxy, preferrably on the U.S west coast, and downloading goes like a rocket.

WTF?

Since i edited my hosts file to connect only to the one server i haven't had any real issues downloading.
For the last day or 2 it's been a bit iffy- quite a bit slower than it has been & sometimes takes a couple of tries before it finally starts downloading.

Same here, there are a few hiccups now and then but its mostly pretty smooth. Out of interest, which IP Address are you using? .13 or .18?

.13

Thanks, it seems like thats the best out of the two as my testing showed that to be the most stable.
ID: 1211810 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1211811 - Posted: 30 Mar 2012, 6:58:02 UTC - in response to Message 1211810.  


Thanks, it seems like thats the best out of the two as my testing showed that to be the most stable.

Stable? You can't HANDLE stable.......LOL.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1211811 · Report as offensive
Profile Graham Middleton

Send message
Joined: 1 Sep 00
Posts: 1517
Credit: 86,815,638
RAC: 0
United Kingdom
Message 1211841 - Posted: 30 Mar 2012, 8:26:14 UTC - in response to Message 1211811.  


Stable? You can't HANDLE stable.......LOL.


No - the horses get in the way, make piles, and are a bit too heavy to just lift out of the way! ;-)

Happy Crunching,

Graham

ID: 1211841 · Report as offensive
Profile S@NL - XP_Freak

Send message
Joined: 10 Jul 99
Posts: 99
Credit: 6,248,265
RAC: 0
Netherlands
Message 1211874 - Posted: 30 Mar 2012, 11:15:39 UTC - in response to Message 1211811.  

Stable as a brick

Goodbye Seti Classic
ID: 1211874 · Report as offensive
Previous · 1 . . . 5 · 6 · 7 · 8 · 9 · Next

Message boards : Number crunching : Panic Mode On (72) Server problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.