Panic Mode On (93) Server Problems?

Message boards : Number crunching : Panic Mode On (93) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 12 · 13 · 14 · 15 · 16 · 17 · 18 . . . 24 · Next

AuthorMessage
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1617893 - Posted: 23 Dec 2014, 16:27:02 UTC - in response to Message 1617888.  

One more hour should place Eric and Matt in the Colo. I'll be crossing my fingers that they find the right spot to tickle the servers to get them running properly again.

That was yesterday. Today is Outrage Day ;)
ID: 1617893 · Report as offensive
Aurora Borealis
Volunteer tester
Avatar

Send message
Joined: 14 Jan 01
Posts: 3075
Credit: 5,631,463
RAC: 0
Canada
Message 1617899 - Posted: 23 Dec 2014, 16:42:57 UTC - in response to Message 1617893.  

One more hour should place Eric and Matt in the Colo. I'll be crossing my fingers that they find the right spot to tickle the servers to get them running properly again.

That was yesterday. Today is Outrage Day ;)

You're right. Comfusion on my part. Maybe, they can add Jeff to the mix today. I'll still be crossing my fingers.
ID: 1617899 · Report as offensive
BetelgeuseFive Project Donor
Volunteer tester

Send message
Joined: 6 Jul 99
Posts: 158
Credit: 17,117,787
RAC: 19
Netherlands
Message 1617909 - Posted: 23 Dec 2014, 17:13:12 UTC

Up- and downloads seem to be working again.
Fingers crossed ...

Tom
ID: 1617909 · Report as offensive
WezH
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 576
Credit: 67,033,957
RAC: 95
Finland
Message 1617912 - Posted: 23 Dec 2014, 17:14:32 UTC - in response to Message 1617909.  

Yep, and SSP has been updated, it shows that download server 1, georgem, is disabled.
"Please keep Your signature under four lines so Internet traffic doesn't go up too much"

- In 1992 when I had my first e-mail address -
ID: 1617912 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11361
Credit: 29,581,041
RAC: 66
United States
Message 1617933 - Posted: 23 Dec 2014, 22:40:37 UTC

During today's outage 1 of my crunchers ran dry on the GPU and is now crunching Einstein, that is cause for panic!
ID: 1617933 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 1617956 - Posted: 23 Dec 2014, 23:14:49 UTC - in response to Message 1617933.  
Last modified: 23 Dec 2014, 23:17:08 UTC

During today's outage 1 of my crunchers ran dry on the GPU and is now crunching Einstein, that is cause for panic!


Both of mine have been doing that for upwards of 12 hours now. At one point, got a few MBs for the GPUs, but that was a mirage, it seems. No mas!

I'm going away tomorrow until New Year's (visiting my 2 daughters in New Orleans); thinking of powering down for the interim, as it doesn't look like things are going to get fixed soon...I'll check it out tomorrow morning and decide.
ID: 1617956 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1618007 - Posted: 24 Dec 2014, 0:29:47 UTC

I have (the last I looked) ten download errors. Whatever was wrong with the server(s) still is.

My question is, why do these go in my stats as errors instead of it just trying the download again?
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1618007 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11361
Credit: 29,581,041
RAC: 66
United States
Message 1618060 - Posted: 24 Dec 2014, 2:06:52 UTC - in response to Message 1617956.  
Last modified: 24 Dec 2014, 2:18:37 UTC

During today's outage 1 of my crunchers ran dry on the GPU and is now crunching Einstein, that is cause for panic!


Both of mine have been doing that for upwards of 12 hours now. At one point, got a few MBs for the GPUs, but that was a mirage, it seems. No mas!


I agree nada, I'm on my 3rd BRP Einstein task, no joy.
The great dreaded
12/23/2014 6:16:35 PM | SETI@home | Tasks for AMD/ATI GPU are available, but your preferences are set to not accept them

ID: 1618060 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13731
Credit: 208,696,464
RAC: 304
Australia
Message 1618077 - Posted: 24 Dec 2014, 3:12:13 UTC - in response to Message 1618060.  

MB splitters show as running, unfortunately they're still only producing a bit more than 1/3rd of the work they usually do.
Not enough to fill the ready to send buffer, not enough to even meet normal demand, let alone the demand after an outage.
One of my systems was lucky enough to get enough work to get it through the "Project has no tasks available" responses. The other has run out of work. Combination of "Project has no tasks available" when it did request work, and the rapid escalation of the backoff times when it missed out on getting any.
Grant
Darwin NT
ID: 1618077 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13731
Credit: 208,696,464
RAC: 304
Australia
Message 1618211 - Posted: 24 Dec 2014, 9:39:05 UTC - in response to Message 1618153.  

Still getting multiple splitters on the one file, and AP channels ending in error.
At least the MB output has picked up. We may yet get a ready-to-send buffer.
Grant
Darwin NT
ID: 1618211 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1618269 - Posted: 24 Dec 2014, 12:21:15 UTC

Thanks whomever gave me the idea, you saved me money.
The past couple of weeks the PC next to our TV would start up with a loud noise, a fan was definitely not happy to start from cold. It was difficult for me to figure out which fan it was, the CPU fan or the PSU fan, because each time as soon as it hit Windows desktop the noise would stop. So no use installing Speedfan or another app as by the time they had started, the noise would be gone.

I had to take the thing apart a moment ago anyway, as the AMD HD6850 in it ran 52C at idle, a tad too much. So after cleaning that out and cleaning the CPU heat sink, I took my spray can of WD40, put the straw in, wrapped the CPU fan up in a paper towel and sprayed the tiniest bit of WD40 in the opening between the fan head and the spindle. Turned the fan a couple of times, repeated the oiling, then put the system together again.

Immediately upon starting up the difference was noticeable. The whole system starts up ultra quiet now. So thanks to you, unknown person, for telling on these forums that you would oil your fan(s) regularly as well. You made me a believer. :)
ID: 1618269 · Report as offensive
Profile ivan
Volunteer tester
Avatar

Send message
Joined: 5 Mar 01
Posts: 783
Credit: 348,560,338
RAC: 223
United Kingdom
Message 1618275 - Posted: 24 Dec 2014, 12:47:04 UTC - in response to Message 1618269.  

I had to take the thing apart a moment ago anyway, as the AMD HD6850 in it ran 52C at idle, a tad too much. So after cleaning that out and cleaning the CPU heat sink, I took my spray can of WD40, put the straw in, wrapped the CPU fan up in a paper towel and sprayed the tiniest bit of WD40 in the opening between the fan head and the spindle. Turned the fan a couple of times, repeated the oiling, then put the system together again.

Immediately upon starting up the difference was noticeable. The whole system starts up ultra quiet now. So thanks to you, unknown person, for telling on these forums that you would oil your fan(s) regularly as well. You made me a believer. :)

Just remember that WD40 is not a lubricant, it is a Water Displacer (not surprisingly, the 40th formulation the US Navy tested...). Don't rely on it for long-term service, though it usually works as a short-term solution. [When you're in an aged Mini, coming back from London, Ont., to catch a flight from Toronto back to Switzerland, in a streaming sleet-storm, you tend to be very glad that the service-station you limped into on less than three cylinders has WD40 for sale, to spray on your spark-plug caps, leads, and the distributor. DAMHIK,IJD-OK?]
ID: 1618275 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1618329 - Posted: 24 Dec 2014, 16:38:19 UTC - in response to Message 1618007.  

I have (the last I looked) ten download errors. Whatever was wrong with the server(s) still is.

My question is, why do these go in my stats as errors instead of it just trying the download again?

Well, I didn't get any more of these. That's a good thing.
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1618329 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1618364 - Posted: 24 Dec 2014, 17:59:35 UTC - in response to Message 1618269.  

Thanks whomever gave me the idea, you saved me money.
The past couple of weeks the PC next to our TV would start up with a loud noise, a fan was definitely not happy to start from cold. It was difficult for me to figure out which fan it was, the CPU fan or the PSU fan, because each time as soon as it hit Windows desktop the noise would stop. So no use installing Speedfan or another app as by the time they had started, the noise would be gone.

I had to take the thing apart a moment ago anyway, as the AMD HD6850 in it ran 52C at idle, a tad too much. So after cleaning that out and cleaning the CPU heat sink, I took my spray can of WD40, put the straw in, wrapped the CPU fan up in a paper towel and sprayed the tiniest bit of WD40 in the opening between the fan head and the spindle. Turned the fan a couple of times, repeated the oiling, then put the system together again.

Immediately upon starting up the difference was noticeable. The whole system starts up ultra quiet now. So thanks to you, unknown person, for telling on these forums that you would oil your fan(s) regularly as well. You made me a believer. :)

Fun fact: if you take the fan out, get a razor blade and peel the sticker back on the back side (touching as little of the sticky surface of the sticker as possible..so that it will re-stick properly), and pop the little rubber plug out of the middle, it gives you direct access to the bearings. Put a single drop of motor oil (like for instance, 10w-40) in there, wipe any spill up from the surface where the sticker goes, put the rubber plug back in, put the sticker back on, and you're good to go.

I've oiled at least 50 fans like this and it works amazingly. The rear fan in my Armor case has been oiled five times now. Sleeve bearings wear out much sooner than ball bearings do, but if you oil them shortly after they start making noise, you don't damage the bearings..and sleeves will get worn and damaged much faster than ball-bearings will once they do start making noise, so you have to jump right on it.

I've also opened up a few power supplies and done those fans, as well as the really loud 40mm server-chassis fans. Oiling is a great way to get more life out of fans.. and to avoid having to figure out how to replace the custom-designed 3x40mm fan array with non-standard connectors on a 1U rackmount chassis without just buying a whole new chassis.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1618364 · Report as offensive
Ulrich Metzner
Volunteer tester
Avatar

Send message
Joined: 3 Jul 02
Posts: 1256
Credit: 13,565,513
RAC: 13
Germany
Message 1618370 - Posted: 24 Dec 2014, 18:12:27 UTC

That is exactly what i do to noisy fans, except, i use vaseline or ball bearing grease instead. It sticks in position better than oil and lasts 'forever' i. e. it lasts until the whole fan dies completely.

BTW:
Merry Christmas to everyone!
Aloha, Uli

ID: 1618370 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1618395 - Posted: 24 Dec 2014, 20:18:15 UTC

Well the AP creation rate is the highest I've seen it in a long time.. 1.14/sec. Sure beats 0.014/sec. Hopefully it stays above 1/sec for a while.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1618395 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13731
Credit: 208,696,464
RAC: 304
Australia
Message 1618424 - Posted: 24 Dec 2014, 22:15:58 UTC - in response to Message 1618408.  

MB output is still struggling, and that's with plenty of VLARs still in the mix. We're going to run out of work very quickly if a shortie storm comes along, even if we have a full ready-to-send buffer at the time, with such poor MB splitter output.

AP in progress is slowly (very, very slowly) increasing. It's now about 1/4 of what it generally gets up to. The AP assimilators still remain offline, so hopefully everything will continue to limp along as it is till next year when the lab has a chance to sort things out properly.
Grant
Darwin NT
ID: 1618424 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13731
Credit: 208,696,464
RAC: 304
Australia
Message 1618425 - Posted: 24 Dec 2014, 22:18:39 UTC - in response to Message 1618364.  

Sleeve bearings wear out much sooner than ball bearings do,

Sleeve bearings are the spawn of Satan.
As mentioned WD40 will get it going again, but light machine oil is best if you want to keep them running.
Grant
Darwin NT
ID: 1618425 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1618437 - Posted: 24 Dec 2014, 22:42:49 UTC - in response to Message 1618425.  

There's a good question (asked by Holly), can you use something like peanut oil or olive oil for this as well, since both of these can withstand very high temperatures?
ID: 1618437 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1618446 - Posted: 24 Dec 2014, 22:55:39 UTC
Last modified: 24 Dec 2014, 22:56:03 UTC

Simple sewing machine oil would be the best to use and the easiest to obtain. ;-)

Cheers.
ID: 1618446 · Report as offensive
Previous · 1 . . . 12 · 13 · 14 · 15 · 16 · 17 · 18 . . . 24 · Next

Message boards : Number crunching : Panic Mode On (93) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.