Sun Dies (Feb 22 2012)

Message boards : Technical News : Sun Dies (Feb 22 2012)
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1198597 - Posted: 22 Feb 2012, 21:34:44 UTC

So... another week another minor server crisis. This one was brewing for a while - we've been getting memory errors/upsets on our main internal file server (which hosts, among other things, all the files that make up the SETI@home web site). We got replacement memory, and were hoping for a quiescent moment to swap it out, but after two crashes in one day (on Tuesday) I just went ahead and did the swap.

So far so good (i.e. no further crashes), except we're still getting memory upsets in the server log. I only replaced 2 of the faulty DIMMs (which were noted as faulty by the motherboard), but maybe others need replacing as well.

In the meantime I found that project recovery today was significantly slowed by the result web pages on our site, so those are turned off at the moment (as I'm writing this).

Meanwhile other tasks this week included cleaning up the lab (the fire marshall is visiting today) and resurrecting SERENDIP code I haven't touched in over a decade. I got it to compile, now I'm just removing the non-fatal compiler warnings one by one. We'll use this code to help process Kepler data (which happens to be in a similar format to our old SERENDIP data). Maybe I'll even get back to analyzing the SERENDIP IV data set (also over a decade old and it may be worth taking another look at it with this code).

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1198597 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 21549
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1198599 - Posted: 22 Feb 2012, 21:39:48 UTC

Thanks for the update Matt.


I have a "cure" for fire marshals - pm me for info ;-)
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1198599 · Report as offensive
Profile Michel448a
Volunteer tester
Avatar

Send message
Joined: 27 Oct 00
Posts: 1338
Credit: 2,970,814
RAC: 0
Canada
Message 1198603 - Posted: 22 Feb 2012, 21:47:18 UTC

Thank you Matt for the update. extremly appreciated.
personnally i like to know whats happening, it helps to calm down some frustrations :)
ID: 1198603 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1198604 - Posted: 22 Feb 2012, 21:50:29 UTC - in response to Message 1198597.  

Thanks for the update Matt,

Claggy
ID: 1198604 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14571
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1198609 - Posted: 22 Feb 2012, 21:59:29 UTC

@ Matt - I've been getting random memory errors on my home server for the last month, and bluescreen lockups, even with ECC memory.

Turned out to be a voltage regulator failure on the motherboard (affecting the memory termination voltage only) - the same memory is working fine in a replacement motherboard.

Might be worth checking voltages, if that motherboard has the right degree of instrumentation.
ID: 1198609 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1198642 - Posted: 23 Feb 2012, 0:05:23 UTC - in response to Message 1198609.  

@ Matt - I've been getting random memory errors on my home server for the last month, and bluescreen lockups, even with ECC memory.

Turned out to be a voltage regulator failure on the motherboard (affecting the memory termination voltage only) - the same memory is working fine in a replacement motherboard.

Might be worth checking voltages, if that motherboard has the right degree of instrumentation.

It also might be worth looking into seeing what the voltages the power supply are putting out. As was discovered/documented in my lengthy capacitor replacement thread, whilst bulged caps were part of the problem, the root of the problem was the power supply. After opening it up, every single capacitor in there is bulged and nearly to burst/ooze stage.

While it isn't very feasible to open the power supply up, you should either be able to see what the board reports for the voltages, or just back-probe connectors with a voltmeter and see what the rails are putting out.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1198642 · Report as offensive
Profile Ronald R CODNEY
Avatar

Send message
Joined: 19 Nov 11
Posts: 87
Credit: 420,920
RAC: 0
United States
Message 1198998 - Posted: 23 Feb 2012, 21:37:55 UTC

Matt: Ur the MAN. Thanks for the communication(s)..
ID: 1198998 · Report as offensive
Mooncalf

Send message
Joined: 5 Jan 11
Posts: 19
Credit: 20,196,239
RAC: 0
United States
Message 1199014 - Posted: 23 Feb 2012, 21:59:02 UTC - in response to Message 1198597.  

For days I have languished over my falling RAC; I am finally reading here that all is finally well in the land of Oz. Mr. Wizard: why, oh why does the server status say all is well in Oz, but in reality no project server can be found, either via proxy or direct?
ID: 1199014 · Report as offensive
Profile Michel448a
Volunteer tester
Avatar

Send message
Joined: 27 Oct 00
Posts: 1338
Credit: 2,970,814
RAC: 0
Canada
Message 1199065 - Posted: 24 Feb 2012, 0:01:17 UTC - in response to Message 1199014.  

Mr. Wizard: why, oh why does the server status say all is well in Oz, but in reality no project server can be found, either via proxy or direct?


are you sure ? works very well since yesterday
ID: 1199065 · Report as offensive
davd

Send message
Joined: 20 May 03
Posts: 1
Credit: 1,551,912
RAC: 0
United States
Message 1199242 - Posted: 24 Feb 2012, 14:23:37 UTC - in response to Message 1198597.  

Is this why you have such a BIG problem keeping me full of SETI work? I've gone from 4,000+ units per day in late Jan (when I joined) thru mid Feb down to 2,400 and have often run compeletly out of work lately.
ID: 1199242 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1199275 - Posted: 24 Feb 2012, 17:02:08 UTC

"results ready to send" = 1, and the creation rate is below 1 per second, NTM that only four "tapes" are "hung"...
.

Hello, from Albany, CA!...
ID: 1199275 · Report as offensive
Wembley
Volunteer tester
Avatar

Send message
Joined: 16 Sep 09
Posts: 429
Credit: 1,844,293
RAC: 0
United States
Message 1199361 - Posted: 24 Feb 2012, 19:43:17 UTC - in response to Message 1198597.  

You getting a visit from this fire marshal?

http://www.youtube.com/watch?v=PlLPogmB8M8
ID: 1199361 · Report as offensive
Profile Michel448a
Volunteer tester
Avatar

Send message
Joined: 27 Oct 00
Posts: 1338
Credit: 2,970,814
RAC: 0
Canada
Message 1199413 - Posted: 24 Feb 2012, 21:20:32 UTC

ID: 1199413 · Report as offensive
Mooncalf

Send message
Joined: 5 Jan 11
Posts: 19
Credit: 20,196,239
RAC: 0
United States
Message 1199465 - Posted: 24 Feb 2012, 23:32:29 UTC - in response to Message 1199275.  

That is exactly my point!! I went from 75K+RAC/day to ZERO. "Project has no tasks available." This does not keep my 5 systems happy at all. Now it is Friday; all personel have left for the weekend; and SETI is sitting idle (it seems).
ID: 1199465 · Report as offensive
Profile Bill Walker
Avatar

Send message
Joined: 4 Sep 99
Posts: 3868
Credit: 2,697,267
RAC: 0
Canada
Message 1199482 - Posted: 25 Feb 2012, 0:12:39 UTC - in response to Message 1199465.  

Of course they left for the weekend. It is the weekend.

Downloading several WU right now.

ID: 1199482 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 31350
Credit: 261,360,520
RAC: 489
Australia
Message 1199484 - Posted: 25 Feb 2012, 0:15:09 UTC - in response to Message 1199465.  

That is exactly my point!! I went from 75K+RAC/day to ZERO. "Project has no tasks available." This does not keep my 5 systems happy at all. Now it is Friday; all personel have left for the weekend; and SETI is sitting idle (it seems).

Your main problem could be the BOINC version that you're using, check out the Top Hosts table and see what most of the main setups are using (work is flowing here at a good rate). ;)

Cheers.
ID: 1199484 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1199718 - Posted: 25 Feb 2012, 17:52:15 UTC - in response to Message 1199484.  
Last modified: 25 Feb 2012, 17:53:26 UTC

That is exactly my point!! I went from 75K+RAC/day to ZERO. "Project has no tasks available." This does not keep my 5 systems happy at all. Now it is Friday; all personel have left for the weekend; and SETI is sitting idle (it seems).

Your main problem could be the BOINC version that you're using, check out the Top Hosts table and see what most of the main setups are using (work is flowing here at a good rate). ;)

Cheers.


He's running 6.12.34, the most recent recommended version of the BOINC client... and well represented in the "top hosts" table
.

Hello, from Albany, CA!...
ID: 1199718 · Report as offensive
Profile cliff
Avatar

Send message
Joined: 16 Dec 07
Posts: 625
Credit: 3,590,440
RAC: 0
United Kingdom
Message 1199729 - Posted: 25 Feb 2012, 19:16:00 UTC

Hi Folks,
I've re-installed the Lunatics apps:-) And once again I'm back to haveing ALL GPU tasks running in high priority mode.

Not only S@H but also E@H.

This doesnt apparently occur with CPU tasks.

Is this 'normal' behaviour?

Also since boinc is configured to use 100% resources in both S@H and E@H it 'normally' works at 50% for each, now however its persistantly running E@H GPU tasks non stop. Unless I suspend E@H, let S@H utilse both CPU & GPU then resume E@H.. If I then suspend S@H to let E@H CPU tasks get a look in [they take a long time and expire faster than S@H ones] Then I'm back to square one with E@H hogging the GPU..

So how does one adjust the priority of the GPU tasks. I dont know if this HP mode is detrimental to my GPU cards or not.. But I'm anyway wondering why its gone into hyperdrive as soon as Lunatics is installed

Another thing, it doesnt pick up dumped tasks and restart them, it grabs a new task and does that..
I see several tasks with various % done waiting to be restarted..
I guess they will eventually get to the top of the queue but I suspect only after all other WU are done..

Regards,

Cliff,
Been there, Done that, Still no damm T shirt!
ID: 1199729 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 31350
Credit: 261,360,520
RAC: 489
Australia
Message 1199780 - Posted: 25 Feb 2012, 21:18:33 UTC - in response to Message 1199718.  

That is exactly my point!! I went from 75K+RAC/day to ZERO. "Project has no tasks available." This does not keep my 5 systems happy at all. Now it is Friday; all personel have left for the weekend; and SETI is sitting idle (it seems).

Your main problem could be the BOINC version that you're using, check out the Top Hosts table and see what most of the main setups are using (work is flowing here at a good rate). ;)

Cheers.


He's running 6.12.34, the most recent recommended version of the BOINC client... and well represented in the "top hosts" table

Funny that, I see more of the later 6.10.xx versions myself there.

Cheers.
ID: 1199780 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1200026 - Posted: 26 Feb 2012, 16:54:39 UTC - in response to Message 1199780.  

That is exactly my point!! I went from 75K+RAC/day to ZERO. "Project has no tasks available." This does not keep my 5 systems happy at all. Now it is Friday; all personel have left for the weekend; and SETI is sitting idle (it seems).

Your main problem could be the BOINC version that you're using, check out the Top Hosts table and see what most of the main setups are using (work is flowing here at a good rate). ;)

Cheers.


He's running 6.12.34, the most recent recommended version of the BOINC client... and well represented in the "top hosts" table

Funny that, I see more of the later 6.10.xx versions myself there.

Cheers.


yes, but 6.12.34 is in there - so it isn't a problem with the client version... There's just more 6.10.60 clients out there...

.

Hello, from Albany, CA!...
ID: 1200026 · Report as offensive
1 · 2 · Next

Message boards : Technical News : Sun Dies (Feb 22 2012)


 
©2023 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.