Fill 'er up (Jun 23 2010)

Message boards : Technical News : Fill 'er up (Jun 23 2010)
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1007541 - Posted: 23 Jun 2010, 21:40:02 UTC

Since last I wrote a lot has happened. Looking at the traffic graphs it's like feast or famine - either we are unable to create/send out workunits, or we're sending out as many as we can fit through the pipe. Mostly it's been the usual gremlins.

However regarding the past 24 hours it was a new problem: the result space on the upload server filled up unexpectedly, which would have been fine except this (perhaps) inspired some RAID freakout on the system. We couldn't really sort it out until this morning. From the looks of it we had something like a six drive simultaneous failure. Jeff and I beat on it for a while - we eventually assumed this was just a hardware blip, and the data was more or less intact on the drives, but the RAID metadata got a little screwed up. Long story short we were able to carefully bring down the RAID and recreate the meta devices from scratch with the data intact, and all was well. Phew. For the record we do have a virtually-up-to-date result storage backup at all times in case of catastrophic failure on this system.

In any case, the main culprit was our disks filling up, so as I write this we're keeping the project down until major queues drain and the constituent workunit/result files can be deleted.

On a more happy (perhaps) note, yesterday the core group of us were in the same place at the same time (which is rare) and we had an ad hoc meeting about our current project status/plans, especially in light of many recent server problems, increasingly random schedules, and embarrassingly low funding. We're all kind of tired and beaten up and wanting some results already - so I like to think this paved the way for several large and ultimately positive changes in the future.

Also Jeff has been working on this nagging mysterious problem where some of our raw data files are only getting partially processed (which vastly increases our "burn rate" and leads to unexpected workunit shortages). He found some major clues today, and we brainstormed why this is happening and what the exact effect is. At least there's a smoking gun on that front.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1007541 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1007545 - Posted: 23 Jun 2010, 21:48:23 UTC - in response to Message 1007541.  

Thanks for the update Matt,

Claggy
ID: 1007545 · Report as offensive
Profile Taxman59

Send message
Joined: 9 Feb 00
Posts: 13
Credit: 3,940,341
RAC: 3
United States
Message 1007546 - Posted: 23 Jun 2010, 21:48:42 UTC - in response to Message 1007541.  

Thanks for the update. You guys do a "Yeoman" job. I am curious, what would it cost for a new server? It seems that if the people who seem to be venting the most are those who haven't given cash. That is not to say they haven't donated time and electrical costs, but the project could use more of the cash. Maybe a "fundraiser" for a new server or two would shake loose a few dollars that may make your lives a bit easier.

Again thanks for the update and the day to day work you do to keep the project going.

ID: 1007546 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30592
Credit: 53,134,872
RAC: 32
United States
Message 1007547 - Posted: 23 Jun 2010, 21:51:01 UTC - in response to Message 1007541.  
Last modified: 23 Jun 2010, 21:57:23 UTC

Thanks for the excellent news item. Most of us are a patient lot and understand you are reduced to rock soup. Don't let the others get you down.
ID: 1007547 · Report as offensive
Profile Blurf
Volunteer tester

Send message
Joined: 2 Sep 06
Posts: 8962
Credit: 12,678,685
RAC: 0
United States
Message 1007548 - Posted: 23 Jun 2010, 21:52:28 UTC

I know you guys have your hands full-sometimes those sudden meetings are the best to have occurred.


ID: 1007548 · Report as offensive
Profile Byron Leigh Hatch @ team Carl Sagan
Volunteer tester
Avatar

Send message
Joined: 5 Jul 99
Posts: 4548
Credit: 35,667,570
RAC: 4
Canada
Message 1007553 - Posted: 23 Jun 2010, 22:00:53 UTC - in response to Message 1007541.  

Thank you Matt.
David A, Eric, Dan, Jeff, Josh Von Korff, Robert Bankay, Kevin Douglas.
You folks show outstanding dedication to SETI@home.
Best wishes and peace.
from Vancouver, Canada.
Byron.


ID: 1007553 · Report as offensive
KB7RZF
Volunteer tester
Avatar

Send message
Joined: 15 Aug 99
Posts: 9549
Credit: 3,308,926
RAC: 2
United States
Message 1007590 - Posted: 23 Jun 2010, 22:46:10 UTC

Matt, thank you very much for the update, it really shed's a lot of light on the recent problems. Here's hoping you guys get it running smooth again, or as smooth as you can for a bit. LOL Have a great rest of the week!
ID: 1007590 · Report as offensive
Profile Codeman05

Send message
Joined: 16 Dec 01
Posts: 33
Credit: 15,457,430
RAC: 0
United States
Message 1007653 - Posted: 24 Jun 2010, 1:52:38 UTC

Thank you for the update Matt and for all the hard work you and the team have put in!
ID: 1007653 · Report as offensive
ront

Send message
Joined: 25 Aug 01
Posts: 77
Credit: 386,336
RAC: 0
United States
Message 1007658 - Posted: 24 Jun 2010, 2:29:24 UTC

Thanks for the update. I, also, would like to know the cost of the type of server you need.

Again, thanks for informing us so promptly of the issues involved.

Be Blessed & Be A Blessing,

ront
ID: 1007658 · Report as offensive
Profile Todd Hebert
Volunteer tester
Avatar

Send message
Joined: 16 Jun 00
Posts: 648
Credit: 228,292,957
RAC: 0
United States
Message 1007666 - Posted: 24 Jun 2010, 2:54:12 UTC

Thanks Matt! I'm interested in donating to the cause for new hardware.
Please let us know.
Todd
ID: 1007666 · Report as offensive
MarkJ Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 08
Posts: 1139
Credit: 80,854,192
RAC: 5
Australia
Message 1007727 - Posted: 24 Jun 2010, 12:03:17 UTC
Last modified: 24 Jun 2010, 12:05:11 UTC

Hi Matt,

Thanks for the update and the great job you guys do to keep things running.

Since Blurf decided to reduce his involvement we don't seem to have anyone to coordinate hardware (or other) funding drives. Is is possible to get the list of required equipment updated so we have an idea of what is currently needed please.

Cheers
BOINC blog
ID: 1007727 · Report as offensive
BlackBiker
Volunteer tester

Send message
Joined: 14 Feb 00
Posts: 10
Credit: 5,912,242
RAC: 0
Message 1007732 - Posted: 24 Jun 2010, 13:26:24 UTC

Hi Guys,

On behalf of the SETI@Netherlands team I would like to thank you for all your hard work you are putting in.
No budget, no manpower and no luck. It's a bitch.
But please, don't get discouraged and keep up the good work.

When you've got the flow running again, we will again do our part in the search :)
ID: 1007732 · Report as offensive
Profile robin
Volunteer tester

Send message
Joined: 19 Nov 01
Posts: 1
Credit: 10,090,937
RAC: 10
United States
Message 1007772 - Posted: 24 Jun 2010, 15:43:09 UTC

Thanks for all your hard work & for keeping us updated. I was concerned when I ran out of work.


ID: 1007772 · Report as offensive
Christian Coryell
Volunteer tester
Avatar

Send message
Joined: 23 Apr 06
Posts: 66
Credit: 22,696,660
RAC: 0
United States
Message 1007790 - Posted: 24 Jun 2010, 16:28:52 UTC

Hey folks. We need to do something about this funding issue. SETI has more crunchers than just about any other project. This is a huge resource, and it feels like the operation at Berkeley is slowly deteriorating. The guys there can only do so much to keep things running and eventually, critical hardware will fail that they cannot afford to replace.

Given the broad spectrum of people who contribute to this project, there have to be some profs who can help pursue grants to reinvigorate the project. Any ideas?
ID: 1007790 · Report as offensive
Profile perryjay
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 20,676,751
RAC: 0
United States
Message 1007809 - Posted: 24 Jun 2010, 17:28:15 UTC
Last modified: 24 Jun 2010, 17:29:58 UTC

We're on our way!!! Uploads and downloads are flowing again!!


Sorry, I was so excited to see things moving again I forgot to thank all the guys at SAH for getting us there!


PROUD MEMBER OF Team Starfire World BOINC
ID: 1007809 · Report as offensive
Scarecrow

Send message
Joined: 15 Jul 00
Posts: 4520
Credit: 486,601
RAC: 0
United States
Message 1007816 - Posted: 24 Jun 2010, 17:45:30 UTC - in response to Message 1007813.  

Shouldn't the thread title be "Fill 'er up (Jun 23 2010)" ?

It ain't July yet.


Obviously a severe bug in the poster's date conversion module. Roll 'im back.
ID: 1007816 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65689
Credit: 55,293,173
RAC: 49
United States
Message 1007831 - Posted: 24 Jun 2010, 18:41:15 UTC - in response to Message 1007809.  
Last modified: 24 Jun 2010, 18:41:40 UTC

We're on our way!!! Uploads and downloads are flowing again!!


Sorry, I was so excited to see things moving again I forgot to thank all the guys at SAH for getting us there!

Only If Your Quota isn't a quota of 904,250,952 tasks... And so I can't download either cpu or gpu tasks...
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1007831 · Report as offensive
Profile Sharpshooter

Send message
Joined: 26 Mar 00
Posts: 47
Credit: 5,127,861
RAC: 2
United States
Message 1007868 - Posted: 24 Jun 2010, 21:07:22 UTC - in response to Message 1007790.  

It might be a good idea to get on Congress's case. They have for years been wringing their hands over the fact that American kids are woefully lacking in math and science. A few grants might just be the thing from Congress to SETI in the spirit of getting young people more interested in science. SETI is science and fun at that. I intend to rattle their cages on it. By not signing on we can accuse them of not being serious when they bring up our young people being behind in science and math. They certainly have a long history of pissing away money on projects that do us very little good.
ID: 1007868 · Report as offensive
Profile Donald L. Johnson
Avatar

Send message
Joined: 5 Aug 02
Posts: 8240
Credit: 14,654,533
RAC: 20
United States
Message 1007883 - Posted: 24 Jun 2010, 22:00:17 UTC - in response to Message 1007868.  

It might be a good idea to get on Congress's case. They have for years been wringing their hands over the fact that American kids are woefully lacking in math and science. A few grants might just be the thing from Congress to SETI in the spirit of getting young people more interested in science. SETI is science and fun at that. I intend to rattle their cages on it. By not signing on we can accuse them of not being serious when they bring up our young people being behind in science and math. They certainly have a long history of pissing away money on projects that do us very little good.


Last time I looked, Congress doesn't do grants. Earmarks, yes, grants, no. (8{)

Congress allocates money to Federal agencies such as NASA, DoD, the National Institute for Standards and Testing (NIST), The National Science Foundation, etc. to be given as grants for scientific research, but the Congress doesn't hand out grants directly to researchers.

So don't bug your Congresscritter about a grant to help SETI. There are still some in Congress who remember Sen. Proxmire and his "Golden Fleece" awards for wasting Federal tax dollars on such things as SETI, $600 toilet seats on military transports, etc.
ID: 1007883 · Report as offensive
Profile Sharpshooter

Send message
Joined: 26 Mar 00
Posts: 47
Credit: 5,127,861
RAC: 2
United States
Message 1007894 - Posted: 24 Jun 2010, 22:33:50 UTC - in response to Message 1007883.  

I'm not here to split hairs with you or anyone else. Congress cut the money off in the early 90's to NASA on SETI. Short-sighted people like Sen. Richard Bryan and Rep. Silvio Conte were two of the culprits. I can't help it of Proxmire is short-sighted as well. Restoring the funding will bring SETI more into the public eye. Indirectly it may help the project. I fail to see where it would hurt. We're talking about only 4 mil. That's almost nothing to Congress. Doing nothing will not get us anywhere. Anyone on this board that thinks SETI deserves a Golden Fleese Award shouldn't be here.
ID: 1007894 · Report as offensive
1 · 2 · 3 · Next

Message boards : Technical News : Fill 'er up (Jun 23 2010)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.