Unexpected outage (March 30-31)

Message boards : News : Unexpected outage (March 30-31)

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 5 · Next

AuthorMessage
Eric Korpela
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 3 Apr 99
Posts: 1225
Credit: 18,714,903
RAC: 7,956
United States
Message 1858467 - Posted: 30 Mar 2017, 23:43:29 UTC
Last modified: 31 Mar 2017, 0:00:19 UTC

We've found some corruption in a part of our science database that holds the power vs time profiles of detected pulses. We're running some table checks to see if we can localize the problem and come up with a solution that minimizes the lost workunits that need to be redone.
@SETIEric

ID: 1858467 · Report as offensive    Reply Quote
Profile Joshua Nicoll
Avatar

Send message
Joined: 23 Apr 16
Posts: 15
Credit: 5,020,032
RAC: 4,050
Ireland
Message 1858471 - Posted: 31 Mar 2017, 0:11:03 UTC - in response to Message 1858467.  

Oh man, that really sucks, was there a lot of data lost or are you still assessing the damage? I hate getting corrupted archives, it annoys me.
ID: 1858471 · Report as offensive    Reply Quote
robertmiles
Volunteer tester

Send message
Joined: 16 Jan 12
Posts: 174
Credit: 1,730,868
RAC: 1,705
United States
Message 1858474 - Posted: 31 Mar 2017, 0:20:20 UTC

It looks like uploading the output files from client tasks is currently allowed, but reporting these tasks as finished is currently not allowed. Is this correct?
ID: 1858474 · Report as offensive    Reply Quote
Profile Gary CharpentierCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 19911
Credit: 25,885,102
RAC: 31,570
United States
Message 1858476 - Posted: 31 Mar 2017, 0:26:41 UTC

Thanks for the information. Hope you are able to isolate the data that is bad. and find why it was corrupted. Any possibility that you might have a good backup of that part of the data?
ID: 1858476 · Report as offensive    Reply Quote
Sir Thomas W. Kilburn

Send message
Joined: 5 Apr 15
Posts: 4
Credit: 372,497
RAC: 674
Turks and Caicos Islands
Message 1858499 - Posted: 31 Mar 2017, 2:12:36 UTC - in response to Message 1858467.  

do what you can. I will just sit here and wait while the computers process what data they have in storage.
ID: 1858499 · Report as offensive    Reply Quote
mknunan

Send message
Joined: 23 Nov 15
Posts: 1
Credit: 139,879
RAC: 224
United States
Message 1858501 - Posted: 31 Mar 2017, 2:34:41 UTC - in response to Message 1858467.  

Oh, well. S**t happens. It sure beats getting wiped out by and asteroid! Best of luck and skill in the repair department.
ID: 1858501 · Report as offensive    Reply Quote
Profile betregerProject Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 5600
Credit: 13,267,940
RAC: 12,729
United States
Message 1858502 - Posted: 31 Mar 2017, 2:36:55 UTC - in response to Message 1858476.  
Last modified: 31 Mar 2017, 2:37:29 UTC

Any possibility that you might have a good backup of that part of the data?

Gary methinks at worst they have the source data and can have it rerun.
ID: 1858502 · Report as offensive    Reply Quote
Miranda

Send message
Joined: 27 May 11
Posts: 1
Credit: 82,082
RAC: 23
Australia
Message 1858508 - Posted: 31 Mar 2017, 3:29:42 UTC - in response to Message 1858467.  

This is the third outage in as many weeks. Get your act together !!!!!
Otherwise i won't bother supporting you again. Enough is enough !!!!!
ID: 1858508 · Report as offensive    Reply Quote
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 937
Credit: 8,201,754
RAC: 2,745
New Zealand
Message 1858510 - Posted: 31 Mar 2017, 3:40:01 UTC - in response to Message 1858508.  

This is the third outage in as many weeks. Get your act together !!!!!
Otherwise i won't bother supporting you again. Enough is enough !!!!!

It is all part and parcel of distributed computing if you decide to leave enjoy other projects that you contribute to
ID: 1858510 · Report as offensive    Reply Quote
Profile Ozmoses
Volunteer tester
Avatar

Send message
Joined: 9 Jun 03
Posts: 18
Credit: 28,407,687
RAC: 5,354
Australia
Message 1858514 - Posted: 31 Mar 2017, 4:09:30 UTC

That's unfortunate. However, as I've seen you through bad disks, failing systems, dodgy power, bad backups, crappy data splitters, weird migration problems, curious scripts and data corruption in the past, I see no reason why I should stop supporting SETI now!!
I look forward to being able to process SETI data again once you get things back on line.
Then, we can get back on with finding ET.

Keep up the good work team.
See you when you are back on line!!

Cheers,

Chris
ID: 1858514 · Report as offensive    Reply Quote
Profile Gary CharpentierCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 19911
Credit: 25,885,102
RAC: 31,570
United States
Message 1858515 - Posted: 31 Mar 2017, 4:30:27 UTC - in response to Message 1858502.  

Any possibility that you might have a good backup of that part of the data?

Gary methinks at worst they have the source data and can have it rerun.

I was thinking of a backup of the informix DB, so that the damaged data would not have to be rerun. Hopefully they have been able to store all the raw data, but that is a huge store and also subject to corruption / bit rot.
ID: 1858515 · Report as offensive    Reply Quote
Jellwood

Send message
Joined: 24 Feb 02
Posts: 3
Credit: 136,955,722
RAC: 182,757
United States
Message 1858519 - Posted: 31 Mar 2017, 5:03:12 UTC - in response to Message 1858510.  

As a member who has friends that are pioneers in the Space Stuff~~~and I know where Green Bank IS!~~~And have More $$$$$ invested than the whiners, and THINK that SETI is a GOOD thing! My Total Credit speaks for itself, My Average Credit SPEAKS for itself, and OUR Team Scream what a Few GOOD Guys can DO! My ONLY question IS~~~Can I have More Work Units in my QUE? And is there a BOOK with all of the STUFF in It that I need to Know to be a BETTER Contributor? From Flint, MI and for my Friend and Mentor in the Ukraine~~~~We are Proud to Help! John Ellwood Burtscher, Flint, MI and Igor aka IGOGO aka MyFavoriteGadgets! the Ukraine!
ID: 1858519 · Report as offensive    Reply Quote
Mike Ryan

Send message
Joined: 24 Jun 99
Posts: 16
Credit: 7,995,320
RAC: 26,841
United States
Message 1858525 - Posted: 31 Mar 2017, 6:47:13 UTC - in response to Message 1858519.  
Last modified: 31 Mar 2017, 6:52:50 UTC

Anyone that's processed over 125,000,000 credits could probably WRITE the book on how to be a "better" contributor. THAT'S INCREDIBLE!

However, I think for the most part you're on your own to find the most effective way to process data. This is because EVERY combination of computer(s) and parts may need to be tweaked slightly differently to maximize the work you're doing. The best you can do is try something, and wait a week or so to see if it helped or hurt. Then repeat, repeat, repeat.

In my case I currently have two computers on 24/7. One is my desktop which has a 4 core / 4 thread i5-4590, 3.3 GHz, and an Nvidia GTX 1060 6GB. So far I get the most credit per day by having the 1060 crunch 4 work units at a time and dedicate 0.25 CPU core to each task. This leaves 3 cores free to clobber non-GPU tasks. So I'm usually crunching 7 tasks at a time. This leaves my GPU 98-100% busy all the time it has taks, and the CPU around 95-98% busy. That small amount of "free" time keeps the system from bogging down while I'm doing other light stuff like web browsing. It also allows the CPU to stay in "turbo" mode at 3.5 GHz nearly all the time.

I could also be crunching an Intel GPU (HD 4600) at the same time, but I've found that doing so usually doubles the time the CPU work units take. Overall, my daily credit is slightly lower when using the iGPU on this machine.

On the other hand, my laptop is an i3-7100u @ 2.4 GHz. It's capable of running up to 4 threads too, but it completes tasks on average much better if I limit it to running only two CPU tasks at a time. I found that I can also have it run an Intel GPU task without significantly increasing the CPU task processing time. So in this case, the iGPU really is giving me a small bit of a "free lunch".

And like you I have found that it's REALLY a pain that only a maximum of 100 of any type of work unit are sent to any given computer. While 100 CPU work units is several DAYS worth of work for my little laptop, my desktop's GTX 1060 can EASILY rip through over 100 work units in less than a day. (Closer to 150 - 200 last I checked.) So yeah, when the project is down for very long (even for the weekly outage every Tuesday) I usually end up running out of GPU work. I suppose I could also fire up other projects to keep my GPU busy, but I prefer to just wait it out until the data starts flowing again.
ID: 1858525 · Report as offensive    Reply Quote
Profile Chris SCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 19 Nov 00
Posts: 39362
Credit: 27,966,417
RAC: 43,229
United Kingdom
Message 1858530 - Posted: 31 Mar 2017, 7:29:10 UTC

Many thanks to Eric for keeping us informed of what the problems are. When it's things like this that they cannot control, we understand, and they have all our best wishes to solve it quickly.

After 16 years I'm staying around for the long haul, those that are not happy with the project can of course move on to other Boinc projects if they wish. Having said that if I run out of work I'll go crunch Einstein for a bit, but I'll be back.

Lets try to keep positive guys.
ID: 1858530 · Report as offensive    Reply Quote
Profile John.T.Daly
Avatar

Send message
Joined: 23 Nov 13
Posts: 11
Credit: 14,684,016
RAC: 23,418
United Kingdom
Message 1858533 - Posted: 31 Mar 2017, 7:43:01 UTC - in response to Message 1858467.  

Thanx for info Eric......................thought problem was at my end ...........been having issues with my connection.......hope everything gets back on track soon :)
There is anybody out there!
ID: 1858533 · Report as offensive    Reply Quote
Profile Jamie

Send message
Joined: 26 Jan 15
Posts: 2
Credit: 1,285,208
RAC: 4,258
United Kingdom
Message 1858535 - Posted: 31 Mar 2017, 7:51:28 UTC

Hah, and I just started processing work units again.

Good luck guys!
ID: 1858535 · Report as offensive    Reply Quote
Bastian Marugg

Send message
Joined: 30 Apr 16
Posts: 1
Credit: 110,543
RAC: 204
Switzerland
Message 1858538 - Posted: 31 Mar 2017, 8:56:01 UTC

That's very Bad. Hope you can Repair the Problem soon

Thank you for keeping us Inform

Nice Greetings Bastian Marugg Switzerland
ID: 1858538 · Report as offensive    Reply Quote
RobertN

Send message
Joined: 12 Oct 09
Posts: 2
Credit: 772,419
RAC: 1,869
Netherlands
Message 1858541 - Posted: 31 Mar 2017, 9:39:37 UTC
Last modified: 31 Mar 2017, 9:40:12 UTC

Is there an ETA when SETI will be functional again?
I am part of a big Dutch team that was preparing to put all our calculation resources on this project this coming month.
ID: 1858541 · Report as offensive    Reply Quote
Profile Eng. Lucas Divina

Send message
Joined: 12 Feb 06
Posts: 5
Credit: 1,154,444
RAC: 307
Czech Republic
Message 1858545 - Posted: 31 Mar 2017, 10:14:06 UTC - in response to Message 1858467.  

I thanks, for your information me I looking for the problem if it on my side, but I don´t know how to repair it.
I'm Ing. Lukáš Divina for the SETI@home
ID: 1858545 · Report as offensive    Reply Quote
Stephen "Heretic"Project Donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 1981
Credit: 23,057,158
RAC: 155,545
Australia
Message 1858546 - Posted: 31 Mar 2017, 10:18:13 UTC - in response to Message 1858510.  

This is the third outage in as many weeks. Get your act together !!!!!
Otherwise i won't bother supporting you again. Enough is enough !!!!!

It is all part and parcel of distributed computing if you decide to leave enjoy other projects that you contribute to


. . I am surprised that they are surprised. I thought everyone knew there was an outage every week ...

Stephen

?
ID: 1858546 · Report as offensive    Reply Quote
1 · 2 · 3 · 4 . . . 5 · Next

Message boards : News : Unexpected outage (March 30-31)


 
©2017 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.