Database Rebuild Status update.

Message boards : News : Database Rebuild Status update.
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Eric Korpela Project Donor
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 3 Apr 99
Posts: 1382
Credit: 54,506,847
RAC: 60
United States
Message 1607963 - Posted: 2 Dec 2014, 2:22:45 UTC

We've now inserted 231,770,000 of the 306,362,193 AstroPulse signals into the database. Over the last week the insert rate has slowly declined from 1.5 million per hour to 870,000 per hour. At that rate we're looking at another 86 hours (+24 for index building), which means Saturday morning (US Pacific time).

On the plus side, Arecibo has four full drives of data that they can send us, so there might be SETI@home work before then if we're really lucky. I'll be posting updates in the Tech News forum through out the week.
@SETIEric@qoto.org (Mastodon)

ID: 1607963 · Report as offensive
Mithotar
Avatar

Send message
Joined: 11 Apr 01
Posts: 88
Credit: 66,037,385
RAC: 50
United States
Message 1607964 - Posted: 2 Dec 2014, 2:34:01 UTC - in response to Message 1607963.  

Thanks for the update.......wish the news was better. I'll clean some dust bunnies and await new workunits to process.
ID: 1607964 · Report as offensive
Profile TimeLord04
Volunteer tester
Avatar

Send message
Joined: 9 Mar 06
Posts: 21140
Credit: 33,933,039
RAC: 23
United States
Message 1607977 - Posted: 2 Dec 2014, 3:47:59 UTC

Thanks Eric. :-)
TimeLord04
Have TARDIS, will travel...
Come along K-9!
Join Calm Chaos
ID: 1607977 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 31015
Credit: 53,134,872
RAC: 32
United States
Message 1607980 - Posted: 2 Dec 2014, 4:07:42 UTC

News is always appreciated.
ID: 1607980 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 36868
Credit: 261,360,520
RAC: 489
Australia
Message 1607986 - Posted: 2 Dec 2014, 4:27:06 UTC
Last modified: 2 Dec 2014, 4:33:51 UTC

The news is welcome Eric, but I still think that both News and Technical News threads shouldn't be open for just everyone to have a general discussion in as that would stop those that are to lazy to read through all the "thank you" posts (and other stupid crap) just to get the info that they need without having to ask stupid questions that have already been answered because they're to lazy to read through everything (I'm sure that you and the other guys know how thankful we are for these updates without us having to say so). ;-)

If someone wants to ask a question, surely they can do that it the Number Crunching or in Questions & answers sections.

Cheers.
ID: 1607986 · Report as offensive
Profile Mentor397
Avatar

Send message
Joined: 16 May 99
Posts: 25
Credit: 6,794,344
RAC: 108
United States
Message 1608020 - Posted: 2 Dec 2014, 5:18:15 UTC

Thanks for all your hard work.

- Jim
ID: 1608020 · Report as offensive
KLiK
Volunteer tester

Send message
Joined: 31 Mar 14
Posts: 1304
Credit: 22,994,597
RAC: 60
Croatia
Message 1608034 - Posted: 2 Dec 2014, 6:49:56 UTC

Thx 4 ᛏdate...will try not to import new units for nVidia card from Saturday... ;)


non-profit org. Play4Life in Zagreb, Croatia, EU
ID: 1608034 · Report as offensive
Profile Jeff-Bull
Avatar

Send message
Joined: 21 Sep 11
Posts: 3
Credit: 4,594,426
RAC: 0
United Kingdom
Message 1608087 - Posted: 2 Dec 2014, 10:19:23 UTC

Why, when there are so little (or no) units to process do I still have a backlog of units awaiting validation, one is 50+ days old? I was kinda hoping that this drought would have a silver lined cloud, but no.

Has anyone else noticed this 'problem' with their Awaiting-Validation units?

Jeff Bull (SETI.UK)
ID: 1608087 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1856
Credit: 268,616,081
RAC: 1,349
United States
Message 1608112 - Posted: 2 Dec 2014, 11:26:02 UTC - in response to Message 1608087.  
Last modified: 2 Dec 2014, 11:36:24 UTC

Why, when there are so little (or no) units to process do I still have a backlog of units awaiting validation, one is 50+ days old? I was kinda hoping that this drought would have a silver lined cloud, but no.

Has anyone else noticed this 'problem' with their Awaiting-Validation units?

Jeff Bull (SETI.UK)

Well, there are two issues. If you look at workunits awaiting validation, you will see one of a few scenarios. First, your "wingman" may still be crunching the job or may have defaulted on it (either by timeout or error). It won't be subjected to validation until that is complete. Assuming that's not the problem, right now, you probably see a ton of AP jobs waiting validation, as I do, because that portion of the whole server farm is not currently in operation. This is because the databases that store the results are down, and cannot accept the results of validation.
In your case, mostly it looks as though the AP issue is not the problem. You are waiting on your wingman to complete their piece of the work, as you can see if you click on the workunit link on your jobs waiting validation. Hey, 16 pending isn't too bad; I think I'm over 100 at this point:)
Cheers, Jim ...
ID: 1608112 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1608131 - Posted: 2 Dec 2014, 12:17:32 UTC - in response to Message 1608112.  

Hey, 16 pending isn't too bad; I think I'm over 100 at this point:)

Realy not worry about that, i have: Validation pending (2243)
ID: 1608131 · Report as offensive
Profile Jeff-Bull
Avatar

Send message
Joined: 21 Sep 11
Posts: 3
Credit: 4,594,426
RAC: 0
United Kingdom
Message 1608152 - Posted: 2 Dec 2014, 14:30:22 UTC

Well! That's puts me and my little problem in our rightful place!
Many thanks for your feedback. I shall now have a good laugh ... at my own self :-) Please feel free to join in.

Regards

Jeff at Seti.UK
ID: 1608152 · Report as offensive
Profile cwburch

Send message
Joined: 23 Jul 00
Posts: 67
Credit: 61,429,161
RAC: 27
United States
Message 1608178 - Posted: 2 Dec 2014, 16:05:04 UTC - in response to Message 1608152.  

Well, my 24-CPU setup got a slew of about 24 Wu's last night some time, all with filenames ending end _2. So apparently, I'm doing a bit lf "validating". Better than nothing I guess. My personal goal is to reach a total 20M WU's by the end of the year. Prior to the ongoing shortage, that was easily going to be obtained around then end of November. Now, with my total just over 19,980,000 and the fact that 4 hard drives of raw data are on the way, I just "might" do it. But we'll see.
I still remember my first year of doing this back in the early 90's with a 2400bps dialup modem on a "lighting fast" 200Ghz processor with an "astounding" 4MB of memory. I was disappointed that I was but one WU short of 1000 at midnight, December 31st of 1997-1998 I think.
ID: 1608178 · Report as offensive
Cheopis

Send message
Joined: 17 Sep 00
Posts: 156
Credit: 18,451,329
RAC: 0
United States
Message 1608186 - Posted: 2 Dec 2014, 16:31:46 UTC

Eric,

I do not know if this makes any sense at all, or if you have the tools to check it, but perhaps the slowdown is due to physical characteristics of the disks?

Years and years ago, it used to be moderately important to organize data on disks physically, keeping frequently accessed data towards the outside of the disk and less frequently used data to the inside.

This allowed faster reading and writing of data to the outside of the disk, because at a constant RPM, the write heads crossed more sectors at the outside part of the disk over time than the inside. Every sector is physically the same size.

Because hard drives these days are so huge, very rarely does one encounter a database that literally uses up a substantial part of the disk, at least in day-to-day operations of us normal folks.

I'm certain that there are probably database programs out there which have methods of addressing physical location of data on disks. I could imagine RAID mirroring with inside-out and outside-in data organization, for example, but for most organizations, it might simply be a better solution to expand the array size, and split the data onto more disks. That's probably not an option here.

If you look at the "fullness" of the drives and then do a surface area comparison to see where the write heads are writing to on the drive, you might find that the write heads are in a place on the disks where 50% data transfer as compared to maximum observed data transfer is expected.

With a little more analysis, if you have been tracking data throughput from the beginning, you can probably graph the data transfer rate and compare it against the expected curve of a disk filling from the outside in.

Then again, everything I just said might be complete bunk.
ID: 1608186 · Report as offensive
Profile budwinmayo
Avatar

Send message
Joined: 21 Jan 12
Posts: 2
Credit: 261,654
RAC: 0
United States
Message 1608209 - Posted: 2 Dec 2014, 20:12:59 UTC

Thanks for the heads up. Before I actually looked at the news regarding the site, I thought I had broken something or changed a setting. Look forward to continuing our search for ET! JM
ID: 1608209 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 1608218 - Posted: 2 Dec 2014, 20:47:47 UTC - in response to Message 1608178.  

I still remember my first year of doing this back in the early 90's with a 2400bps dialup modem on a "lighting fast" 200Ghz processor with an "astounding" 4MB of memory. I was disappointed that I was but one WU short of 1000 at midnight, December 31st of 1997-1998 I think.


SETI@home didn't open up to the public until May 17th, 1999. Some early testers were able to get access in late 1998, but nothing was going on in the early 90's.
ID: 1608218 · Report as offensive
Profile Thomas Tilson

Send message
Joined: 2 Aug 99
Posts: 5
Credit: 5,546,520
RAC: 7
United States
Message 1608264 - Posted: 2 Dec 2014, 21:55:22 UTC

Thanks Eric...keeping the CPU's busy with Rosetta. No issues hanging out for WU's to start to flow (been doing this SINCE 1999...so I understand the process and appreciate the hard work you guys do!)

Tom
ID: 1608264 · Report as offensive
Profile Blurf
Volunteer tester

Send message
Joined: 2 Sep 06
Posts: 8964
Credit: 12,678,685
RAC: 0
United States
Message 1608267 - Posted: 2 Dec 2014, 22:02:39 UTC - in response to Message 1608087.  

Why, when there are so little (or no) units to process do I still have a backlog of units awaiting validation, one is 50+ days old? I was kinda hoping that this drought would have a silver lined cloud, but no.

Has anyone else noticed this 'problem' with their Awaiting-Validation units?

Jeff Bull (SETI.UK)


Jeff-With the database crash, resends aren't going out so the quoroum to validate your WU's isn't being met. Have to wait on those....


ID: 1608267 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 36868
Credit: 261,360,520
RAC: 489
Australia
Message 1608272 - Posted: 2 Dec 2014, 22:11:11 UTC



Jeff-With the database crash, resends aren't going out so the quoroum to validate your WU's isn't being met. Have to wait on those....

What are you talking about Blurf? :-O

Resends are being sent out for all task types or are what I'm getting just figments of someone's imagination?

Maybe you should take a look at this thread, http://setiathome.berkeley.edu/forum_thread.php?id=76115&sort_style=6&start=0. ;-)

Cheers.
ID: 1608272 · Report as offensive
Profile cwburch

Send message
Joined: 23 Jul 00
Posts: 67
Credit: 61,429,161
RAC: 27
United States
Message 1608303 - Posted: 2 Dec 2014, 22:51:43 UTC - in response to Message 1608218.  

Thanks. I wasn't sure of the timeline. But I know I lost of few years of credit when I moved from Hawaii to Florida after retiring. I never could recall the password for the original email address I started under. So that credit was just gone forever.
Thing is, I left Hawaii in August of '98 and I was participating prior to leaving. So perhaps I was one of the early ones during the period of establishment? I really don't recall.
ID: 1608303 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1608309 - Posted: 2 Dec 2014, 23:16:08 UTC - in response to Message 1608303.  

You still got that "200 GHz" processor though? We're all jealous. :)
ID: 1608309 · Report as offensive
1 · 2 · 3 · Next

Message boards : News : Database Rebuild Status update.


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.