Once Upon A Time!

Message boards : Number crunching : Once Upon A Time!
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Cliff Harding
Volunteer tester
Avatar

Send message
Joined: 18 Aug 99
Posts: 1432
Credit: 110,967,840
RAC: 67
United States
Message 1625779 - Posted: 9 Jan 2015, 13:36:32 UTC

Once upon a time in the not too distinct past, we who desired to work only AP tasks had a more or less steady diet. There was a time, prior to the recent database problems each AP splitter had it own file to digest and my queue stayed full even through those times when only MB tasks were available. Since the troubles, I see that some AP files are being worked on by as many as 3 splitters, which slows down splitting time and thus slows down work getting to the many Setizens in the universe. It seems that I can't get any AP tasks to d/l unless I also accept MB tasks as well. I'm lucky if I get 3 days of AP GPU & 6 days of AP CPUpu work now days. If I turn off MB work, once I run out of all tasks, I get nothing, CPU or GPU until I turn on MB again, and then AP comes in drips and drabs. Will we ever get back to the good old days when we can more or less depend on a steady AP diet or are those days gone forever?

While I'm at it, does anyone know when we will get to the point where the max GPU tasks will be governed not by the fact that there is a GPU, but by the number of GPUs on a host, regardless of type (ATI or NVidia)? And, now that GPUs have gotten to such a powerful state of being, when will we be allowed to process VLARs on GPUs? I remember a time when I rescheduled VLARs to run on my GTX460SC and they ran quite well even though the scheduler still classified them as CPU tasks.


I don't buy computers, I build them!!
ID: 1625779 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1625784 - Posted: 9 Jan 2015, 13:49:36 UTC
Last modified: 9 Jan 2015, 13:50:50 UTC

It was mentioned that the lack of disk space is what was slowing down splitting not x number of splitters processing one data set.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1625784 · Report as offensive
Profile Cliff Harding
Volunteer tester
Avatar

Send message
Joined: 18 Aug 99
Posts: 1432
Credit: 110,967,840
RAC: 67
United States
Message 1625913 - Posted: 9 Jan 2015, 22:26:19 UTC - in response to Message 1625784.  

It was mentioned that the lack of disk space is what was slowing down splitting not x number of splitters processing one data set.


So do we need to start a new funding program to get more disk space?


I don't buy computers, I build them!!
ID: 1625913 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1625915 - Posted: 9 Jan 2015, 22:45:10 UTC - in response to Message 1625913.  

It was mentioned that the lack of disk space is what was slowing down splitting not x number of splitters processing one data set.


So do we need to start a new funding program to get more disk space?

They were planning to increase the allotted area of the drive array for WU storage. This may or may not have happened yet. The information was received 3rd hand in the Panic thread.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1625915 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1625923 - Posted: 9 Jan 2015, 23:08:06 UTC - in response to Message 1625915.  

It was mentioned that the lack of disk space is what was slowing down splitting not x number of splitters processing one data set.


So do we need to start a new funding program to get more disk space?

They were planning to increase the allotted area of the drive array for WU storage. This may or may not have happened yet. The information was received 3rd hand in the Panic thread.


From what I understood from Matt's message (and others) was that the Main Database is still hurting from the crash awhile back. It is currently rebuilding it's self but needs space, and time, to do it and it's fighting for space with the new WU's. Lucky they have a backup to compare the two sets of data.

The crash a few hours ago I'm sure was caused by the splitters. S@H took a big jump in splitting of 80k in a short time, then the system locked up. Don't know what happened there for sure.
ID: 1625923 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1625925 - Posted: 9 Jan 2015, 23:46:41 UTC

Yeah, basically.., "the database is still sick, and is being fixed and repaired. It will take time. In the meantime, assimilation can't happen until then, which means those WUs are still on disk, and the disk space got filled. More space is being added and that should allow new work to be split again."

I then calculated that the AP WUs sitting on disk right now roughly equate to about 7TB of data, with all of MB being ~1TB, meaning it is reasonable to assume the WU storage area was 8TB and more had to be added to it.


However, I remember in the past that multiple splitters working on one tape didn't have any notable effects. It actually makes the individual tapes get done and removed from the list faster. There's nothing wrong with multiple splitters working on the same tape.

The reason we saw the output drop was because the WU storage was full and was waiting for MBs to be returned, validated, assimilated, and then run through the purge stage to free up more space for new WUs to be made. We're going to be in choppy waters for a little while longer.. until the AP database is given a clean bill of health and assimilation can begin, which will then move those WUs along to the purge stage and reclaim that disk space.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1625925 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1625926 - Posted: 9 Jan 2015, 23:47:44 UTC - in response to Message 1625923.  

It was mentioned that the lack of disk space is what was slowing down splitting not x number of splitters processing one data set.


So do we need to start a new funding program to get more disk space?

They were planning to increase the allotted area of the drive array for WU storage. This may or may not have happened yet. The information was received 3rd hand in the Panic thread.


From what I understood from Matt's message (and others) was that the Main Database is still hurting from the crash awhile back. It is currently rebuilding it's self but needs space, and time, to do it and it's fighting for space with the new WU's. Lucky they have a backup to compare the two sets of data.

The crash a few hours ago I'm sure was caused by the splitters. S@H took a big jump in splitting of 80k in a short time, then the system locked up. Don't know what happened there for sure.

I don't imagine the disk array is near full. It's just a matter of the amount of space allocated to the temp WU storage. If the databases are on the same array they would be in different partitions. At least I would hope so.
This is the message I was referencing. http://setiathome.berkeley.edu/forum_thread.php?id=76430&postid=1625067#1625067
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1625926 · Report as offensive

Message boards : Number crunching : Once Upon A Time!


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.