bigger WUs

Message boards : Number crunching : bigger WUs
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Lanisa

Send message
Joined: 5 Jun 99
Posts: 25
Credit: 688,811
RAC: 0
Germany
Message 152275 - Posted: 16 Aug 2005, 17:18:48 UTC

Wouldn't it be better to have bigger WUs in order to drop the pressure from the storage server.

Twice or tripple the size of a unit would greatly minimize the results uploaded on the server.

Every A64 or P4 should be capable to doing such units in a reasonably timeframe.

But we have to take care of older computer and slower connection. So depending on the benchmark results or bandwith an algorithm could select the best solution.
ID: 152275 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 152277 - Posted: 16 Aug 2005, 17:25:40 UTC

Some of us are already testing Seti Beta units. They take around 133,000 - 140,000+ seconds on a P4, are you sure you want them to be enabled in here? ;)


ID: 152277 · Report as offensive
EdwardPF
Volunteer tester

Send message
Joined: 26 Jul 99
Posts: 389
Credit: 236,772,605
RAC: 374
United States
Message 152278 - Posted: 16 Aug 2005, 17:32:34 UTC

Bring 'em on!! 24-48 hr's sounds just about perfect to me!!

ID: 152278 · Report as offensive
Profile Tigher
Volunteer tester

Send message
Joined: 18 Mar 04
Posts: 1547
Credit: 760,577
RAC: 0
United Kingdom
Message 152283 - Posted: 16 Aug 2005, 17:43:02 UTC

Well it has been suggested to the devs, and they thinking about it, compressed WUs that are batched and stored in a compressed state. Say 20 WUs at a time. This reduces server connections, bandwidth needs, excessive dial up time, and of course server spaces while compressed. I'm sure they will let us know if its a do-able thing.
ID: 152283 · Report as offensive
Profile Dorsai
Avatar

Send message
Joined: 7 Sep 04
Posts: 474
Credit: 4,504,838
RAC: 0
United Kingdom
Message 152287 - Posted: 16 Aug 2005, 18:14:10 UTC
Last modified: 16 Aug 2005, 18:15:15 UTC

When I first started seti classic, only last year, my PC at the time took about 24 hours per WU.

I currently take, on average, 3h:20m, on seti-boinc.

I would have no objection to larger WUs that took longer. As long as the cerdit granted/claimed reflected the larger WU's longer processing time.

If I did only one a day on this PC, but got an average of 200Cr (what this PC gets now, doing 7) I would be no worse off, and SETI would have one WU to split/validate/assimilate, instead of 7.

Take this accross the board, and there would be a huge decrese in the number of results to validate.

My PCs are not fast, but a "24 hour" Wu would result in a 7 fold decrease in the number of results my pc's return.
Seems a good idea to me.



Foamy is "Lord and Master".
(Oh, + some Classic WUs too.)
ID: 152287 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 152297 - Posted: 16 Aug 2005, 19:11:29 UTC - in response to Message 152275.  

Wouldn't it be better to have bigger WUs in order to drop the pressure from the storage server.

Twice or tripple the size of a unit would greatly minimize the results uploaded on the server.

Every A64 or P4 should be capable to doing such units in a reasonably timeframe.

But we have to take care of older computer and slower connection. So depending on the benchmark results or bandwith an algorithm could select the best solution.

... or just more science from the same WU.

It's been done before -- back in the early days of "classic."
ID: 152297 · Report as offensive
Profile [B@H] Ray
Volunteer tester
Avatar

Send message
Joined: 1 Sep 00
Posts: 485
Credit: 45,275
RAC: 0
United States
Message 152303 - Posted: 16 Aug 2005, 19:29:17 UTC
Last modified: 16 Aug 2005, 19:36:33 UTC

I would be all for that, fewer files to validate seams like it would help SETI out a lot. Mite be better to measure them in size, than others can figure there time from that. The current ones are 354K (a few at 364K). Going to App. 1 Meg would cut the mumber way down and still ba accessable with a dial up connection.

2 Megs files I think would also be good for dial up, and really allow the validators to catch up. Much more credit per file using the same celcs. as now since they are based on the CPU speed and time. This would work out to the same credit per hour as now.

The current size was established a long time ago with classic seti and slower computers, most computers were taking 12 to 18 hours then with the size files that they are now using. Now with counting credits rather than the number of units they can be any size, even if they take a week to do.

Has anyone suggested this to Matt?



Pizza@Home Rays Place Rays place Forums
ID: 152303 · Report as offensive
Profile andydied
Volunteer tester

Send message
Joined: 18 Dec 00
Posts: 35
Credit: 1,402,874
RAC: 74
South Africa
Message 152307 - Posted: 16 Aug 2005, 19:51:22 UTC

I'm also for bigger WU's taking more time to crunch.
Wish it could happen soon....
and maybee also more science from the current WU's as Ned Ludd suggested below.
(see message http://setiathome.berkeley.edu/forum_reply.php?thread=18668#152297)

Have a great day
andydied


ID: 152307 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 152308 - Posted: 16 Aug 2005, 19:51:59 UTC - in response to Message 152303.  

I would be all for that, fewer files to validate seams like it would help SETI out a lot. Mite be better to measure them in size, than others can figure there time from that. The current ones are 354K (a few at 364K). Going to App. 1 Meg would cut the mumber way down and still ba accessable with a dial up connection.

I'm doing this from distant memory.

The original SETI (classic) application did from -50 hz drift rate to +50 hz, and used a smaller step size between -10 and +10 hz.

A later version changed the inner range to -30 to +30.

BOINC/SETI inherited this from classic, I believe, and this is why the percentages are not always linear: the first 20 Hz goes quicker than the middle 60, and then we go back to "fast."

One "fix" might be to simply make the steps smaller -- do a deeper search in each WU.

That way, the science app. wouldn't have to be changed to handle a larger WU.

ID: 152308 · Report as offensive
Lanisa

Send message
Joined: 5 Jun 99
Posts: 25
Credit: 688,811
RAC: 0
Germany
Message 152310 - Posted: 16 Aug 2005, 20:12:20 UTC - in response to Message 152277.  

Some of us are already testing Seti Beta units. They take around 133,000 - 140,000+ seconds on a P4, are you sure you want them to be enabled in here? ;)




i would suggest 6 hours on a p4 2,4GHz with the standard client core would be a good.

133,000 secs is to high for an old P3 450 MHz or even worse for crunchers with p1 computers.

@Ray Brown
1 - 2 megs sounds good if the task takes linear time to complete. meaning 6 times the size of a WU results in 6 times the time to complete crunching it.

more science is also welcome if SETI really benifits from the deeper search.
ID: 152310 · Report as offensive
Profile Nightlord
Avatar

Send message
Joined: 17 Oct 01
Posts: 117
Credit: 1,316,241
RAC: 0
United Kingdom
Message 152311 - Posted: 16 Aug 2005, 20:13:50 UTC - in response to Message 152275.  

I have a couple of questions that trouble me about longer WU's, but first let me make two assumptions.

First, I assume given the sample size, that the signals in the WU's are random in nature and second, that they are distributed evenly accross the many millions of WU's we crunch.

So now to the questions:

Q1. If the validators had to check larger WU's then is it not the case they still have to validate the same ammount of information returned from the clients? Will this be quicker?

Say you have two standard WU's each with a single spike. Then consider a WU twice the size of the standard unit. This large unit has two spikes. The validator still has to verify the same information returned over the same period of time.

Maybe the validator directories will be smaller, but would we create a problem for the splitters? I don't know the answer.

Q2. With larger WU's is there not a greater probability of noise masking the data.

Say you have a standard WU with the golden gaussian (the WOW signal). We would hopefully detect and validate it. Now say you have this signal in a larger unit, which statistically will have a greater probability of having a noise burst. The WOW signal is more likely to be missed due to the noise burst forcing the WU to abort early.

Any thoughts?

ID: 152311 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 152319 - Posted: 16 Aug 2005, 20:40:59 UTC - in response to Message 152310.  
Last modified: 16 Aug 2005, 20:42:13 UTC

On Beta we're searching for more radiowaves, so a broader band than Seti does. Hence we're looking through the same units for more information, hence why it takes longer.

The amounts I mentioned are with the standard application. If you would take the optimized app, also availble (though not for the latest app 4.04), then you'd be crunching at half that time. All dependend on whatever CPU you use. :)


ID: 152319 · Report as offensive
Lanisa

Send message
Joined: 5 Jun 99
Posts: 25
Credit: 688,811
RAC: 0
Germany
Message 152323 - Posted: 16 Aug 2005, 20:54:15 UTC

@Nightlord


Q1. If the validators had to check larger WU's then is it not the case they still have to validate the same ammount of information returned from the clients? Will this be quicker?


if i follow your assumption. it takes the same time or maybe a bit more if you have to put two results together in order to validate a larger one.

But validation is not the bottleneck. it's the amount of (results) files stored on the server and the time to look up each.

So the goal is to reduce the amount of files.


Q2. With larger WU's is there not a greater probability of noise masking the data.


I think there must be a great code change in order to resolve this. Skipping moisy parts of a WU might do. But I'm not scientist at all (even if i'm interested in the topic of distributed computing and finding E.T.)

so i can't answer the question. That must do some scientist involved with SETI.
ID: 152323 · Report as offensive
Divide Overflow
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 365
Credit: 131,684
RAC: 0
United States
Message 152325 - Posted: 16 Aug 2005, 20:55:48 UTC
Last modified: 16 Aug 2005, 20:58:51 UTC

Currently there's no estimate on when (or even if!) the seti_enhanced application on the Beta test project will make it over here. The Beta crunch times are longer and the credit is correspondingly higher.

The WU file size appear to be the same as here, the differences are how they are analyzed.

ID: 152325 · Report as offensive
Profile [B@H] Ray
Volunteer tester
Avatar

Send message
Joined: 1 Sep 00
Posts: 485
Credit: 45,275
RAC: 0
United States
Message 152327 - Posted: 16 Aug 2005, 21:21:08 UTC - in response to Message 152310.  

Some of us are already testing Seti Beta units. They take around 133,000 - 140,000+ seconds on a P4, are you sure you want them to be enabled in here? ;)


@Ray Brown
1 - 2 megs sounds good if the task takes linear time to complete. meaning 6 times the size of a WU results in 6 times the time to complete crunching it.

more science is also welcome if SETI really benifits from the deeper search.


It dues sound good, but would be up to the SETI staff to determine if they would still get good science first, or if they could make the science better.

Seems like getting more science out of the same size would be better, but the same amount per Meg would do for the systems to catch up maby. I would go for tha larger ones if no info is lost in going to them, than see what they think of the Beta program. If the Bata is good that looks like it would be better to do and get more out of the same data.



Pizza@Home Rays Place Rays place Forums
ID: 152327 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 152333 - Posted: 16 Aug 2005, 21:38:04 UTC - in response to Message 152323.  

But validation is not the bottleneck. it's the amount of (results) files stored on the server and the time to look up each.

It's not even that. It's the amount of directories and sub-directories to search through on the server that is the bottleneck.


ID: 152333 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 152337 - Posted: 16 Aug 2005, 21:44:30 UTC - in response to Message 152303.  
Last modified: 16 Aug 2005, 21:49:52 UTC

I would be all for that, fewer files to validate seams like it would help SETI out a lot. Mite be better to measure them in size, than others can figure there time from that. The current ones are 354K (a few at 364K). Going to App. 1 Meg would cut the mumber way down and still ba accessable with a dial up connection.

2 Megs files I think would also be good for dial up, and really allow the validators to catch up. Much more credit per file using the same celcs. as now since they are based on the CPU speed and time. This would work out to the same credit per hour as now.

The current size was established a long time ago with classic seti and slower computers, most computers were taking 12 to 18 hours then with the size files that they are now using. Now with counting credits rather than the number of units they can be any size, even if they take a week to do.

Has anyone suggested this to Matt?


Haven't heard of any plans to make the wu-size larger than the current 350 KB, and with the more detailed tests being done by SETI@Home Enhanced currently in beta-testing giving much longer cpu-times it's unlikely the wu-size will increase from 350 KB for some time...

Expected cpu-time and deadline will be dependent on AngleRange, and based on a 3h10m-cpu-time with the "standard" unoptimized-client the currently expected cpu-times is something like this:

AR - cpu-time (h)
0.11 - 12.11 (same for smaller angle-range)
0.12 - 12.92
0.225 - 18.13
0.226 - 56.12
0.30 - 39.27
0.40 - 27.41
0.417 - 26.02 (the most common angle-range)
0.50 - 20.74
0.60 - 16.51
0.70 - 13.61
0.80 - 11.52
0.90 - 9.94
1.00 - 8.72
1.13 - 7.48
1.14 - 4.13 (same for larger angle-range)

This gives a difference of 13.59x between largest and smallest, clearly showing why "1 wu = 1 credit" isn't such a good idea...

Now, it's important to remember, this is the current theoretical numbers used by the splitter, and can be changed again. Also, the actual times can be wastly different. Even if it's very close to actual for one angle-range doesn't mean it's not 2x from the expected for another angle-range.

It's also possible other parameters influencing run-times appart for angle-range can change between wu, or I've screwed-up the calculations... :oops:


BTW, the only real crunch-times I've got is with angle-range 0.4439, there the theoretical is 24.06h, but the actual is 36.36h so...

Oh, and this is with the unoptimized application...
ID: 152337 · Report as offensive
Profile cliff west

Send message
Joined: 7 May 01
Posts: 211
Credit: 16,180,728
RAC: 15
United States
Message 152339 - Posted: 16 Aug 2005, 21:52:28 UTC

i don't mind if it takes 24/48 hours to do a WU.. i'm first computer took 20 days to do a "classic" when i started... it make me to want to upgrade....my last computer i had made i told the guy that i wanted a system that could finish a WU every 2 hours...and i did get it... last i heared was that it is not that hard to get a (1) computer to do a WU every hour...lets make the game a little harder (the hole jone's thing).... buring on the pain of a 48 hr per work unit.. PLEASE!!!!!!
ID: 152339 · Report as offensive
Profile cliff west

Send message
Joined: 7 May 01
Posts: 211
Credit: 16,180,728
RAC: 15
United States
Message 152340 - Posted: 16 Aug 2005, 21:57:14 UTC

"unoptimized-client"

Okay you lost me... more info on a "optimized-client" is this like the old seti driver?
ID: 152340 · Report as offensive
KB7RZF
Volunteer tester
Avatar

Send message
Joined: 15 Aug 99
Posts: 9549
Credit: 3,308,926
RAC: 2
United States
Message 152347 - Posted: 16 Aug 2005, 22:25:32 UTC - in response to Message 152339.  
Last modified: 16 Aug 2005, 22:31:40 UTC

i don't mind if it takes 24/48 hours to do a WU.. i'm first computer took 20 days to do a "classic" when i started... it make me to want to upgrade....my last computer i had made i told the guy that i wanted a system that could finish a WU every 2 hours...and i did get it... last i heared was that it is not that hard to get a (1) computer to do a WU every hour...lets make the game a little harder (the hole jone's thing).... buring on the pain of a 48 hr per work unit.. PLEASE!!!!!!


My 2nd computer does a WU in about 2 horus 30 minutes, with just the optimized Core Client (4.45). Once the wu's run out, I gotta figure out which optimized seti app to use and could probably cut that time in half or more.

More information about an "optimized Client" can be found Here. This is Pauls Wiki page, a very informative website that can answer pretty much any and all questions.

Theres 2 places I know of that you can d/l optimized stuff from. 1 is from Here. The other is Here.

A very good thread to read is Here.

Hope this helps.

Jeremy

[edit]

Here is another thread about optimized clients. I did a search for them, and found them all very very helpful. Lots of people here with a lot of information and more than willing to help you out. Search through them. You will need to also know what your machine supports, SSE, SSE2, SSE3, MMX, and select the optimized client that will work with your machine. You will definately notice a difference with an optimized client. Good luck.


ID: 152347 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : bigger WUs


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.