Father Padilla Meets the Perfect Gnat (Dec 03 2007)

Message boards : Technical News : Father Padilla Meets the Perfect Gnat (Dec 03 2007)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3

AuthorMessage
DJStarfox

Send message
Joined: 23 May 01
Posts: 1066
Credit: 1,226,053
RAC: 2
United States
Message 688931 - Posted: 5 Dec 2007, 13:29:56 UTC - in response to Message 688851.  

I'm not saying it hurts anything. I know it doesn't hurt anything. It just annoys me, mainly because I like turning work around promptly. I'm not trying to hammer the servers, and I'm perfectly willing to wait the normal two and a half hours between result upload and result reporting. I just don't want to wait 24 hours. It seems overly long, to me.

How is it pounding the system to be waiting a couple hours in the first place? How does increasing that to 24 hours help?


BOINC will report all results sooner if it tries to get more work before 24 hours. With deadlines in the several weeks range, a few hours of delay in reporting won't make a significant difference in terms of credit granted.
ID: 688931 · Report as offensive
JLDun
Volunteer tester
Avatar

Send message
Joined: 21 Apr 06
Posts: 573
Credit: 196,101
RAC: 0
United States
Message 689115 - Posted: 6 Dec 2007, 3:13:53 UTC - in response to Message 688669.  

Since it´s the same software that do this job over the same data?

Check out Result 644605495 (Comp. 3770617) versus Result 648500138 (Comp. 3941110); (Workunit 172612436)
Two differences:
1. One is an AMD64 Duo, one is an Intel Quad
2. One found Spike=1 Pulse=30 -9 Overflow, One found Spike=1 Pulse=0.
ID: 689115 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 689168 - Posted: 6 Dec 2007, 5:16:41 UTC - in response to Message 688811.  

And of course, the 5.10.14+ clients force you to sit on your completed results for a full 24 hours before reporting, which annoys me greatly.

This is on purpose to stop the pounding of the system. It was meant to be this way in the first place, and is finally working the way it's supposed to.

It doesn't hurt anything.


Actually, it does hurt if your result is the one that is going to form a quorum. This means that there is a forced delay to forming the quorum and thus being able to possibly determine validity, declare a canonical result, and move the work down the assimilation pathway to be able to clear it from storage.

What I'm seeing develop is some BOINC developers are not fully thinking out their ideas. I appreciate that the goal (Rom's?) was to cut down on the database hits on result reporting. However, one must consider the effects of such all the way down the chain. If they thought about the consequence as for result storage and it was determined that the benefits outweighed the risks, then great. If, however, nobody even thought about that (much like it is apparent that there wasn't much thought put into auto-hiding messages if someone is banned), then that is a rushed design process and is ultimately not good for BOINC as a whole.

Brian

As usual, there are trade offs.

Reporting the work immediately pounds the Database Server much harder than collecting several reports and a work request together. There is an analysis of exactly why around somewhere. The basic problem is there is a very large penalty for spinning up a Database connection and getting to the users data to do the report or work fetch.

However, delaying reports can hurt data storage somewhat.

Most projects, including, I believe, S@H, have more trouble with the Database Server and its processes than with data storage.


BOINC WIKI
ID: 689168 · Report as offensive
Brian Silvers

Send message
Joined: 11 Jun 99
Posts: 1681
Credit: 492,052
RAC: 0
United States
Message 689171 - Posted: 6 Dec 2007, 5:42:12 UTC - in response to Message 689168.  


Most projects, including, I believe, S@H, have more trouble with the Database Server and its processes than with data storage.


Yes, and that's why you're correct about stating a tradeoff. Hopefully the positives outweighed the negatives...
ID: 689171 · Report as offensive
Profile The Gas Giant
Volunteer tester
Avatar

Send message
Joined: 22 Nov 01
Posts: 1904
Credit: 2,646,654
RAC: 0
Australia
Message 689231 - Posted: 6 Dec 2007, 11:49:44 UTC
Last modified: 6 Dec 2007, 11:50:07 UTC

Sorry, but I haven't read the whole thread but....regarding the issue of the wu's having to "stick" around on the disk until the "wingman" returns the result, why set the deadlines so long? If you can only have a max cache level of 10 days, then why have a deadline that is longer than 12 days from the date the result is sent out? This would mean that any result not returned by the wingman would be turned around more quickly. I still don't understand why the deadlines are soooo looong.

Live long and BOINC!
Paul
(S@H1 8888)
And proud of it!
ID: 689231 · Report as offensive
Profile Jim-R.
Volunteer tester
Avatar

Send message
Joined: 7 Feb 06
Posts: 1494
Credit: 194,148
RAC: 0
United States
Message 689290 - Posted: 6 Dec 2007, 16:03:07 UTC - in response to Message 689231.  

Sorry, but I haven't read the whole thread but....regarding the issue of the wu's having to "stick" around on the disk until the "wingman" returns the result, why set the deadlines so long? If you can only have a max cache level of 10 days, then why have a deadline that is longer than 12 days from the date the result is sent out? This would mean that any result not returned by the wingman would be turned around more quickly. I still don't understand why the deadlines are soooo looong.

Live long and BOINC!

I believe the reason for longer deadlines is because it seems the project is using "any Pentium class computer" as a minimum requirement. My P3 500mhz computer requires around two days to process a workunit, and slower comps have much longer crunch times per wu.
I agree that the present deadlines could possibly be shortened somewhat without further restricting the minimum computer requirements. Too much shortening of the deadlines will raise the minimum requirements. There has already been much discussion of the value and economy of running such old hosts, however I know from experience that some of us just simply can't afford anything better. (My absolute best computer is a 1.7ghz but it has problems and I've never been able to run it and my best crunching computer is a 1.3ghz Celeron with power supply problems at the moment) So while the deadlines as they are do seem to be excessive, any decrease in them would have to be done carefully with due consideration to the minimum computer requirements.
Jim

Some people plan their life out and look back at the wealth they've had.
Others live life day by day and look back at the wealth of experiences and enjoyment they've had.
ID: 689290 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 689292 - Posted: 6 Dec 2007, 16:08:46 UTC - in response to Message 689231.  

Sorry, but I haven't read the whole thread but....regarding the issue of the wu's having to "stick" around on the disk until the "wingman" returns the result, why set the deadlines so long? If you can only have a max cache level of 10 days, then why have a deadline that is longer than 12 days from the date the result is sent out? This would mean that any result not returned by the wingman would be turned around more quickly. I still don't understand why the deadlines are soooo looong.

Recent versions of BOINC allow setting a 10 day connect interval plus 10 days of extra work, total 20 days queue. But work fetch is done based on the Whetstone benchmark, project Duration Correction Factor, and usage stats of the host, so is not directly comparable to deadlines.

Work fetch and deadlines do share a common factor, the estimate of how much work is in a particular WU. The splitter produces that estimate using formulas which were developed very early in the setiathome_enhanced development, and those only very roughly match actual crunch times. The deadline theoretically allows a host with a 1000 Whetstone MIPS benchmark to participate in 30 projects with equal shares, or one with a 33.33 Whetstone MIPS benchmark to do S@H only, in both cases assuming 24/7 crunching. A typical home or small business system host which is turned off when not in use for other purposes might only give BOINC a few hours of CPU time per day.

Thus, the "soooo looong" deadlines are a combination of inaccurate estimates and what sort of minimum contribution is supported. Perhaps the minimum contribution should gradually grow over time, but it was already tripled for _enhanced and looks reasonable to me. If the estimate formulas were adjusted to reflect the actual crunch times, the longest deadline might drop from nearly 4 months to about 2 months and the shortest deadline increase from 8.7 days to nearly 12 days.

Bringing this discussion back to server related issues, correcting the estimates would also make the Duration Correction Factor on each host much more stable, and the Scheduler would be assigning about the right number of tasks to fulfill work requests. Transition from midrange work to high slew (high angle range) work would assign 3.5 to 4 times as many WUs to a host rather than the current 6 or 7, for instance. That wouldn't make high slew work trouble-free, but achieving download flow of 100 to 120 MB/s seems more likely than the 180 to 210 needed to gracefully handle the transition now.
                                                                Joe
ID: 689292 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 689299 - Posted: 6 Dec 2007, 16:55:05 UTC - in response to Message 688931.  
Last modified: 6 Dec 2007, 17:00:10 UTC

I'm not saying it hurts anything. I know it doesn't hurt anything. It just annoys me, mainly because I like turning work around promptly. I'm not trying to hammer the servers, and I'm perfectly willing to wait the normal two and a half hours between result upload and result reporting. I just don't want to wait 24 hours. It seems overly long, to me.

How is it pounding the system to be waiting a couple hours in the first place? How does increasing that to 24 hours help?


BOINC will report all results sooner if it tries to get more work before 24 hours. With deadlines in the several weeks range, a few hours of delay in reporting won't make a significant difference in terms of credit granted.



BOINC will also immediately report results if you hit the "update" button. Important to know if (like me) some or all of your computers are still on dial-up!

(Only one of the computers I crunch with has a 24/7 I-net connection!)
.

Hello, from Albany, CA!...
ID: 689299 · Report as offensive
Profile [KWSN]John Galt 007
Volunteer tester
Avatar

Send message
Joined: 9 Nov 99
Posts: 2444
Credit: 25,086,197
RAC: 0
United States
Message 689304 - Posted: 6 Dec 2007, 17:03:02 UTC - in response to Message 689299.  

I'm not saying it hurts anything. I know it doesn't hurt anything. It just annoys me, mainly because I like turning work around promptly. I'm not trying to hammer the servers, and I'm perfectly willing to wait the normal two and a half hours between result upload and result reporting. I just don't want to wait 24 hours. It seems overly long, to me.

How is it pounding the system to be waiting a couple hours in the first place? How does increasing that to 24 hours help?


BOINC will report all results sooner if it tries to get more work before 24 hours. With deadlines in the several weeks range, a few hours of delay in reporting won't make a significant difference in terms of credit granted.



BOINC will also immediately report results if you hit the "update" button. Important to know if (like me) some or all of your computers are still on dial-up!

(Only one of the computers I crunch with has a 24/7 I-net connection!)


Dial-up??? Dial-up what??

;-)>

I remeber those days...even as far back as 300 baud...I can read faster than that....

Clk2HlpSetiCty:::PayIt4ward

ID: 689304 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 689306 - Posted: 6 Dec 2007, 17:08:37 UTC - in response to Message 689304.  
Last modified: 6 Dec 2007, 17:09:36 UTC

[snip]
BOINC will report all results sooner if it tries to get more work before 24 hours. With deadlines in the several weeks range, a few hours of delay in reporting won't make a significant difference in terms of credit granted.



BOINC will also immediately report results if you hit the "update" button. Important to know if (like me) some or all of your computers are still on dial-up!

(Only one of the computers I crunch with has a 24/7 I-net connection!)


Dial-up??? Dial-up what??

;-)>


Dial-up modems, of course... I use a 56K, and network all three of my home computers (with more on the horizon!) through it.

I remember those days...even as far back as 300 baud...I can read faster than that....


How about 110 Baud over a Teletype 33? I actually ran games on this combination back in '73...
.

Hello, from Albany, CA!...
ID: 689306 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 689309 - Posted: 6 Dec 2007, 17:15:10 UTC - in response to Message 689292.  

Sorry, but I haven't read the whole thread but....regarding the issue of the wu's having to "stick" around on the disk until the "wingman" returns the result, why set the deadlines so long? If you can only have a max cache level of 10 days, then why have a deadline that is longer than 12 days from the date the result is sent out? This would mean that any result not returned by the wingman would be turned around more quickly. I still don't understand why the deadlines are soooo looong.

Recent versions of BOINC allow setting a 10 day connect interval plus 10 days of extra work, total 20 days queue. But work fetch is done based on the Whetstone benchmark, project Duration Correction Factor, and usage stats of the host, so is not directly comparable to deadlines.

Work fetch and deadlines do share a common factor, the estimate of how much work is in a particular WU. The splitter produces that estimate using formulas which were developed very early in the setiathome_enhanced development, and those only very roughly match actual crunch times. The deadline theoretically allows a host with a 1000 Whetstone MIPS benchmark to participate in 30 projects with equal shares, or one with a 33.33 Whetstone MIPS benchmark to do S@H only, in both cases assuming 24/7 crunching. A typical home or small business system host which is turned off when not in use for other purposes might only give BOINC a few hours of CPU time per day.

Thus, the "soooo looong" deadlines are a combination of inaccurate estimates and what sort of minimum contribution is supported. Perhaps the minimum contribution should gradually grow over time, but it was already tripled for _enhanced and looks reasonable to me. If the estimate formulas were adjusted to reflect the actual crunch times, the longest deadline might drop from nearly 4 months to about 2 months and the shortest deadline increase from 8.7 days to nearly 12 days.

Bringing this discussion back to server related issues, correcting the estimates would also make the Duration Correction Factor on each host much more stable, and the Scheduler would be assigning about the right number of tasks to fulfill work requests. Transition from midrange work to high slew (high angle range) work would assign 3.5 to 4 times as many WUs to a host rather than the current 6 or 7, for instance. That wouldn't make high slew work trouble-free, but achieving download flow of 100 to 120 MB/s seems more likely than the 180 to 210 needed to gracefully handle the transition now.
                                                                Joe

I see I'm starting to get high slew 'shorty' work from 15no06aa - I hope that's worked its way out of the download queue, and is no longer active at the head of the splitter list, when the project starts up again tomorrow morning after the electrical supply outage.
ID: 689309 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 689387 - Posted: 7 Dec 2007, 18:10:18 UTC - in response to Message 689309.  

[snip]
I see I'm starting to get high slew 'shorty' work from 15no06aa - I hope that's worked its way out of the download queue, and is no longer active at the head of the splitter list, when the project starts up again tomorrow morning after the electrical supply outage.


No Such Luck - 15no06aa is still there after the restart.
.

Hello, from Albany, CA!...
ID: 689387 · Report as offensive
Brian Silvers

Send message
Joined: 11 Jun 99
Posts: 1681
Credit: 492,052
RAC: 0
United States
Message 689389 - Posted: 7 Dec 2007, 18:24:58 UTC - in response to Message 689387.  

[snip]
I see I'm starting to get high slew 'shorty' work from 15no06aa - I hope that's worked its way out of the download queue, and is no longer active at the head of the splitter list, when the project starts up again tomorrow morning after the electrical supply outage.


No Such Luck - 15no06aa is still there after the restart.


I wish I had this much luck at predicting when it comes to lottery numbers or stocks to buy (and when to sell them)...
ID: 689389 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 691449 - Posted: 14 Dec 2007, 18:17:36 UTC - in response to Message 688619.  

While I see a debate if 2/2 or 3/2 is the best choice... I wonder if effort was already directed to 1/1 scheme.

Why exactly this aproach is not possible ?

Think about double the virtual machine power is a worthy reward...


...just ideas, by a newbee user...

It isn't desirable because all of the relevant software is open source, and the machines doing the crunching are not operated by the project.

An over-optimized SETI science application might process work really fast, but return erroneous results. Each work unit is crunched by two computers so they may be compared, and if needed sent to more computers until there are at least two matching results.

Without the cross-check, valid signals might be lost.
ID: 691449 · Report as offensive
Previous · 1 · 2 · 3

Message boards : Technical News : Father Padilla Meets the Perfect Gnat (Dec 03 2007)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.