About Deadlines or Database reduction proposals

Message boards : Number crunching : About Deadlines or Database reduction proposals
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 8 · 9 · 10 · 11 · 12 · 13 · 14 . . . 16 · Next

AuthorMessage
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19490
Credit: 40,757,560
RAC: 67
United Kingdom
Message 2034859 - Posted: 1 Mar 2020, 18:31:43 UTC - in response to Message 2034789.  

I'm going to say, I believe the terminology used on the Server Staus page is correct, and therefore must reject your theory.


Believing something doesn't make it true. Have a look at your own tasks waiting for validation: how many of them already have reached their quorum and are waiting for the validators to do their work? Obviously I didn't check all 500-something of them by hand, but I couldn't find any.

Couldn't agree more, but in this case, without confirmation, one way or the other, by person(s) I can trust, I couldn't say I know that the terminology on the SS screen is correct.

It is not the validators that are the problem. it is the ~800 tasks that have been validated before 18:00 29th Feb, 24 hours ago, but not yet purged, which normally happens.
https://setiathome.berkeley.edu/results.php?userid=8083616&offset=220&show_names=0&state=4&appid=29, therefore my assumption is that there is a blockage between Validation and the end of the purging process.
ID: 2034859 · Report as offensive     Reply Quote
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 2034861 - Posted: 1 Mar 2020, 18:51:10 UTC - in response to Message 2034849.  
Last modified: 1 Mar 2020, 18:55:11 UTC

If you are not worried about RAC then get rid of those excess ~10000 tasks and live with the 150 tasks per GPU.

Why? If my host crunch all of them in less than 1.5 days?
Give me a valid reason, not just because you not like that!

But as i posted before: RAC is totally out of topic! Please respect that!

This thread is About Deadlines or Database reduction proposals

You and anybody could start your own thread about RAC.
ID: 2034861 · Report as offensive     Reply Quote
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 2034862 - Posted: 1 Mar 2020, 18:51:17 UTC - in response to Message 2034806.  

Actually, your Host is showing a Turnaround time of over a Day, Average turnaround time: 1.22 days
Testing has shown any Task over the Daily production just adds Bloat to the Database, increases the turnaround time, and basically doesn't help matters. It would be best if Everyone lowered their cache to a Day or lower. If the cache is Lower than a Day it doesn't matter if the Cache is small or large, the machine will produce the same amount of work with either, and that is the number which effects the Database.

My suggestion to expedite the clear of this WU's is to send them only to the top 100 hosts (or whatever number needed to manage this retries) with the lowest possible APR with a very small dateline (lees than a week, maybe 3-5 days only). They will clear that very fast an squeeze the db. This hosts are very stable but of course some could crash, the WU still could not been validated after the new crunch (non canonical result), etc. In this case the small dateline will make the WU been send to another fast host to rinse & retry.


Cherry picking of which hosts get work will NEVER happen so forget that idea. The reason is quite simple SETI is a SCIENCE PROJECT and should NOT BE TREATED AS A COMPETITION. If you want a huge RAC then there are projects that pay massive credits and have stupidly short deadlines, go over to one of them, don't try and bully every project into having those project's low standards.

Sorry.... Who's talking about competition or RAC? Totally of topic!.... It's a reasonable idea to SQUEEZE the DB size, nothing else! Look at Richard post who prove i'm not wrong to suggest that!

BTW I not care about RAC. If that is what you think about me. Sorry.

Still believe: In desperate times you need to take desperate measures, and NEW ideas must be welcomed!
ID: 2034862 · Report as offensive     Reply Quote
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14687
Credit: 200,643,578
RAC: 874
United Kingdom
Message 2034863 - Posted: 1 Mar 2020, 19:04:31 UTC - in response to Message 2034862.  

Actually, your Host is showing a Turnaround time of over a Day, Average turnaround time: 1.22 days
And the average turnround time for the project as a whole (MB v8 tasks, on SSP) is 30.46 hours, or 1.269 days. He's pretty much on the button for reporting at the same time as the average wingmate, which is the most efficient of all.
ID: 2034863 · Report as offensive     Reply Quote
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 2034864 - Posted: 1 Mar 2020, 19:07:12 UTC - in response to Message 2034863.  

Actually, your Host is showing a Turnaround time of over a Day, Average turnaround time: 1.22 days
And the average turnround time for the project as a whole (MB v8 tasks, on SSP) is 30.46 hours, or 1.269 days. He's pretty much on the button for reporting at the same time as the average wingmate, which is the most efficient of all.

Thanks Richard for that info. If i understand that correct i'm fine with my settings.
ID: 2034864 · Report as offensive     Reply Quote
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 2034865 - Posted: 1 Mar 2020, 19:10:21 UTC - in response to Message 2034863.  
Last modified: 1 Mar 2020, 19:11:24 UTC

Yes, and if everyone lowered their cache to a Day, the Turnaround Time would probably drop significantly. I'd guess most people out there have their cache set a 10 days. Imagine if they All dropped it to a Day...
ID: 2034865 · Report as offensive     Reply Quote
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14687
Credit: 200,643,578
RAC: 874
United Kingdom
Message 2034867 - Posted: 1 Mar 2020, 19:23:18 UTC - in response to Message 2034865.  
Last modified: 1 Mar 2020, 19:26:49 UTC

Yes. If you spot any host in the database with a 10 day average turnround, please let us know.

I'll start the ball rolling with host 8873865 (average turnround 62.14 days) - that's gone up from 53.85 days when I first saw it.
ID: 2034867 · Report as offensive     Reply Quote
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 2034869 - Posted: 1 Mar 2020, 19:32:33 UTC - in response to Message 2034867.  

Strawman....
I was referring to Cache size, Not Turnaround Time. Although lowering the Cache size usually lowers the Turnaround time.
ID: 2034869 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2034870 - Posted: 1 Mar 2020, 19:37:46 UTC - in response to Message 2034867.  

So we should start a new thread titled "Hosts with excessive cache settings" or something like we have for the "Invalid Host Messaging" thread and begin a PM campaign to contact those hosts to reduce their cache settings to one day. Correct?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2034870 · Report as offensive     Reply Quote
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 2034871 - Posted: 1 Mar 2020, 19:40:33 UTC
Last modified: 1 Mar 2020, 19:49:57 UTC

This WU is a perfect example why the deadline must be reduced:

https://setiathome.berkeley.edu/workunit.php?wuid=3808664962

It's from Dec 27 and the deadline is 4 Mar and the second host not respond any valid WU, so it will need to be resent.

The wingmen host never return a valid WU all his WU ends on Timed out - no response.

So it's stay for almost 3 months on the DB.
ID: 2034871 · Report as offensive     Reply Quote
Kevin Olley

Send message
Joined: 3 Aug 99
Posts: 906
Credit: 261,085,289
RAC: 572
United Kingdom
Message 2034873 - Posted: 1 Mar 2020, 19:50:28 UTC
Last modified: 1 Mar 2020, 19:53:39 UTC

I got one like that

https://setiathome.berkeley.edu/workunit.php?wuid=3715153965

Crunched it 29th Oct it will roll over again on 17th Mar

Got 20+ from Dec
Kevin


ID: 2034873 · Report as offensive     Reply Quote
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 2034875 - Posted: 1 Mar 2020, 19:53:13 UTC - in response to Message 2034873.  
Last modified: 1 Mar 2020, 19:54:44 UTC

I got one like that

https://setiathome.berkeley.edu/workunit.php?wuid=3715153965

Crunched it 29th Oct it will roll over again on 17th Mar

An that is even worst... the new wingmen host also ends on time outs.
ID: 2034875 · Report as offensive     Reply Quote
Kevin Olley

Send message
Joined: 3 Aug 99
Posts: 906
Credit: 261,085,289
RAC: 572
United Kingdom
Message 2034876 - Posted: 1 Mar 2020, 19:55:16 UTC

Looking at the tail end of our pendings is not a pretty sight:-(
Kevin


ID: 2034876 · Report as offensive     Reply Quote
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19490
Credit: 40,757,560
RAC: 67
United Kingdom
Message 2034877 - Posted: 1 Mar 2020, 20:04:09 UTC - in response to Message 2034871.  

This one is from 24th Dec, https://setiathome.berkeley.edu/workunit.php?wuid=3804671429, the first wingman timed out on the 15th Feb.
Was then sent out to the 3rd computer, with new deadline of 8th April, but it looks like the last contact was when it downloaded the task.
Lets hope wingman #4 can complete it rapidly.
ID: 2034877 · Report as offensive     Reply Quote
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22666
Credit: 416,307,556
RAC: 380
United Kingdom
Message 2034878 - Posted: 1 Mar 2020, 20:04:36 UTC

No it is NOT, I've just gone through ALL the tasks on my cruncher, only 5 have been resent as a result of time-outs, but something around 140 are the results (that is on ~2700 tasks total of in-work, pending, inconclusive and valid.
As I said earlier the solution is to get rid of the backlog, and that CANNOT be done by sending out more tasks, but by clearing the not inconsiderable backlog of tasks. Then VERY SLOWLY start to allow users to have new tasks, and for EVERY user to live within the allowances set.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 2034878 · Report as offensive     Reply Quote
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 2034879 - Posted: 1 Mar 2020, 20:04:38 UTC - in response to Message 2034876.  
Last modified: 1 Mar 2020, 20:06:12 UTC

Looking at the tail end of our pendings is not a pretty sight:-(

My pendings are in the range of 6900 WU, about 15% are in this category "scary WU". That is why i believe if they reduce the deadline will reduce the DB size by at least 10%.
ID: 2034879 · Report as offensive     Reply Quote
Kevin Olley

Send message
Joined: 3 Aug 99
Posts: 906
Credit: 261,085,289
RAC: 572
United Kingdom
Message 2034881 - Posted: 1 Mar 2020, 20:29:24 UTC

Inconclusive's are just as bad:-(

https://setiathome.berkeley.edu/workunit.php?wuid=3774387032
Kevin


ID: 2034881 · Report as offensive     Reply Quote
Darrell Wilcox Project Donor
Volunteer tester

Send message
Joined: 11 Nov 99
Posts: 303
Credit: 180,954,940
RAC: 118
Vietnam
Message 2034899 - Posted: 2 Mar 2020, 0:02:53 UTC - in response to Message 2034881.  

Suggestion for NOW:

Reduce the number of tasks per CPU being sent. This is EASY and FAST to do by the
SETI stafff by changing the <max_wus_in_progress>. I suggest maybe 10 to start.

Reduce the number of tasks per GPU being sent also. This is EASY and FAST to do by the
SETI stafff by changing the <max_wus_in_progress_gpu>. I suggest 100 to start.

BEFORE the stones and arrows start flying, this is a temporary change suggested. It does
NOT require a longer term software change to implement some of the other suggestions,
and it doesn't require a fund raiser, purchase, install, transition for a bigger, faster server.

It is FAST and EASY, and can be done TODAY for the maintenance outage tomorrow,
since it does require a restart of the software (not a reboot).

<analogy>
Right now we have 13 or 14 seats on our 20 seat bus being occupied by people who won't
get off. We are still able to use the remaining 6 or 7 seats until we can get them off, one by
one, as their bus passes expire. And no one is being left behind, not even granny who can
barely walk up the bus ramp (and who also pays taxes to support the Metro).
</analogy>
ID: 2034899 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13884
Credit: 208,696,464
RAC: 304
Australia
Message 2034963 - Posted: 2 Mar 2020, 6:34:55 UTC - in response to Message 2034899.  

Suggestion for NOW:

Reduce the number of tasks per CPU being sent. This is EASY and FAST to do by the
SETI stafff by changing the <max_wus_in_progress>. I suggest maybe 10 to start.

Reduce the number of tasks per GPU being sent also. This is EASY and FAST to do by the
SETI stafff by changing the <max_wus_in_progress_gpu>. I suggest 100 to start.

BEFORE the stones and arrows start flying, this is a temporary change suggested. It does
NOT require a longer term software change to implement some of the other suggestions,
and it doesn't require a fund raiser, purchase, install, transition for a bigger, faster server.

It is FAST and EASY, and can be done TODAY for the maintenance outage tomorrow,
since it does require a restart of the software (not a reboot).
And will be of no benefit in reducing the database size in any significant or meaning full way, so why even do it?
The problem is the Results returned and awaiting validation number, not the Work in progress number. Why fiddle with something that isn't going to have any effect on the actual problem???
Grant
Darwin NT
ID: 2034963 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13884
Credit: 208,696,464
RAC: 304
Australia
Message 2034964 - Posted: 2 Mar 2020, 6:42:24 UTC - in response to Message 2034787.  

I'm going to say, I believe the terminology used on the Server Staus page is correct, and therefore must reject your theory.
What i posted isn't a theory, it's a statement of facts as they presently stand.
If you wish to contribute anything of value to this discussion, you need to understand what the problem is, and how it came about. And that requires understanding what is being discussed, which means understanding what the terms mean & apply to.
Since you chose to ignore the facts, then any input you continue to provide on this subject will not be of any relevance or use.
Grant
Darwin NT
ID: 2034964 · Report as offensive     Reply Quote
Previous · 1 . . . 8 · 9 · 10 · 11 · 12 · 13 · 14 . . . 16 · Next

Message boards : Number crunching : About Deadlines or Database reduction proposals


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.