The spoofed client - Whatever about

Message boards : Number crunching : The spoofed client - Whatever about
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2005619 - Posted: 4 Aug 2019, 3:28:53 UTC - in response to Message 2004427.  

Also think about it, the only time the 100 WU limit has any affect is during an outage. The rest of the time your GPU's can return work as quickly as they like and it makes no difference.
My fastest machine is around 2.5 mins a task, it never "runs out" during normal working so the 100 task limit has no affect and I don't notice it .


. . That is only true up a processing rate of 100 tasks (the request limit) per 5 mins (the request interval). There was a time when raising this relationship would have been cause for raucous laughter, but when you consider the current processing rate of the heavy hitters is it really that far off? 8-{

Stephen

:)
ID: 2005619 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2005622 - Posted: 4 Aug 2019, 3:34:30 UTC - in response to Message 2004451.  
Last modified: 4 Aug 2019, 3:36:41 UTC

I mean, why should some people have Days of cache while others only have Hours? Sound fair to Me...
<nods head>

And by the same argument there shouldn't be any optimised applications because it means they will run out of work in outages. Nor should there be any recent high performance hardware used for crunching.
Sorry, but facetious arguments don't do anything to support your position, this one actually counters it and supports the other view.


. . Sorry Grant but you skipped the part just before that quote. He was arguing that it is NOT fair that low performance hosts can sit on 20 days worth of work until it times out, while high performance hosts cannot get more than a couple of hours of work at any one time.

Stephen

PS and I agree with him on that ... and apart from not fair it is NOT good for the project.
ID: 2005622 · Report as offensive
pututu

Send message
Joined: 21 Jul 16
Posts: 12
Credit: 10,108,801
RAC: 6
United States
Message 2005623 - Posted: 4 Aug 2019, 3:35:37 UTC

I think all the BOINC projects that I've participated/contributed, there is a max number of tasks that you can download per client (not per machine which can be overcome by creating multiple instances other than project like GPUGrid which does not allow multiple instances). It appears that SETI does not have this limitation on the server side. I'm not sure if my statement is accurate or not.

Perhaps something that the project admin can consider.
ID: 2005623 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 2005625 - Posted: 4 Aug 2019, 4:00:07 UTC - in response to Message 2005613.  
Last modified: 4 Aug 2019, 4:03:41 UTC

(personally my philosophy has always been your cache does not need to be bigger than 1 days work)


I wish I could hold even a day's worth. On my fastest host and 6400 tasks, I'll exhaust that in about 8-9hrs under a normal WU distribution. I could reschedule it to hold up to 10,000 WUs, but BOINC gets really laggy with that many.

That system processes about 17k WU per day.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2005625 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2005626 - Posted: 4 Aug 2019, 4:05:21 UTC - in response to Message 2004461.  

Sure, if everyone had the same daily limit, that would be best


. . As long as that limit matches that host's daily productivity.

, but that wouldn't be the case, and you would then be limiting slower systems.
And given the work slower systems can do, and what the faster systems can do, the reduction in the slower systems caches wouldn't be nearly enough to fill the faster system's caches.

. . With the high ratio of low productivity hosts to high productivity hosts and the size of caches of 'wasted' WUs sitting on machines that will not process them for days or weeks or maybe not ever, I would not be so sure that the numbers are that different.

Seti has never guaranteed work 24/7/365, and one of the main aim's of Seti is to make it possible for the widest number of systems to participate.

. . NO! The purpose of SETI is to search the heavens for detectable signs of alien technology. The purpose of SETI@home is to recruit as many rank and file volunteers as possible to provide the processing power to do this. The philosophical intent is to enable and encourage 'anyone and everyone' to take an interest and take part. But SETI@home does not have a responsibility to stroke egos for people wanting to say "I've got 2000 tasks on my computer for SETI" when they are only processing 2 a day. The function of systematic limits is NOT about making individuals feel good but rather about having the system function smoothly, reliably and efficiently. With the ever-increasing data available to be processed the project needs as many volunteers as possible with as much crunching power as possible, we are losing the race. We are losing the 'Ma and pa' casual volunteers for many reasons and many of the newer recruits still have unrealistic expectations so they do not stay, so we need to make the most of those long term volunteers who stick it out and particularly the types motivated enough to throw lots of hardware(money) and expertise into the pot. Sacrificing project productivity to stroke a few egos is not practical to say the very least.

Stephen
ID: 2005626 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2005627 - Posted: 4 Aug 2019, 4:09:26 UTC - in response to Message 2004466.  

It's a numbers game. Consider 80000 machines with say 100 unused tasks each on them. Those tasks would easily compensate for the few extra hours the say 100 faster machines would use Why would setting the cache at one day limit the slower machines? The only change would be fewer unused tasks sitting on them.

Actually I don't care if everyones machines goes down a few hours a day, I just thought it would be nice to try to Help SETI when they ask for more production. As I see it the tasks are available, they just need to be available to the machines that can use them. Right now that doesn't appear to be a problem, I certainly don't see SETI complaining about tasks being made available to the machines that need them. The only complaints I see are from a few private individuals on this board.


. . I am ROTFL, for once I had the dummy spit and you gave a nice quiet relaxed response to the same message :)

Stephen

:)
ID: 2005627 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13722
Credit: 208,696,464
RAC: 304
Australia
Message 2005628 - Posted: 4 Aug 2019, 4:41:07 UTC
Last modified: 4 Aug 2019, 4:42:38 UTC

Stephen, this was all done & dusted over a week ago.
It would have been be a good idea for you to read the thread in it's entirety, instead of posting & commenting on individual posts that had been resolved (or at least people agreed to disagree) many posts later in the thread.
Grant
Darwin NT
ID: 2005628 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2005636 - Posted: 4 Aug 2019, 6:05:44 UTC - in response to Message 2005605.  
Last modified: 4 Aug 2019, 6:06:24 UTC

This PC is not mine but I give a thumb up for his/her effort to find ways to download more tasks per rig.
Running multiple BOINC instances on one PC. Appears there are 4 instances running on one 1080 card most likely due to memory size limitation hence the run time for 1080 appears about 4 times slower than mine.
https://setiathome.berkeley.edu/show_host_detail.php?hostid=8782554

. . I think there is more at play than multiple instances of BOINC on that machine. Who knows of an Intel 2.2GHz CPU with 250 cores? And I doubt it has 10 x GTX1080s.

Stephen

:(
ID: 2005636 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2005640 - Posted: 4 Aug 2019, 6:19:19 UTC - in response to Message 2005616.  

Hi Stephen, thanks for the comments. Would you believed I flunked out of the only programming class I took in college 40 year ago.

. . If they could see you now :)

So I had to read the instructions on the BOINC website. And finally through a lot of trial and error I succeeded. I can follow along and get the gist of what the code is doing since it is all in high level C that reads like normal text. So in any particular small part of code I can actually understand what it is doing. DA or the other developers put in little helpful comments for various sections that for the most part explain what the code is supposed to do. Putting the whole of the code together to get the bigger picture is harder for me.

Sleuthing out the parts that control how many tasks are allowed on board a host at any time is easy to find and change. Tbar hinted that was all he did to generate the 7.4.44 client. Change one value and voila you get 10,000 tasks allowed instead of 1000. Add a zero is all I did. It took another team member to piece the spoofed gpu count together with my assistance since his programming knowledge was only 30 years old and not 40 years like me. So it was a team effort that created the spoofed client through a lot of iteration and testing and trial and error.

. . And that's the whole point of distributed computing, it is a team effort.

. . But as for this task ... it is the error part that concerns me :(

Stephen

:)
ID: 2005640 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2005641 - Posted: 4 Aug 2019, 6:25:01 UTC - in response to Message 2005625.  
Last modified: 4 Aug 2019, 6:34:59 UTC

(personally my philosophy has always been your cache does not need to be bigger than 1 days work)

I wish I could hold even a day's worth. On my fastest host and 6400 tasks, I'll exhaust that in about 8-9hrs under a normal WU distribution. I could reschedule it to hold up to 10,000 WUs, but BOINC gets really laggy with that many.
That system processes about 17k WU per day.

. . Exactly, with the faster machines at work now the current limits are unrealistic. But as TBar asks, why should slow machines have 20 days worth of WUs sitting in limbo while the highly productive units are only getting enough work for an hour or 2. My units are far from bleeding edge but they empty the standard cache allowance in a few hours. But as you point out, there is a cost with very large caches in that the manager contracts 'one armed paper hanger' syndrome ... so there are limits that get you one way or another :)

Stephen

:)
ID: 2005641 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2005643 - Posted: 4 Aug 2019, 6:40:36 UTC - in response to Message 2005628.  

Stephen, this was all done & dusted over a week ago.
It would have been be a good idea for you to read the thread in it's entirety, instead of posting & commenting on individual posts that had been resolved (or at least people agreed to disagree) many posts later in the thread.


. . Yep, I came late to the party again ... :(

Stephen

:)
ID: 2005643 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 2005686 - Posted: 4 Aug 2019, 11:45:48 UTC - in response to Message 2005643.  

Stephen, this was all done & dusted over a week ago.
It would have been be a good idea for you to read the thread in it's entirety, instead of posting & commenting on individual posts that had been resolved (or at least people agreed to disagree) many posts later in the thread.


. . Yep, I came late to the party again ... :(

Stephen

:)

Beter later than never.

:)
ID: 2005686 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2005699 - Posted: 4 Aug 2019, 13:47:57 UTC - in response to Message 2005686.  

. . Yep, I came late to the party again ... :(
Stephen
:)

Beter later than never.
:)

. . Thanks :)

Stephen

:)
ID: 2005699 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 2005701 - Posted: 4 Aug 2019, 14:14:00 UTC - in response to Message 2005641.  
Last modified: 4 Aug 2019, 14:17:37 UTC

. . Exactly, with the faster machines at work now the current limits are unrealistic.

IMHO whats is unrealistic is to keep the size of the WU small (around 720K) with the current top crunchers who can crunch a WU in less than 30 sec.

Instead of that small WU, could be created a new "large" MB WU who could be sended only to the fastest hosts. Like GPUgrid does with it's small/large WUs. That will make the spoofed client or rescheduling unnecessary.

As Ian posted even the 6400WU cache is to small for the current top GPU crunchers. An over this number the lag is to high to allow to use larger buffers.

For now the 100 CPU WU limit is still valid, but with the arrival of the new Ryzens with a lot of cores even that 100 CPU WU Limit will be come unrealistic too.
ID: 2005701 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22160
Credit: 416,307,556
RAC: 380
United Kingdom
Message 2005716 - Posted: 4 Aug 2019, 16:17:21 UTC

Increasing the size of the task would mean a massive re-write of the whole system. There are many reasons why doing so is low on the list of priorities, not least of which would be having to re-run just about every data file, and of course the lack of people to do the required coding and do the pre-beta testing.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 2005716 · Report as offensive
Profile Joseph Stateson Project Donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 309
Credit: 70,759,933
RAC: 3
United States
Message 2005725 - Posted: 4 Aug 2019, 17:16:36 UTC - in response to Message 2005716.  

Increasing the size of the task would mean a massive re-write of the whole system. There are many reasons why doing so is low on the list of priorities, not least of which would be having to re-run just about every data file, and of course the lack of people to do the required coding and do the pre-beta testing.


Milkyway had similar problem and some time ago bundled 4 "jobs" into a single work unit. I am guessing it was "unpacked" when handing off to the OpenCL app. Something like this might be easy to implement.
ID: 2005725 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 2005731 - Posted: 4 Aug 2019, 17:52:05 UTC
Last modified: 4 Aug 2019, 18:13:50 UTC

I believe I've already proven the fix is to simply change the cache to One Day, or even Half a Day. This will provide the fastest Hosts enough WUs to last through an average Tuesday Outage. The current problem is the Fastest 20% of SETI Hosts are about 300% faster than the other 80%. This results in any attempt to run a smaller cache self defeating as it results in a Large increase in the Pending tasks for those 20% of Hosts. There isn't any impact on the Database's 'results in the field' by those 20% of Fast Hosts running a vastly increased cache as the Pending Tasks number drops as the Cache number increases.
The same Three Hosts running different caches;
State: All (31078) · In progress (500) · Validation pending (17067) · Validation inconclusive (390) · Valid (13121)
State: All (29443) · In progress (3600) · Validation pending (13368) · Validation inconclusive (456) · Valid (12019)
State: All (30827) · In progress (6500) · Validation pending (11465) · Validation inconclusive (456) · Valid (12315)

The Host with the Smallest Cache has the Largest Impact on the 'Results in the Field' with the current system.
ID: 2005731 · Report as offensive
Profile Joseph Stateson Project Donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 309
Credit: 70,759,933
RAC: 3
United States
Message 2005749 - Posted: 4 Aug 2019, 20:11:44 UTC - in response to Message 2005731.  
Last modified: 4 Aug 2019, 20:13:30 UTC

I believe I've already proven the fix is to simply change the cache to One Day, or even Half a Day. This will provide the fastest Hosts enough WUs to last through an average Tuesday Outage. The current problem is the Fastest 20% of SETI Hosts are about 300% faster than the other 80%. This results in any attempt to run a smaller cache self defeating as it results in a Large increase in the Pending tasks for those 20% of Hosts. There isn't any impact on the Database's 'results in the field' by those 20% of Fast Hosts running a vastly increased cache as the Pending Tasks number drops as the Cache number increases.
The same Three Hosts running different caches;
State: All (31078) · In progress (500) · Validation pending (17067) · Validation inconclusive (390) · Valid (13121)
State: All (29443) · In progress (3600) · Validation pending (13368) · Validation inconclusive (456) · Valid (12019)
State: All (30827) · In progress (6500) · Validation pending (11465) · Validation inconclusive (456) · Valid (12315)

The Host with the Smallest Cache has the Largest Impact on the 'Results in the Field' with the current system.


Missing something here, changing size did nothing for me.
at both "seti preferences" website and at "boincstats" and at my local PC "boinc preferences" (all 3 just to be sure) did the follwing

Mimimum buffer days 0.1 => 2.0
Additional work buffer days 0.25 => 2.0
did an update and a sync repeatedly, no change in work unit count. Still just under 900 work units (9 GPUs)

verfied as shown below

root@tb85-nvidia:/var/lib/boinc-client# grep -i ">2.000" *.xml
global_prefs_override.xml:   <work_buf_min_days>2.000000</work_buf_min_days>
global_prefs_override.xml:   <work_buf_additional_days>2.000000</work_buf_additional_days>
sched_request_setiathome.berkeley.edu.xml:   <work_buf_min_days>2.000000</work_buf_min_days>
sched_request_setiathome.berkeley.edu.xml:   <work_buf_additional_days>2.000000</work_buf_additional_days>

root@tb85-nvidia:/var/lib/boinc-client# grep -i ">2.0" *.xml
global_prefs.xml:<work_buf_min_days>2.0</work_buf_min_days>
global_prefs.xml:<work_buf_additional_days>2.0</work_buf_additional_days>


So I should have picked up addition work units in 7.14.2 ?
ID: 2005749 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2005752 - Posted: 4 Aug 2019, 20:28:23 UTC - in response to Message 2005749.  

I believe I've already proven the fix is to simply change the cache to One Day, or even Half a Day. This will provide the fastest Hosts enough WUs to last through an average Tuesday Outage. The current problem is the Fastest 20% of SETI Hosts are about 300% faster than the other 80%. This results in any attempt to run a smaller cache self defeating as it results in a Large increase in the Pending tasks for those 20% of Hosts. There isn't any impact on the Database's 'results in the field' by those 20% of Fast Hosts running a vastly increased cache as the Pending Tasks number drops as the Cache number increases.
The same Three Hosts running different caches;
State: All (31078) · In progress (500) · Validation pending (17067) · Validation inconclusive (390) · Valid (13121)
State: All (29443) · In progress (3600) · Validation pending (13368) · Validation inconclusive (456) · Valid (12019)
State: All (30827) · In progress (6500) · Validation pending (11465) · Validation inconclusive (456) · Valid (12315)

The Host with the Smallest Cache has the Largest Impact on the 'Results in the Field' with the current system.


Missing something here, changing size did nothing for me.
at both "seti preferences" website and at "boincstats" and at my local PC "boinc preferences" (all 3 just to be sure) did the follwing

Mimimum buffer days 0.1 => 2.0
Additional work buffer days 0.25 => 2.0
did an update and a sync repeatedly, no change in work unit count. Still just under 900 work units (9 GPUs)

verfied as shown below

root@tb85-nvidia:/var/lib/boinc-client# grep -i ">2.000" *.xml
global_prefs_override.xml:   <work_buf_min_days>2.000000</work_buf_min_days>
global_prefs_override.xml:   <work_buf_additional_days>2.000000</work_buf_additional_days>
sched_request_setiathome.berkeley.edu.xml:   <work_buf_min_days>2.000000</work_buf_min_days>
sched_request_setiathome.berkeley.edu.xml:   <work_buf_additional_days>2.000000</work_buf_additional_days>

root@tb85-nvidia:/var/lib/boinc-client# grep -i ">2.0" *.xml
global_prefs.xml:<work_buf_min_days>2.0</work_buf_min_days>
global_prefs.xml:<work_buf_additional_days>2.0</work_buf_additional_days>


So I should have picked up addition work units in 7.14.2 ?

With 9 gpus you are allotted 900 tasks. So your cache is full. There are no more to be retrieved no matter how many days of work you ask for.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2005752 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22160
Credit: 416,307,556
RAC: 380
United Kingdom
Message 2005753 - Posted: 4 Aug 2019, 20:28:39 UTC

Reduce the additional work day to a much lower number - round about 0.01, thus the refreshes will be much smoother and much less prone to the humps and bumps that happen with large ( >1 ) additional days.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 2005753 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · Next

Message boards : Number crunching : The spoofed client - Whatever about


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.