Message boards :
Number crunching :
The spoofed client - Whatever about
Message board moderation
Previous · 1 · 2 · 3 · 4 · 5 · Next
Author | Message |
---|---|
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
Also think about it, the only time the 100 WU limit has any affect is during an outage. The rest of the time your GPU's can return work as quickly as they like and it makes no difference. . . That is only true up a processing rate of 100 tasks (the request limit) per 5 mins (the request interval). There was a time when raising this relationship would have been cause for raucous laughter, but when you consider the current processing rate of the heavy hitters is it really that far off? 8-{ Stephen :) |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
I mean, why should some people have Days of cache while others only have Hours? Sound fair to Me... . . Sorry Grant but you skipped the part just before that quote. He was arguing that it is NOT fair that low performance hosts can sit on 20 days worth of work until it times out, while high performance hosts cannot get more than a couple of hours of work at any one time. Stephen PS and I agree with him on that ... and apart from not fair it is NOT good for the project. |
pututu Send message Joined: 21 Jul 16 Posts: 12 Credit: 10,108,801 RAC: 6 |
I think all the BOINC projects that I've participated/contributed, there is a max number of tasks that you can download per client (not per machine which can be overcome by creating multiple instances other than project like GPUGrid which does not allow multiple instances). It appears that SETI does not have this limitation on the server side. I'm not sure if my statement is accurate or not. Perhaps something that the project admin can consider. |
Ian&Steve C. Send message Joined: 28 Sep 99 Posts: 4267 Credit: 1,282,604,591 RAC: 6,640 |
(personally my philosophy has always been your cache does not need to be bigger than 1 days work) I wish I could hold even a day's worth. On my fastest host and 6400 tasks, I'll exhaust that in about 8-9hrs under a normal WU distribution. I could reschedule it to hold up to 10,000 WUs, but BOINC gets really laggy with that many. That system processes about 17k WU per day. Seti@Home classic workunits: 29,492 CPU time: 134,419 hours |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
Sure, if everyone had the same daily limit, that would be best . . As long as that limit matches that host's daily productivity. , but that wouldn't be the case, and you would then be limiting slower systems. . . With the high ratio of low productivity hosts to high productivity hosts and the size of caches of 'wasted' WUs sitting on machines that will not process them for days or weeks or maybe not ever, I would not be so sure that the numbers are that different. Seti has never guaranteed work 24/7/365, and one of the main aim's of Seti is to make it possible for the widest number of systems to participate. . . NO! The purpose of SETI is to search the heavens for detectable signs of alien technology. The purpose of SETI@home is to recruit as many rank and file volunteers as possible to provide the processing power to do this. The philosophical intent is to enable and encourage 'anyone and everyone' to take an interest and take part. But SETI@home does not have a responsibility to stroke egos for people wanting to say "I've got 2000 tasks on my computer for SETI" when they are only processing 2 a day. The function of systematic limits is NOT about making individuals feel good but rather about having the system function smoothly, reliably and efficiently. With the ever-increasing data available to be processed the project needs as many volunteers as possible with as much crunching power as possible, we are losing the race. We are losing the 'Ma and pa' casual volunteers for many reasons and many of the newer recruits still have unrealistic expectations so they do not stay, so we need to make the most of those long term volunteers who stick it out and particularly the types motivated enough to throw lots of hardware(money) and expertise into the pot. Sacrificing project productivity to stroke a few egos is not practical to say the very least. Stephen |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
It's a numbers game. Consider 80000 machines with say 100 unused tasks each on them. Those tasks would easily compensate for the few extra hours the say 100 faster machines would use Why would setting the cache at one day limit the slower machines? The only change would be fewer unused tasks sitting on them. . . I am ROTFL, for once I had the dummy spit and you gave a nice quiet relaxed response to the same message :) Stephen :) |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13731 Credit: 208,696,464 RAC: 304 |
Stephen, this was all done & dusted over a week ago. It would have been be a good idea for you to read the thread in it's entirety, instead of posting & commenting on individual posts that had been resolved (or at least people agreed to disagree) many posts later in the thread. Grant Darwin NT |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
This PC is not mine but I give a thumb up for his/her effort to find ways to download more tasks per rig. . . I think there is more at play than multiple instances of BOINC on that machine. Who knows of an Intel 2.2GHz CPU with 250 cores? And I doubt it has 10 x GTX1080s. Stephen :( |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
Hi Stephen, thanks for the comments. Would you believed I flunked out of the only programming class I took in college 40 year ago. . . If they could see you now :) So I had to read the instructions on the BOINC website. And finally through a lot of trial and error I succeeded. I can follow along and get the gist of what the code is doing since it is all in high level C that reads like normal text. So in any particular small part of code I can actually understand what it is doing. DA or the other developers put in little helpful comments for various sections that for the most part explain what the code is supposed to do. Putting the whole of the code together to get the bigger picture is harder for me. . . And that's the whole point of distributed computing, it is a team effort. . . But as for this task ... it is the error part that concerns me :( Stephen :) |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
(personally my philosophy has always been your cache does not need to be bigger than 1 days work) . . Exactly, with the faster machines at work now the current limits are unrealistic. But as TBar asks, why should slow machines have 20 days worth of WUs sitting in limbo while the highly productive units are only getting enough work for an hour or 2. My units are far from bleeding edge but they empty the standard cache allowance in a few hours. But as you point out, there is a cost with very large caches in that the manager contracts 'one armed paper hanger' syndrome ... so there are limits that get you one way or another :) Stephen :) |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
Stephen, this was all done & dusted over a week ago. . . Yep, I came late to the party again ... :( Stephen :) |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
Stephen, this was all done & dusted over a week ago. Beter later than never. :) |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
. . Yep, I came late to the party again ... :( . . Thanks :) Stephen :) |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
. . Exactly, with the faster machines at work now the current limits are unrealistic. IMHO whats is unrealistic is to keep the size of the WU small (around 720K) with the current top crunchers who can crunch a WU in less than 30 sec. Instead of that small WU, could be created a new "large" MB WU who could be sended only to the fastest hosts. Like GPUgrid does with it's small/large WUs. That will make the spoofed client or rescheduling unnecessary. As Ian posted even the 6400WU cache is to small for the current top GPU crunchers. An over this number the lag is to high to allow to use larger buffers. For now the 100 CPU WU limit is still valid, but with the arrival of the new Ryzens with a lot of cores even that 100 CPU WU Limit will be come unrealistic too. |
rob smith Send message Joined: 7 Mar 03 Posts: 22190 Credit: 416,307,556 RAC: 380 |
Increasing the size of the task would mean a massive re-write of the whole system. There are many reasons why doing so is low on the list of priorities, not least of which would be having to re-run just about every data file, and of course the lack of people to do the required coding and do the pre-beta testing. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
Joseph Stateson Send message Joined: 27 May 99 Posts: 309 Credit: 70,759,933 RAC: 3 |
Increasing the size of the task would mean a massive re-write of the whole system. There are many reasons why doing so is low on the list of priorities, not least of which would be having to re-run just about every data file, and of course the lack of people to do the required coding and do the pre-beta testing. Milkyway had similar problem and some time ago bundled 4 "jobs" into a single work unit. I am guessing it was "unpacked" when handing off to the OpenCL app. Something like this might be easy to implement. |
TBar Send message Joined: 22 May 99 Posts: 5204 Credit: 840,779,836 RAC: 2,768 |
I believe I've already proven the fix is to simply change the cache to One Day, or even Half a Day. This will provide the fastest Hosts enough WUs to last through an average Tuesday Outage. The current problem is the Fastest 20% of SETI Hosts are about 300% faster than the other 80%. This results in any attempt to run a smaller cache self defeating as it results in a Large increase in the Pending tasks for those 20% of Hosts. There isn't any impact on the Database's 'results in the field' by those 20% of Fast Hosts running a vastly increased cache as the Pending Tasks number drops as the Cache number increases. The same Three Hosts running different caches; State: All (31078) · In progress (500) · Validation pending (17067) · Validation inconclusive (390) · Valid (13121) State: All (29443) · In progress (3600) · Validation pending (13368) · Validation inconclusive (456) · Valid (12019) State: All (30827) · In progress (6500) · Validation pending (11465) · Validation inconclusive (456) · Valid (12315) The Host with the Smallest Cache has the Largest Impact on the 'Results in the Field' with the current system. |
Joseph Stateson Send message Joined: 27 May 99 Posts: 309 Credit: 70,759,933 RAC: 3 |
I believe I've already proven the fix is to simply change the cache to One Day, or even Half a Day. This will provide the fastest Hosts enough WUs to last through an average Tuesday Outage. The current problem is the Fastest 20% of SETI Hosts are about 300% faster than the other 80%. This results in any attempt to run a smaller cache self defeating as it results in a Large increase in the Pending tasks for those 20% of Hosts. There isn't any impact on the Database's 'results in the field' by those 20% of Fast Hosts running a vastly increased cache as the Pending Tasks number drops as the Cache number increases. Missing something here, changing size did nothing for me. at both "seti preferences" website and at "boincstats" and at my local PC "boinc preferences" (all 3 just to be sure) did the follwing Mimimum buffer days 0.1 => 2.0 Additional work buffer days 0.25 => 2.0 did an update and a sync repeatedly, no change in work unit count. Still just under 900 work units (9 GPUs) verfied as shown below root@tb85-nvidia:/var/lib/boinc-client# grep -i ">2.000" *.xml global_prefs_override.xml: <work_buf_min_days>2.000000</work_buf_min_days> global_prefs_override.xml: <work_buf_additional_days>2.000000</work_buf_additional_days> sched_request_setiathome.berkeley.edu.xml: <work_buf_min_days>2.000000</work_buf_min_days> sched_request_setiathome.berkeley.edu.xml: <work_buf_additional_days>2.000000</work_buf_additional_days> root@tb85-nvidia:/var/lib/boinc-client# grep -i ">2.0" *.xml global_prefs.xml:<work_buf_min_days>2.0</work_buf_min_days> global_prefs.xml:<work_buf_additional_days>2.0</work_buf_additional_days> So I should have picked up addition work units in 7.14.2 ? |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
I believe I've already proven the fix is to simply change the cache to One Day, or even Half a Day. This will provide the fastest Hosts enough WUs to last through an average Tuesday Outage. The current problem is the Fastest 20% of SETI Hosts are about 300% faster than the other 80%. This results in any attempt to run a smaller cache self defeating as it results in a Large increase in the Pending tasks for those 20% of Hosts. There isn't any impact on the Database's 'results in the field' by those 20% of Fast Hosts running a vastly increased cache as the Pending Tasks number drops as the Cache number increases. With 9 gpus you are allotted 900 tasks. So your cache is full. There are no more to be retrieved no matter how many days of work you ask for. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
rob smith Send message Joined: 7 Mar 03 Posts: 22190 Credit: 416,307,556 RAC: 380 |
Reduce the additional work day to a much lower number - round about 0.01, thus the refreshes will be much smoother and much less prone to the humps and bumps that happen with large ( >1 ) additional days. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.