Message boards :
Number crunching :
For the betterment of BOINC
Message board moderation
Previous · 1 · 2 · 3 · 4 · Next
Author | Message |
---|---|
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
Thanks for starting us off, Grant. I'd like as many answers as possible by the end of the week, please, and then I'll tot them up. |
tullio Send message Joined: 9 Apr 04 Posts: 8797 Credit: 2,930,782 RAC: 1 |
As a BOINC user in six projects, there is one feature which I value most and that is the easiness of contact with the projects admin/developers, which was easy in SETI@home until the Breakthrough Listen project started and people moved over there for good reasons. There are projects where you can have an answer to a problem in a short time and those in which you never receive an answer. E.g. almost all my LHC@home tasks fail now, while I was an Alpha tester in Test4Theory@home. Nothing has changed in my HW/SW configuration, and I could get no help, so I abandoned the project. Same for Einstein@home on my Windows 10 PC after the Creators Update. They all fail, both on CPU and GPU, so I am running only SETI@ home and SETI Beta on that PC, my fastest. Tullio |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
Although it's a little off-topic for this discussion, I can tell you that the project administrator for the LHC Classic (Sixtrack) project is very pro-actively chasing down some infrastructure bugs right now, but hasn't yet reached a stage where he can post success on the message boards. But the general point of good communication between projects (scientists and administrators alike) and their volunteers is well taken, and I would add that I would personally like to see better communication between projects and the core BOINC team, too. |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
How about adding Seti Beta to "the list".The add-project list is only populated by production projects, the alpha- and beta-projects aren't listed. If a production project has an alpha- or beta-project of their own, they usually post about that on their front page, or in a stickied thread. If they do not, best ask that they do. |
Darrell Send message Joined: 14 Mar 03 Posts: 267 Credit: 1,418,681 RAC: 0 |
Can I ask a serious question at this point - without giving away any clues by suggesting possible answers. I want to take a straw poll to get a feeling for what people are expecting. From running Boinc with caches set to zero (i.e. getting one task at a time) if a project has both CPU and GPU tasks, and you have selected to use both, the current scheduler will only give you GPU tasks when attached to multiple projects. With the current GPU scheduler when attached to multiple projects that have GPU tasks and you set some of the projects to say run two tasks at a time on the GPU, the scheduler does not give you two tasks from the same project, but one from project A and one from project B. For resource allocation I would like to Boinc to first look at the number of cores/threads. With my six core processor, if I set Boinc to use 5 cores (84% of CPU) this leaves one core free to run the system with little screen lag. For each GPU task on the GPU I want Boinc to give it one core to feed each task. So if I have one running GPU task, the Boinc should run 4 CPU tasks based upon project shares. If I have two GPU tasks running, then Boinc should run three CPU tasks. Boinc also needs to take in account for CPU tasks the number of cores need by multi-threaded apps and the number of cores to be used by multi-threaded apps should be easily settable. Next resource allocation to be looked at should be time. PrimeGrid is the project of concern here. It has short GPU tasks ( < 1hr ), medium GPU tasks ( 1 -2 hrs), and long GPU tasks (24 hours and greater). So if a PrimeGrid long task runs, the Boinc needs to back-off work requests from the project, to allow the other projects to get their share of time. Ideally, it would be nice to set the PrimeGrid scheduler to something like give 4 short tasks or 2 medium tasks for every long task run, and balance that out with the sharing of time with other projects. ... and still I fear, and still I dare not laugh at the Mad Man! Queen - The Prophet's Song |
Shaggie76 Send message Joined: 9 Oct 09 Posts: 282 Credit: 271,858,118 RAC: 196 |
I'm not sure if it's a BOINC problem or a SETI@Home problem but the Tuesday blackouts would be painless if high-throughput hosts were allowed to buffer more than 100 tasks. This topic has been flogged plenty and there's been some good ideas suggested -- would be nice to pick one and get it done. I don't know about anyone else but I've had problems with CPU-sharing with some applications and have resorted to using the exclusive list to pause BOINC when needed; if there was a separation of the exclusive application list to allow GPU tasks to continue and only pause CPU tasks in this situation you might get more work done on some systems. FWIW I'm not personally concerned about the credit system; I'll admit that from the data I've analyzed to compare GPU throughput for different types of task it seems like there's some serious inconsistencies but this doesn't seem to matter to the science getting done. Never the less since there seems to be a lot of people upset about it this might be better for the public-relations of the project to find something to make people happy. I would volunteer code effort but frankly after I farted around with some of the SETI code I was extremely discouraged by how my patches were received. I know the projects are separate but I must admit a healthy amount of apprehension. |
Darrell Send message Joined: 14 Mar 03 Posts: 267 Credit: 1,418,681 RAC: 0 |
I'm not sure if it's a BOINC problem or a SETI@Home problem but the Tuesday blackouts would be painless if high-throughput hosts were allowed to buffer more than 100 tasks. This topic has been flogged plenty and there's been some good ideas suggested -- would be nice to pick one and get it done. This is a Seti limitation. Now it would be nice if Boinc could monitor the error status for each project and if it detected say three failed tasks in a row, suspend computation for that project and notify the user to check his system. I would volunteer code effort but frankly after I farted around with some of the SETI code I was extremely discouraged by how my patches were received. I know the projects are separate but I must admit a healthy amount of apprehension. Some projects like assistance, some don't. But for this thread, what we are looking for help with coding is the Boinc control program, Boinc Manager, Boinc server code, and the Boinc API library code. And not to be forgotten, Boinc Android code. ... and still I fear, and still I dare not laugh at the Mad Man! Queen - The Prophet's Song |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
Can we please try to avoid a historical recap of the entire development of BOINC in this thread? Factual references to earlier developments are of course welcome, but let's concentrate on moving forward from where we find ourselves now. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13722 Credit: 208,696,464 RAC: 304 |
Something Jason G has mentioned in the past- more flexible Computing Resource allocation. Presently we have CPU and GPU processing, and with the advent of USB Type C and Thunderbolt interfaces the possibility of "Other" Computing Resources. Also the present Manager is able to select the most efficient (fastest) application out of several candidates (although the mechanism needs work. On Beta on one run I ended up with the slowest application being selected due to the work that was allocated to each application while it was determining which was fastest - CUDA42 got almost all Arecibo mid range and shorty work , SoG almost all GBT WUs or Arecibo VLARs so CUDA42 looked like it was the fastest). On Seti, due to issues with older applications & older Nvidia hardware, Arecibo VLARs were blocked from going to all NVidia hardware. However with current highend hardware & the SoG application Arecibo VLARs can be run without issues. Under the present system when work is downloaded, it is allocated to either CPU, GPU, or Other (when it becomes available). Yet with Seti, GBT work is processed much faster on the CPU than much of the Arecibo work, Arecibo work is processed much faster on the GPU than GBT work. It would be best if the work was able to be downloaded & allocated dynamically to the Computing Resource most able to process it. It would also be good of there were project settings that allowed those that feel the need to allocate work to certain resources to be able to do so. eg systems with highend NVidia hardware capable of handling Arecibo VLARs without issue can opt to accept it for GPU processing, yet it remains blocked by default for all others. By default, the Manager will allocate work to the most capable Computing Resource. Given cache settings, server side limits, turn around times & WU deadlines etc, etc if a particular Computing Resource has reached it's cache limit, then new work downloaded will be allocated to the next most capable Computing Resource, down the line, till all such Computing Resources have reached their limits. As work is processed and returned, work is re-allocated to the most capable Computing Resource, and new work received from the servers allocated to the most capable, next most capable & so on till all Computing Resources have once again reached their set limits. Grant Darwin NT |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
... more flexible Computing Resource allocation.Something somebody said recently made me think again about the Xeon Phi. Those are now available in the UK (built to order) from at least one of the regular consumer mail order outlets, running Windows 10. So, usable, if not yet affordable. But I think we'd need another coprocessor option. |
Darrell Send message Joined: 14 Mar 03 Posts: 267 Credit: 1,418,681 RAC: 0 |
A couple of articles from January of this year: https://www.theinquirer.net/inquirer/news/3002448/pc-sales-down-in-2016-and-will-continue-declining-in-2017-say-analysts https://phys.org/news/2017-01-personal-sales-fall-year-row.html From these two articles we see that more people are using their smartphones now than they use their PCs. PCs buyers are businesses, gamers, and enthusiasts. Thus the number of systems that Boinc runs on is decreasing. Businesses won't take a chance on Boinc and gamers won't want Boinc interfering with their games. Enthusiasts are Boincers and miners. Businesses and miners won't use Boinc. Currently gamers have to specify each game executable in an exclusion list to keep Boinc from processing while they are gaming, but if this could be automated, maybe some of them would be more willing to run Boinc after they are done playing. Boinc development on smartphones has been woefully inadequate and this is the area that really needs the most attention. It is the one area that could provide for the most potential growth in use. ... and still I fear, and still I dare not laugh at the Mad Man! Queen - The Prophet's Song |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
Boinc development on smartphones has been woefully inadequate and this is the area that really needs the most attention. It is the one area that could provide for the most potential growth in use.I agree with your comment that recent smartphone development (well, development since launch) has been inadequate. Could you help us out with some numbers, please? Ratio of (number of smartphones times speed of a single smartphone) to (number of PCs times speed of a single PC)? In short, ratio of the total power of the (available to BOINC) smartphone community to the total power of the PC community? I suspect it's numbers like that that will influence the science project community which way to go. And I don't know which to press for. Edit - we probably need to factor in the "time available for crunching" figures too, for each platform. |
Darrell Send message Joined: 14 Mar 03 Posts: 267 Credit: 1,418,681 RAC: 0 |
From this site we have the statistics of the number of smartphones in use in the world: https://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/ Since the number for years 2016 -2020 are still estimates, I'll go with the 2015 number of 1.86 billion smartphones used. Now smartphones are like PCs in that they come in many flavors with different amounts of cores, memory, and battery power. My Smartphone is a Samsung Galaxy Grand Prime from Straighttalk. It has four cores and 8gb of memory. I let Boinc use two core for running two tasks at a time. This phone's battery is not one of the longer lasting one's and thus only allow computation while it is plugged in to the charger. Here is the most recent task it completed: https://setiathome.berkeley.edu/result.php?resultid=5892790260 Completed the task in a little over 21hrs and the phone is rated at a whopping 1.17 GFlops For my CPU from NFS: (Bionc only gives me Seti GPU tasks) https://escatter11.fullerton.edu/nfs/result.php?resultid=89825375 Task time and CPU rating of 3.39 GFLOPs, this is for my 9 year old AMD Phenom II X6 1100t processor. Seti GPU Guppie: (running two at a time) Completed task in 9.5 minutes at a rating of 180 GFlops From BoincStats: https://boincstats.com/en/stats/-1/project/detail/host In the last 60 days, the number of active Boinc hosts has gone from 1.2 million down to 877 thousand. This number probably includes some smartphones and tablets. The current Boinc Android client works on my smartphone, but does not work on my Polaroid Tablet. ... and still I fear, and still I dare not laugh at the Mad Man! Queen - The Prophet's Song |
rob smith Send message Joined: 7 Mar 03 Posts: 22160 Credit: 416,307,556 RAC: 380 |
A massive dis-entanglement of the code and improved in-line documentation wouldn't go amiss - far too many "real world" functions are sprayed all over the place which must (does) make maintenance a nightmare. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
tullio Send message Joined: 9 Apr 04 Posts: 8797 Credit: 2,930,782 RAC: 1 |
On the BOINC home page there is a link to an article by David Anderson on a new project sponsored by the National Science Foundation, UC Berkeley, Purdue University and theTexas Advanced Computing Center. Although I could not exactly grasp all its meaning, here is a quotation from "Nature Index" which summarizes it: "Anderson wants to connect BOINC to major supercomputer facilities in the United States, to reduce the lengthy wait researchers have to process their data. He is working to add the network to the Texas Advanced Computing Center as an additional resource for researchers". Tullio |
MarkJ Send message Joined: 17 Feb 08 Posts: 1139 Credit: 80,854,192 RAC: 5 |
As already mentioned I would separate the CPU work fetch/scheduling from the GPU work fetch/scheduling so they are independent of each other. Make credit standard across projects (maybe time based). Fix the resource share across projects so it's fairer, maybe also time based (separated into CPU and GPU). A cluster-aware client would be nice for those with farms so it could feed the compute nodes. It would use MPI to communicate with the science apps. On the server side to help projects get going maybe YouTube videos showing how to setup the virtual image and get jobs going, a checklist of things to define/configure. MarkJ BOINC blog |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13722 Credit: 208,696,464 RAC: 304 |
As already mentioned I would separate the CPU work fetch/scheduling from the GPU work fetch/scheduling so they are independent of each other. That's the problem at the moment, they are separate. Computing resources need to just be a pool, that the manager can allocate (and re-allocate) work to as required. A cluster-aware client would be nice for those with farms so it could feed the compute nodes. The above method of a computing resource pool would also allow (with a bit of work of course) a single Manager to allocate work to multiple machines without the need for having the Manager on each and every one. eg Install BOINC on a particular system, attach to a project & off you go. Got another computer available? Make it accessible on the network with the original machine, tell the Manager on the original system that the other computing resources (CPU, GPU, other) on the other physical computer are available for it's use. Probably only suitable for a few machines, with not too many cores/GPUs/other processors as network traffic would probably grow rather quickly (sending of WUs to process, receiving of completed WU results, and all the updates on WU progress, completion times etc). Grant Darwin NT |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
Hi all, First I'd like to thank those of you who took the time to post your grievances, feature requests and suggestions. We have quite a list already, but we'd love more. I can tell you that all of what makes it through to the list will be spoken about in detail between the group members and possibly back to you who asked about it. Just so you know, this work group isn't some low people who were aimlessly waiting around in the lobby of some university, instead as I hinted in some posts already, these people aren't the least in BOINC, they do have something to say. They are senior project scientists and administrators, and other significant contributors. Meet: - Christian Beer - Einstein@Home - David Wallom - Climateprediction.net - Eric Korpela - Seti@Home - Kevin Reed - IBM/World Community Grid - Laurence Field - CERN/LHC - Tristan Olive - Grid Republic - David Anderson - BOINC So yes, we're quite honored in that they asked us to gather around the table to discuss a lot of things about BOINC, and so should you. All the material Richard and I send in is talked about via email/a dedicated website. There's so much to talk about even, that some of it made it impossible for us to do all the Agenda points on the last weekly meetings. :) Weekly we talk via teleconference with each other, which is really nice as at least that way the others can hear the urgency in my voice on anything I have to say. And that's not all just serious talk, there's a lot of humor passing by as well. Weirdly enough, sometimes these guys sound and act like humans. :-D With that out of the way, please, continue. Give us a lot more to talk about, to decide upon, to vote on how we should change things. We're proud of you. We're BOINC, resistance is futile. :) Jord. Richard. |
Brent Norman Send message Joined: 1 Dec 99 Posts: 2786 Credit: 685,657,289 RAC: 835 |
More venues, with the advent of iPhones, Android, etc, etc, It would be better to have more venues available to be more selective for which devices get which apps. I'm not sure how other Projects handle this but ... a better/standard application scheduling system to ease which applications go to which devices. SETI has had many problems with this. A matrix if you will on the server side to select app1 can go to dev1, dev3, dev9, dev32 ... And also on the client side to mirror the matrix for what you want to receive, with unavailable options greyed out (or at least a warning if selected). And of course the cache size to allow for at least 12 hours to cover maintenance. This can be check by turn around time/consecutive valids. If a computer is returning tasks at a fast rate and reliable rate, give them more. This could be calculated something like the greater of 100 OR 100/TurnAroundTime. Slow computers would still get 100 (100/10d==100 tasks), and fast computers (100/0.2d=500 tasks). The 100 value could be modified so that everyone has the same cache size (in days). <EDIT> I heard mention of some projects wanting to turn to supercomputers. This type of caching could help home users crunch data only if they can return the work FAST. Coupled with aggressive task deadlines could get a lot of data through fast. To add, SETI has way too long for deadlines. Two weeks should be good, not 7. In todays wired world what computer doesn't have access for 2 weeks? If your on holidays with your laptop your not going to be crunching much anyways. </EDIT> <EDIT2>Adding an option to not accept tasks that will take longer than XX hours would be nice too</EDIT2> I think the others have covered most of the other things I have thought of. |
Dimly Lit Lightbulb 😀 Send message Joined: 30 Aug 08 Posts: 15399 Credit: 7,423,413 RAC: 1 |
Can I ask a serious question at this point - without giving away any clues by suggesting possible answers. I want to take a straw poll to get a feeling for what people are expecting. Time. Member of the People Encouraging Niceness In Society club. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.