For the betterment of BOINC

Message boards : Number crunching : For the betterment of BOINC
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1879878 - Posted: 23 Jul 2017, 7:31:06 UTC - in response to Message 1879727.  

Thanks for starting us off, Grant. I'd like as many answers as possible by the end of the week, please, and then I'll tot them up.
ID: 1879878 · Report as offensive
Profile tullio
Volunteer tester

Send message
Joined: 9 Apr 04
Posts: 8797
Credit: 2,930,782
RAC: 1
Italy
Message 1879896 - Posted: 23 Jul 2017, 11:43:31 UTC

As a BOINC user in six projects, there is one feature which I value most and that is the easiness of contact with the projects admin/developers, which was easy in SETI@home until the Breakthrough Listen project started and people moved over there for good reasons. There are projects where you can have an answer to a problem in a short time and those in which you never receive an answer. E.g. almost all my LHC@home tasks fail now, while I was an Alpha tester in Test4Theory@home. Nothing has changed in my HW/SW configuration, and I could get no help, so I abandoned the project. Same for Einstein@home on my Windows 10 PC after the Creators Update. They all fail, both on CPU and GPU, so I am running only SETI@ home and SETI Beta on that PC, my fastest.
Tullio
ID: 1879896 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1879898 - Posted: 23 Jul 2017, 12:29:39 UTC - in response to Message 1879896.  

Although it's a little off-topic for this discussion, I can tell you that the project administrator for the LHC Classic (Sixtrack) project is very pro-actively chasing down some infrastructure bugs right now, but hasn't yet reached a stage where he can post success on the message boards.

But the general point of good communication between projects (scientists and administrators alike) and their volunteers is well taken, and I would add that I would personally like to see better communication between projects and the core BOINC team, too.
ID: 1879898 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1879967 - Posted: 23 Jul 2017, 20:54:56 UTC - in response to Message 1879627.  

How about adding Seti Beta to "the list".
The add-project list is only populated by production projects, the alpha- and beta-projects aren't listed.
If a production project has an alpha- or beta-project of their own, they usually post about that on their front page, or in a stickied thread. If they do not, best ask that they do.
ID: 1879967 · Report as offensive
Profile Darrell
Volunteer tester
Avatar

Send message
Joined: 14 Mar 03
Posts: 267
Credit: 1,418,681
RAC: 0
United States
Message 1879991 - Posted: 23 Jul 2017, 23:17:23 UTC - in response to Message 1879719.  

Can I ask a serious question at this point - without giving away any clues by suggesting possible answers. I want to take a straw poll to get a feeling for what people are expecting.

Resource allocation. When you assign a resource share to a project, what would you expect to be measured to honor those shares?

[Suppose you give project A twice as much resource share as project B. Twice as much of what?]


From running Boinc with caches set to zero (i.e. getting one task at a time) if a project has both CPU and GPU tasks, and you have selected to use both, the current scheduler will only give you GPU tasks when attached to multiple projects. With the current GPU scheduler when attached to multiple projects that have GPU tasks and you set some of the projects to say run two tasks at a time on the GPU, the scheduler does not give you two tasks from the same project, but one from project A and one from project B.

For resource allocation I would like to Boinc to first look at the number of cores/threads. With my six core processor, if I set Boinc to use 5 cores (84% of CPU) this leaves one core free to run the system with little screen lag. For each GPU task on the GPU I want Boinc to give it one core to feed each task. So if I have one running GPU task, the Boinc should run 4 CPU tasks based upon project shares. If I have two GPU tasks running, then Boinc should run three CPU tasks. Boinc also needs to take in account for CPU tasks the number of cores need by multi-threaded apps and the number of cores to be used by multi-threaded apps should be easily settable.

Next resource allocation to be looked at should be time. PrimeGrid is the project of concern here. It has short GPU tasks ( < 1hr ), medium GPU tasks ( 1 -2 hrs), and long GPU tasks (24 hours and greater). So if a PrimeGrid long task runs, the Boinc needs to back-off work requests from the project, to allow the other projects to get their share of time. Ideally, it would be nice to set the PrimeGrid scheduler to something like give 4 short tasks or 2 medium tasks for every long task run, and balance that out with the sharing of time with other projects.
... and still I fear, and still I dare not laugh at the Mad Man!

Queen - The Prophet's Song
ID: 1879991 · Report as offensive
Profile Shaggie76
Avatar

Send message
Joined: 9 Oct 09
Posts: 282
Credit: 271,858,118
RAC: 196
Canada
Message 1880007 - Posted: 24 Jul 2017, 1:33:52 UTC

I'm not sure if it's a BOINC problem or a SETI@Home problem but the Tuesday blackouts would be painless if high-throughput hosts were allowed to buffer more than 100 tasks. This topic has been flogged plenty and there's been some good ideas suggested -- would be nice to pick one and get it done.

I don't know about anyone else but I've had problems with CPU-sharing with some applications and have resorted to using the exclusive list to pause BOINC when needed; if there was a separation of the exclusive application list to allow GPU tasks to continue and only pause CPU tasks in this situation you might get more work done on some systems.

FWIW I'm not personally concerned about the credit system; I'll admit that from the data I've analyzed to compare GPU throughput for different types of task it seems like there's some serious inconsistencies but this doesn't seem to matter to the science getting done. Never the less since there seems to be a lot of people upset about it this might be better for the public-relations of the project to find something to make people happy.

I would volunteer code effort but frankly after I farted around with some of the SETI code I was extremely discouraged by how my patches were received. I know the projects are separate but I must admit a healthy amount of apprehension.
ID: 1880007 · Report as offensive
Profile Darrell
Volunteer tester
Avatar

Send message
Joined: 14 Mar 03
Posts: 267
Credit: 1,418,681
RAC: 0
United States
Message 1880026 - Posted: 24 Jul 2017, 4:40:18 UTC - in response to Message 1880007.  
Last modified: 24 Jul 2017, 4:56:00 UTC

I'm not sure if it's a BOINC problem or a SETI@Home problem but the Tuesday blackouts would be painless if high-throughput hosts were allowed to buffer more than 100 tasks. This topic has been flogged plenty and there's been some good ideas suggested -- would be nice to pick one and get it done.


This is a Seti limitation.

Now it would be nice if Boinc could monitor the error status for each project and if it detected say three failed tasks in a row, suspend computation for that project and notify the user to check his system.


I would volunteer code effort but frankly after I farted around with some of the SETI code I was extremely discouraged by how my patches were received. I know the projects are separate but I must admit a healthy amount of apprehension.

Some projects like assistance, some don't. But for this thread, what we are looking for help with coding is the Boinc control program, Boinc Manager, Boinc server code, and the Boinc API library code. And not to be forgotten, Boinc Android code.
... and still I fear, and still I dare not laugh at the Mad Man!

Queen - The Prophet's Song
ID: 1880026 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1880039 - Posted: 24 Jul 2017, 9:47:22 UTC

Can we please try to avoid a historical recap of the entire development of BOINC in this thread? Factual references to earlier developments are of course welcome, but let's concentrate on moving forward from where we find ourselves now.
ID: 1880039 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13722
Credit: 208,696,464
RAC: 304
Australia
Message 1880043 - Posted: 24 Jul 2017, 11:03:58 UTC
Last modified: 24 Jul 2017, 11:04:25 UTC

Something Jason G has mentioned in the past- more flexible Computing Resource allocation.

Presently we have CPU and GPU processing, and with the advent of USB Type C and Thunderbolt interfaces the possibility of "Other" Computing Resources.
Also the present Manager is able to select the most efficient (fastest) application out of several candidates (although the mechanism needs work. On Beta on one run I ended up with the slowest application being selected due to the work that was allocated to each application while it was determining which was fastest - CUDA42 got almost all Arecibo mid range and shorty work , SoG almost all GBT WUs or Arecibo VLARs so CUDA42 looked like it was the fastest).
On Seti, due to issues with older applications & older Nvidia hardware, Arecibo VLARs were blocked from going to all NVidia hardware. However with current highend hardware & the SoG application Arecibo VLARs can be run without issues.


Under the present system when work is downloaded, it is allocated to either CPU, GPU, or Other (when it becomes available). Yet with Seti, GBT work is processed much faster on the CPU than much of the Arecibo work, Arecibo work is processed much faster on the GPU than GBT work.
It would be best if the work was able to be downloaded & allocated dynamically to the Computing Resource most able to process it. It would also be good of there were project settings that allowed those that feel the need to allocate work to certain resources to be able to do so. eg systems with highend NVidia hardware capable of handling Arecibo VLARs without issue can opt to accept it for GPU processing, yet it remains blocked by default for all others.

By default, the Manager will allocate work to the most capable Computing Resource. Given cache settings, server side limits, turn around times & WU deadlines etc, etc if a particular Computing Resource has reached it's cache limit, then new work downloaded will be allocated to the next most capable Computing Resource, down the line, till all such Computing Resources have reached their limits. As work is processed and returned, work is re-allocated to the most capable Computing Resource, and new work received from the servers allocated to the most capable, next most capable & so on till all Computing Resources have once again reached their set limits.
Grant
Darwin NT
ID: 1880043 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1880046 - Posted: 24 Jul 2017, 11:34:41 UTC - in response to Message 1880043.  

... more flexible Computing Resource allocation.
Something somebody said recently made me think again about the Xeon Phi. Those are now available in the UK (built to order) from at least one of the regular consumer mail order outlets, running Windows 10. So, usable, if not yet affordable. But I think we'd need another coprocessor option.
ID: 1880046 · Report as offensive
Profile Darrell
Volunteer tester
Avatar

Send message
Joined: 14 Mar 03
Posts: 267
Credit: 1,418,681
RAC: 0
United States
Message 1880129 - Posted: 24 Jul 2017, 20:40:26 UTC

A couple of articles from January of this year:

https://www.theinquirer.net/inquirer/news/3002448/pc-sales-down-in-2016-and-will-continue-declining-in-2017-say-analysts

https://phys.org/news/2017-01-personal-sales-fall-year-row.html

From these two articles we see that more people are using their smartphones now than they use their PCs. PCs buyers are businesses, gamers, and enthusiasts. Thus the number of systems that Boinc runs on is decreasing. Businesses won't take a chance on Boinc and gamers won't want Boinc interfering with their games. Enthusiasts are Boincers and miners.

Businesses and miners won't use Boinc. Currently gamers have to specify each game executable in an exclusion list to keep Boinc from processing while they are gaming, but if this could be automated, maybe some of them would be more willing to run Boinc after they are done playing.

Boinc development on smartphones has been woefully inadequate and this is the area that really needs the most attention. It is the one area that could provide for the most potential growth in use.
... and still I fear, and still I dare not laugh at the Mad Man!

Queen - The Prophet's Song
ID: 1880129 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1880140 - Posted: 24 Jul 2017, 21:14:41 UTC - in response to Message 1880129.  
Last modified: 24 Jul 2017, 21:21:04 UTC

Boinc development on smartphones has been woefully inadequate and this is the area that really needs the most attention. It is the one area that could provide for the most potential growth in use.
I agree with your comment that recent smartphone development (well, development since launch) has been inadequate.

Could you help us out with some numbers, please? Ratio of (number of smartphones times speed of a single smartphone) to (number of PCs times speed of a single PC)? In short, ratio of the total power of the (available to BOINC) smartphone community to the total power of the PC community?

I suspect it's numbers like that that will influence the science project community which way to go. And I don't know which to press for.

Edit - we probably need to factor in the "time available for crunching" figures too, for each platform.
ID: 1880140 · Report as offensive
Profile Darrell
Volunteer tester
Avatar

Send message
Joined: 14 Mar 03
Posts: 267
Credit: 1,418,681
RAC: 0
United States
Message 1880158 - Posted: 24 Jul 2017, 22:37:48 UTC - in response to Message 1880140.  
Last modified: 24 Jul 2017, 22:39:34 UTC

From this site we have the statistics of the number of smartphones in use in the world:

https://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/

Since the number for years 2016 -2020 are still estimates, I'll go with the 2015 number of 1.86 billion smartphones used. Now smartphones are like PCs in that they come in many flavors with different amounts of cores, memory, and battery power. My Smartphone is a Samsung Galaxy Grand Prime from Straighttalk. It has four cores and 8gb of memory. I let Boinc use two core for running two tasks at a time. This phone's battery is not one of the longer lasting one's and thus only allow computation while it is plugged in to the charger.

Here is the most recent task it completed:

https://setiathome.berkeley.edu/result.php?resultid=5892790260

Completed the task in a little over 21hrs and the phone is rated at a whopping 1.17 GFlops

For my CPU from NFS: (Bionc only gives me Seti GPU tasks)

https://escatter11.fullerton.edu/nfs/result.php?resultid=89825375

Task time and CPU rating of 3.39 GFLOPs, this is for my 9 year old AMD Phenom II X6 1100t processor.

Seti GPU Guppie:

(running two at a time) Completed task in 9.5 minutes at a rating of 180 GFlops

From BoincStats:

https://boincstats.com/en/stats/-1/project/detail/host

In the last 60 days, the number of active Boinc hosts has gone from 1.2 million down to 877 thousand. This number probably includes some smartphones and tablets.

The current Boinc Android client works on my smartphone, but does not work on my Polaroid Tablet.
... and still I fear, and still I dare not laugh at the Mad Man!

Queen - The Prophet's Song
ID: 1880158 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22160
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1880239 - Posted: 25 Jul 2017, 5:02:46 UTC

A massive dis-entanglement of the code and improved in-line documentation wouldn't go amiss - far too many "real world" functions are sprayed all over the place which must (does) make maintenance a nightmare.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1880239 · Report as offensive
Profile tullio
Volunteer tester

Send message
Joined: 9 Apr 04
Posts: 8797
Credit: 2,930,782
RAC: 1
Italy
Message 1880250 - Posted: 25 Jul 2017, 7:02:23 UTC

On the BOINC home page there is a link to an article by David Anderson on a new project sponsored by the National Science Foundation, UC Berkeley, Purdue University and theTexas Advanced Computing Center. Although I could not exactly grasp all its meaning, here is a quotation from "Nature Index" which summarizes it:
"Anderson wants to connect BOINC to major supercomputer facilities in the United States, to reduce the lengthy wait researchers have to process their data. He is working to add the network to the Texas Advanced Computing Center as an additional resource for researchers".
Tullio
ID: 1880250 · Report as offensive
MarkJ Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 08
Posts: 1139
Credit: 80,854,192
RAC: 5
Australia
Message 1880256 - Posted: 25 Jul 2017, 10:07:34 UTC

As already mentioned I would separate the CPU work fetch/scheduling from the GPU work fetch/scheduling so they are independent of each other.

Make credit standard across projects (maybe time based).

Fix the resource share across projects so it's fairer, maybe also time based (separated into CPU and GPU).

A cluster-aware client would be nice for those with farms so it could feed the compute nodes. It would use MPI to communicate with the science apps.

On the server side to help projects get going maybe YouTube videos showing how to setup the virtual image and get jobs going, a checklist of things to define/configure.

MarkJ
BOINC blog
ID: 1880256 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13722
Credit: 208,696,464
RAC: 304
Australia
Message 1880258 - Posted: 25 Jul 2017, 10:47:24 UTC - in response to Message 1880256.  

As already mentioned I would separate the CPU work fetch/scheduling from the GPU work fetch/scheduling so they are independent of each other.

That's the problem at the moment, they are separate.
Computing resources need to just be a pool, that the manager can allocate (and re-allocate) work to as required.

A cluster-aware client would be nice for those with farms so it could feed the compute nodes.

The above method of a computing resource pool would also allow (with a bit of work of course) a single Manager to allocate work to multiple machines without the need for having the Manager on each and every one.

eg Install BOINC on a particular system, attach to a project & off you go. Got another computer available? Make it accessible on the network with the original machine, tell the Manager on the original system that the other computing resources (CPU, GPU, other) on the other physical computer are available for it's use. Probably only suitable for a few machines, with not too many cores/GPUs/other processors as network traffic would probably grow rather quickly (sending of WUs to process, receiving of completed WU results, and all the updates on WU progress, completion times etc).
Grant
Darwin NT
ID: 1880258 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1880375 - Posted: 26 Jul 2017, 11:55:46 UTC

Hi all,

First I'd like to thank those of you who took the time to post your grievances, feature requests and suggestions. We have quite a list already, but we'd love more. I can tell you that all of what makes it through to the list will be spoken about in detail between the group members and possibly back to you who asked about it.

Just so you know, this work group isn't some low people who were aimlessly waiting around in the lobby of some university, instead as I hinted in some posts already, these people aren't the least in BOINC, they do have something to say. They are senior project scientists and administrators, and other significant contributors.

Meet:
- Christian Beer - Einstein@Home
- David Wallom - Climateprediction.net
- Eric Korpela - Seti@Home
- Kevin Reed - IBM/World Community Grid
- Laurence Field - CERN/LHC
- Tristan Olive - Grid Republic
- David Anderson - BOINC

So yes, we're quite honored in that they asked us to gather around the table to discuss a lot of things about BOINC, and so should you. All the material Richard and I send in is talked about via email/a dedicated website. There's so much to talk about even, that some of it made it impossible for us to do all the Agenda points on the last weekly meetings. :)

Weekly we talk via teleconference with each other, which is really nice as at least that way the others can hear the urgency in my voice on anything I have to say. And that's not all just serious talk, there's a lot of humor passing by as well. Weirdly enough, sometimes these guys sound and act like humans. :-D

With that out of the way, please, continue. Give us a lot more to talk about, to decide upon, to vote on how we should change things. We're proud of you. We're BOINC, resistance is futile. :)

Jord.
Richard.
ID: 1880375 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1880383 - Posted: 26 Jul 2017, 12:35:46 UTC
Last modified: 26 Jul 2017, 12:55:18 UTC

More venues, with the advent of iPhones, Android, etc, etc, It would be better to have more venues available to be more selective for which devices get which apps.

I'm not sure how other Projects handle this but ... a better/standard application scheduling system to ease which applications go to which devices. SETI has had many problems with this. A matrix if you will on the server side to select app1 can go to dev1, dev3, dev9, dev32 ... And also on the client side to mirror the matrix for what you want to receive, with unavailable options greyed out (or at least a warning if selected).

And of course the cache size to allow for at least 12 hours to cover maintenance. This can be check by turn around time/consecutive valids. If a computer is returning tasks at a fast rate and reliable rate, give them more. This could be calculated something like the greater of 100 OR 100/TurnAroundTime. Slow computers would still get 100 (100/10d==100 tasks), and fast computers (100/0.2d=500 tasks). The 100 value could be modified so that everyone has the same cache size (in days).

<EDIT> I heard mention of some projects wanting to turn to supercomputers. This type of caching could help home users crunch data only if they can return the work FAST. Coupled with aggressive task deadlines could get a lot of data through fast.

To add, SETI has way too long for deadlines. Two weeks should be good, not 7. In todays wired world what computer doesn't have access for 2 weeks? If your on holidays with your laptop your not going to be crunching much anyways. </EDIT>

<EDIT2>Adding an option to not accept tasks that will take longer than XX hours would be nice too</EDIT2>

I think the others have covered most of the other things I have thought of.
ID: 1880383 · Report as offensive
Profile Dimly Lit Lightbulb 😀
Volunteer tester
Avatar

Send message
Joined: 30 Aug 08
Posts: 15399
Credit: 7,423,413
RAC: 1
United Kingdom
Message 1880397 - Posted: 26 Jul 2017, 15:05:37 UTC - in response to Message 1879719.  

Can I ask a serious question at this point - without giving away any clues by suggesting possible answers. I want to take a straw poll to get a feeling for what people are expecting.

Resource allocation. When you assign a resource share to a project, what would you expect to be measured to honour those shares?

[Suppose you give project A twice as much resource share as project B. Twice as much of what?]

Time.

Member of the People Encouraging Niceness In Society club.

ID: 1880397 · Report as offensive
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : For the betterment of BOINC


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.