BOINC needs a overhaul

Message boards : Number crunching : BOINC needs a overhaul
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 974944 - Posted: 1 Mar 2010, 4:10:52 UTC - in response to Message 974929.  

Sigh.

When the developers state exactly why something was done, there are a few users that refuse to listen. Most of the decisions are based on problems encountered when the code was some other way.

1) The project wide back off was made to speed recovery after an outage. Everyone hitting the servers slows everything down. This change has lowered the recovery time from 2 days to 1 day. In other words, it is better for everyone on average.

2) The rule of preventing downloads when there are > 2 * ncpus results waiting to upload was put in place for a couple of projects where the upload time exceeded the compute time. Not having this in place would have infinite work accumulate on your computer waiting to upload. In other words, there has to be SOME limit. Admittedly 2* is a bit arbitrary. You might make more progress with an argument for a different limit.

From a Seti perpective.
The ordinary user installs and forgets, so one assumes that the cache of tasks is ~0.25days, 6 hours. On a modern cpu the shortest tasks, VHAR's, using the stock application, can be done in under an hour.
On Tuesdays the project can be down for the full length of a working day, 8+ hrs. The normal recovery period after the maintenance can be up to 5 or 6 hours.
So if the host is on all of this period and has not been able to upload all of the tasks completed, it could have been on for 14 hrs with only 6 hrs of work.

This I believe is not how BOINC and the projects were meant to work, i.e. use all the spare cpu cycles when the computer is on.

Therefore when setting limits or default settings, the designers need to work out realistic settings, using scenario's from all projects, not just appease the one's that request a change. And if the change requested doesn't fit the majority of projects it must be rejected.

I would suggest that the 2*cpu's rule be rejected on these grounds.

Sorry, you have not made your case.

1) Multi projects are encouraged. So if you cannot get work from one, you could get work from another.

2) There are real cases where a limit is required as explained.

If you can come up with a better limit, propose it and how to calculate it.


BOINC WIKI
ID: 974944 · Report as offensive
Profile RFP
Avatar

Send message
Joined: 21 Jan 10
Posts: 44
Credit: 29,197
RAC: 0
United States
Message 974950 - Posted: 1 Mar 2010, 4:29:14 UTC

Last time I checked the SEI software development process it starts with a set of requirements usualy determined by the org funding/requesting the software. These requirements are key because from them you develope your testing criteria. Software dev for the DOE and DOD was to be done with the SEI modle, but the last time I was involved with that it was a farse.

If you are searching for Intelegent life the rest of the universe is still trying to determine if we rate kindergarden yet.
ID: 974950 · Report as offensive
Profile dnolan
Avatar

Send message
Joined: 30 Aug 01
Posts: 1228
Credit: 47,779,411
RAC: 32
United States
Message 974954 - Posted: 1 Mar 2010, 4:43:08 UTC - in response to Message 974943.  


Hi dnolan...
Well, by my reckoning, out of the 22 or so people who have expressed their opinion about 15 would like to see an overhaul... 70% or so.
That's a very small pool of users though.

- Luke.



Well, aside from the fact that this is an astonishingly small number of users...
I personally don't think there's any major problem with Boinc. As per the original post, I don't see it being bloated, it doesn't use major processing time on my systems. And most of the problems I've seen so far have been specific to science applications, not Boinc. I've missed deadlines in Yoyo, but the applications are not predictable. Muon can run aa little as an hour to over 20 hours without any predictable progression, and Evolution runs as much as 130% or more before finishing (I've see a WU go to over 200%). I ran out of work for Seti over the last problem period, but that really doesn't have anything to do with Boinc, as far as I can tell. And I had work from other projects. Yes, Boinc can back off for long periods when there are project issues, and trying to run 2 different GPU projects on the same system results in odd usage patterns and doesn't seem to respect resource share. As long as I don't miss any deadlines, I don't care if reporting/uploading has to wait for a day or two or five, and GPUs are a recent thing as far as Boinc is concerned. There may need to be a re-write to deal correctly with GPU usage and resource share, but even if there isn't, it mostly works OK now.

The thing I would agree on is that the email list is a bad way for the average user to take part in the feedback process. Personally, I already get way too much email as it is. I don't see the point of having a Boinc forum if it's not going to be used, either. But I'm not a Boinc developer, so I guess they're going to use whatever they like.

-Dave
ID: 974954 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 974958 - Posted: 1 Mar 2010, 5:30:50 UTC - in response to Message 974835.  
Last modified: 1 Mar 2010, 5:36:50 UTC

I count 14 votes...
Grant (SSSF) - Yes


I think Grant (SSSF) actually was a NO vote. If I interpret his reply correctly, I believe he was being sarcastic.

Yep.
Ned Ludd is much more eloquent than myself, and as usuall covered it very well.


By no means is BOINC perfect, but it's no where near as bad as is implied in this thread. No piece of software no matter how well it is designed, written & implemented will make all of it's users happy. There will always be those that don't like something about it and changing things to suit these few users will end up resulting in something that pleases noone.
Grant
Darwin NT
ID: 974958 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 974961 - Posted: 1 Mar 2010, 5:40:43 UTC - in response to Message 974943.  

Well, by my reckoning, out of the 22 or so people who have expressed their opinion about 15 would like to see an overhaul... 70% or so.

The whole voting process itself is fundamentally flawed.

Winterknight wrote:

"Well I for one think having to keep a computer on waiting for uploads to finish before it can get any downloads is STUPID.


... and he's voting for a ground-up rewrite because surely this broken feature would not be included.

Hiamps says:
Quick as possible for WHO? The backoff made mine go into 24 hr wait even after things were going again. Was not good for the project either if machines can't start working again. Plus boinc can't handle large loads which are growing with every Nvidia card added.

... and he'd vote for the re-write because it fixes this problem.

... as would msattler because it prevents him from getting his credit fix.

Someone else says "BOINC doesn't cache enough, by default, to work through the weekly outage -- that can't be right!" and guess what, if you're harnessing a waste product (idle clock cycles on computers that would be turned on anyway) then you're wasting a waste product, and it is not the end of the world.

Just for fun, let's define "code bloat" -- this is the list of all "features" that in the sole opinion of the one individual, is useless or actually does damage.

Get a complete list of "code bloat" from everyone here and the entire BOINC code base is gone.

I won't catalog the rest because they're all pet-peeves, and the "voters" are certain that the brain damaged behaviour they see would surely not be intentionally reinstated as part of a ground-up re-write.

The idea of voting is flawed because the people doing the voting can agree by voting that they want "better" without exploring the fact that each voter has a different (perhaps radically different) idea of better.

Fifteen votes, and fifteen different ideas of what an improved "overhauled" BOINC would do.

(Hell, I'll vote "yes" for better, just be aware of the fact that by "better" I'm pretty much rejecting every stupid idea suggested in this forum -- by which I mean nearly all of them.)

Here is an idea on how to "fix" BOINC.

Take the whole credit system and hide it. Call it canonical credit, and don't publish it anywhere. Ever.

Take the canonical credit granted over the past 30 days for all projects, and figure out how much credit was granted per second for each project over the prior 2,592,000 seconds, and grant that to each project for each CPU second BOINC uses. If all projects are out of work, grant credit anyway.

If credit is over-granted compared to canonical credit, the credit rate will adjust downward some.

Now, no one cares about uploads, and there are no "pending" credits. Credit simply rises like clockwork.

Hide all work units. Remove all of the client-side logs. Encrypt all of the data files (work units, client state, etc.).

Replace the "update project" button with a code segment reports "your update is complete" and nothing else. Maybe a delay so it is believable, or give some bonus cobblestones -- they're meaningless anyway.

This way, everyone sees things moving in a nice smooth positive direction, and nobody can see what happens.

There. Fixed.

... and the developers are free to work through the actual issues without being molested by crazy people who buy computers and throw good money to the power companies just to amass valueless credits.

As far as I can tell, just about everything else in the forums (with the possible exception of the raccoon and TLTPW threads in the Cafe) is pure whining from people who can't be bothered to ask "why" before they start graphically demonstrating their ignorance.
ID: 974961 · Report as offensive
James

Send message
Joined: 1 Nov 09
Posts: 1
Credit: 226,129
RAC: 0
United States
Message 974969 - Posted: 1 Mar 2010, 6:13:25 UTC

Ok...some thoughts from an average user. (If I can be so bold as to call myself average!!)

There are many of us that just like the feeling of being able to help. I have 6-8 hours a day that I can give a little to the big cause. My little 2 core laptop will take years to crunch what some of you do in a week. But keep in mind the amount of work that the average users combined contribute.

That said the BOINC software seems to do the job for me. The problems that we have seen of late have been bothersome to even us average users, but things are getting back on track. I even chose to pick up some work from another project to fill some time.

It may be that I am still new to all of this, but I am still excited by the science being done. The credits are fun to watch and sometimes the boards are fun to read....but is it not about the science/fun?

I understand that my needs are quite small compaired to most of the posters here. I will never have a farm of machines working 24/7. I will never have to worry about cooling or running more electric circuits. So just maybe I am not in a position to see the big picture as some of you see it. I can accept that.

For me the wheel is working...maybe adding a little chrome would be nice. I don't need a new wheel yet. Just keep it running for me.
ID: 974969 · Report as offensive
Profile RFP
Avatar

Send message
Joined: 21 Jan 10
Posts: 44
Credit: 29,197
RAC: 0
United States
Message 974976 - Posted: 1 Mar 2010, 6:42:49 UTC
Last modified: 1 Mar 2010, 6:44:57 UTC

I am haveing a problem with all of this. I notice a large amount of posts with tester associated with the name. If you are a tester what requirements are you testing to? How do you know what features are new and how are they to function and what is the interaction with existing software
I have done a bit of SD and I knopw when the SWAG method is being used and GIGO is always the result.
ID: 974976 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19062
Credit: 40,757,560
RAC: 67
United Kingdom
Message 974980 - Posted: 1 Mar 2010, 7:03:20 UTC - in response to Message 974944.  

Sigh.

When the developers state exactly why something was done, there are a few users that refuse to listen. Most of the decisions are based on problems encountered when the code was some other way.

1) The project wide back off was made to speed recovery after an outage. Everyone hitting the servers slows everything down. This change has lowered the recovery time from 2 days to 1 day. In other words, it is better for everyone on average.

2) The rule of preventing downloads when there are > 2 * ncpus results waiting to upload was put in place for a couple of projects where the upload time exceeded the compute time. Not having this in place would have infinite work accumulate on your computer waiting to upload. In other words, there has to be SOME limit. Admittedly 2* is a bit arbitrary. You might make more progress with an argument for a different limit.

From a Seti perpective.
The ordinary user installs and forgets, so one assumes that the cache of tasks is ~0.25days, 6 hours. On a modern cpu the shortest tasks, VHAR's, using the stock application, can be done in under an hour.
On Tuesdays the project can be down for the full length of a working day, 8+ hrs. The normal recovery period after the maintenance can be up to 5 or 6 hours.
So if the host is on all of this period and has not been able to upload all of the tasks completed, it could have been on for 14 hrs with only 6 hrs of work.

This I believe is not how BOINC and the projects were meant to work, i.e. use all the spare cpu cycles when the computer is on.

Therefore when setting limits or default settings, the designers need to work out realistic settings, using scenario's from all projects, not just appease the one's that request a change. And if the change requested doesn't fit the majority of projects it must be rejected.

I would suggest that the 2*cpu's rule be rejected on these grounds.

Sorry, you have not made your case.

1) Multi projects are encouraged. So if you cannot get work from one, you could get work from another.

2) There are real cases where a limit is required as explained.

If you can come up with a better limit, propose it and how to calculate it.

There is a case for users to participate in more than one project, but it can never be forced and if a user wants to do only one project then as I see it this rule makes BOINC broken. Find a different solution that doesn't affect projects that don't need it.
Now as I am only a user for the near future at least. I haven't the time or the desire to look into why this limit is needed by, as I understand it, only one project.

Also from a personal observation, the way resource share is ignored when downloading. It forces the secondary project to download much too many tasks as soon as the primary project has no work or cannot be contacted, and therefore the idea of BOINC keeping to the resource share is bunkum. My records for the last 12 months indicate, on credits earned, that Einstein with a resource share of 12.5% is actually running at about 17%, even though Einstein is on default apps and Seti running optimised apps.
ID: 974980 · Report as offensive
Luke
Volunteer developer
Avatar

Send message
Joined: 31 Dec 06
Posts: 2546
Credit: 817,560
RAC: 0
New Zealand
Message 974981 - Posted: 1 Mar 2010, 7:12:00 UTC - in response to Message 974961.  
Last modified: 1 Mar 2010, 7:14:00 UTC


Hiamps says:
Quick as possible for WHO? The backoff made mine go into 24 hr wait even after things were going again. Was not good for the project either if machines can't start working again. Plus boinc can't handle large loads which are growing with every Nvidia card added.

... and he'd vote for the re-write because it fixes this problem.

... as would msattler because it prevents him from getting his credit fix.

Someone else says "BOINC doesn't cache enough, by default, to work through the weekly outage -- that can't be right!" and guess what, if you're harnessing a waste product (idle clock cycles on computers that would be turned on anyway) then you're wasting a waste product, and it is not the end of the world.

Just for fun, let's define "code bloat" -- this is the list of all "features" that in the sole opinion of the one individual, is useless or actually does damage.

Get a complete list of "code bloat" from everyone here and the entire BOINC code base is gone.

I won't catalog the rest because they're all pet-peeves, and the "voters" are certain that the brain damaged behaviour they see would surely not be intentionally reinstated as part of a ground-up re-write.

The idea of voting is flawed because the people doing the voting can agree by voting that they want "better" without exploring the fact that each voter has a different (perhaps radically different) idea of better.

Fifteen votes, and fifteen different ideas of what an improved "overhauled" BOINC would do.

(Hell, I'll vote "yes" for better, just be aware of the fact that by "better" I'm pretty much rejecting every stupid idea suggested in this forum -- by which I mean nearly all of them.)

Here is an idea on how to "fix" BOINC.

Take the whole credit system and hide it. Call it canonical credit, and don't publish it anywhere. Ever.

Take the canonical credit granted over the past 30 days for all projects, and figure out how much credit was granted per second for each project over the prior 2,592,000 seconds, and grant that to each project for each CPU second BOINC uses. If all projects are out of work, grant credit anyway.

If credit is over-granted compared to canonical credit, the credit rate will adjust downward some.

Now, no one cares about uploads, and there are no "pending" credits. Credit simply rises like clockwork.

Hide all work units. Remove all of the client-side logs. Encrypt all of the data files (work units, client state, etc.).

Replace the "update project" button with a code segment reports "your update is complete" and nothing else. Maybe a delay so it is believable, or give some bonus cobblestones -- they're meaningless anyway.

This way, everyone sees things moving in a nice smooth positive direction, and nobody can see what happens.

There. Fixed.

... and the developers are free to work through the actual issues without being molested by crazy people who buy computers and throw good money to the power companies just to amass valueless credits.

As far as I can tell, just about everything else in the forums (with the possible exception of the raccoon and TLTPW threads in the Cafe) is pure whining from people who can't be bothered to ask "why" before they start graphically demonstrating their ignorance.


This isn't an argument about credits. Start another thread if you wish to rant about that.

When I say this, I speak for Mark also, Frankly, I don't give a **** about the credits!
When the project came back up after the upload problems, I reported over 3000 credits worth of work that was marked as invalid because it was returned late. I was pissed off that the system didn't accept the science yes, but not for the credits.
I like the fact that credits show how much work you have done, but if I don't receive them, I don't really care. And yes, it is an achievement to pass a milestone, but I'm not going to go round arbitrarily aborting tasks because some give more!

Also, you whine about people whining? Where is the rational argument in that???
- Luke.
ID: 974981 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 974982 - Posted: 1 Mar 2010, 7:14:33 UTC - in response to Message 974930.  

Sometimes I close BOINC down, next time I open it it's stuck in suspend mode and I can't get it out

And you reported this where, exactly? On what versions have you seen it? When reporting it, you posted your client_state.xml entries about this status of BOINC alongside any other information you could give?

What entries? Oh those at the bottom of client_state.xml saying:
<user_run_request>X</user_run_request>
<user_gpu_request>X</user_gpu_request>
<user_network_request>X</user_network_request>
Where X=1 is Always available, X=2 is Based on preferences and X=3 is Suspended.

it gets blocked by UAC in Vista

That's more a permissions/UAC thing in Vista (not the most stable of Windows versions, bar apparently ME) than it's a BOINC thing. There have been quite a lot of reports of UAC doing this to the most mundane of programs, which is why Microsoft fixed UAC for Windows 7 and it now does things better, more civilized.

You can probably get rid of the problem completely by not installing the BOINC directory in the default path of C:\Program Files. By now you ought to know that your OS will not allow any program to write to that directory. That's hardly the program its fault, rather than the overly secure mindset the guys at Microsoft had when they made this contraption. Install BOINC to any other non-restricted directory and you probably never see the problem again. But then, it's Vista.
it can't handle in excess of 4000 tasks well

It can, as long as you don't expect BOINC Manager (the GUI) to update all those 4,000 tasks in real-time (every second). So either you keep the default setting of showing Active tasks only (those that run, are waiting to run or are suspended, the rest aren't active so not showing), instead of demanding that BM shows all the tasks with the Show All tasks option... or you run without BOINC Manager. That is easily done these days even on a non-service installation, just stop BOINC Manager and leave the client and science applications running (option available from the exit dialog).
clicking the icon in W7 seems to open another instance of the program even if one is already running.

It's quite normal for BOINC Manager to open multiple times. This is done so you can run a BM for each remote system you have. BOINC.exe will normally only run once, and I say normally as there is a line-switch for testing to run multiple clients at once.

BOINC has always done this opening of a BM each time you clicked the icon, it's not something newfangled for Windows 7 only.

Are they quickly forgotten because the documentation is poorly done? ;) I can't remember the post here, but that is what someone said.

The official BOINC Wiki is open to all. Register and start adding. Or do you again expect someone else to do all that work for you? What do you do when it's not written in a way or style you approve of? What if it's missing out on things you know the answer to? Again wait for yet another person to come along and add it?
ID: 974982 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 974985 - Posted: 1 Mar 2010, 7:29:02 UTC

I would vote against a complete rewrite/overhaul/whatever you want to call it.

I think bugs should be fixed before worrying about adding new features. I test software for a living 8-5 M-F for the past 11 years. I still don't know why it's not done that way. I really like breaking software. I feel if I haven't made at least one developer cry a day I haven't done my job. :D

The BOINC Alpha list seems to be more for for those willing to do testing & provide feedback. To me it is not really designed for a way for users to report issues with current released versions.

With a user base of around 2,000,000 I would see a traditional defect reporting system to be very inefficient. Having had to personally deal with beta sites during software development. The time consumed in sorting though defect reports from the field can be staggering. At the place I currently work we are lucky enough to have another group of people between the developers & testers.

However the devs want their defects reported to them. Once they get the defect & try to reproduce it. They have to decide how large of an impact it is. If it effects 100 out of 2,000,000 users. Then the chances of it getting fixed sooner are probably not very high, or it might get deemed to be working within acceptable parameters. Something along the lines of "If I click this button 500 times in a minute the software crashes". Would be an example of working within acceptable parameters.

The main problem which I think was already stated well with stating from scratch. Is that all your old defects will crop back up. I make a joke at work whenever I get a new version that was "written from scratch". I say "Would you like all the defects from the old one before I start testing since I'll be writing them up anyway?" Which is more often true then not.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 974985 · Report as offensive
Luke
Volunteer developer
Avatar

Send message
Joined: 31 Dec 06
Posts: 2546
Credit: 817,560
RAC: 0
New Zealand
Message 974986 - Posted: 1 Mar 2010, 7:32:26 UTC
Last modified: 1 Mar 2010, 7:33:00 UTC

Sigh.
I simply post my opinion and a proposal for other users to accept or deny, and I get teared apart and bashed for saying this and that. Yes - I'm not denying the developers do a good job - I'm saying perhaps they could try a new strategy to build BOINC.
I want to know your polite personal opinions, not get told right in my face that "I suppose you want someone else to do it for you" "It's all about the credits, that all that matters to you" "rant rant rant, that's all you seem to do"
Can I not get a courteous gracious opinion without some fool metaphorically punching me in the nose? Is that too hard to expect from fully grown adults???
- Luke.
ID: 974986 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 974988 - Posted: 1 Mar 2010, 7:45:50 UTC - in response to Message 974981.  

This isn't an argument about credits. Start another thread if you wish to rant about that.

What bothers me most about this "community" is how everyone wants to write, but nobody bothers to read.

It's like trying to teach a pig to sing. It is a waste of your time, and it annoys the pig.

Again, for each and every vote for "a better BOINC" there exists a unique individual concept for "a better BOINC."

So, I presented my vision for "a better BOINC" and instead of taking it as an example of how diverse those views might be, you attack the vision, and completely miss the point.

It's not about that.

It's about the fact that, assuming this was a democracy (it is not) the voting process is mind-numbingly flawed.

Read the first half of the message.

Voting is stupid. Software is not developed by democracy. It sure as hell isn't a democracy when you leave out the people who pay for the (overloaded) servers.

Voting on broad concepts like "an overhauled BOINC" is meaningless if everyone can imagine their own idealized "overhauled BOINC."
ID: 974988 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 974989 - Posted: 1 Mar 2010, 8:04:22 UTC - in response to Message 974986.  
Last modified: 1 Mar 2010, 8:23:49 UTC

Simply said, you're not a very good listener. You still want your initial plan to continue, even while it has been refuted plenty of times, including explanations on the why it's a (bad) (stupid) idea. You seem to only want people who stand behind you on this insane plan to post in this thread. (Renaming it to "Petition, only answer if you agree with me" is more suitable).

You do know how long it took for BOINC Beta to start and come out of beta?
April 30th 2002 is the approximate all-BOINC start-date, while the nearest I can find for BOINC Beta to end is 26 Aug 2004 with CPDN opening to the public. So around that time, although I still feel it was December 2004 that the Beta servers were moved over fully to Seti.

Are you willing to redo those 2 years and a bit?
What if within the 3 years after that you come up with all the same 'bugs' again? Restart again?

Remember, it really isn't just Seti that will be affected. You're talking about a complete rewrite of all BOINC, client, server software, documentation, the forum software, everything. You're going to have to sell it to all the projects and users out there, tell them why for the next 2 years there will be no updates on client or server software.

Are you still up for it?
In what way will you help?
ID: 974989 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19062
Credit: 40,757,560
RAC: 67
United Kingdom
Message 974991 - Posted: 1 Mar 2010, 8:30:39 UTC

I am listening, but I still say it needs an overhaul.
Just like MS when it moved from the DOS/Win95 age to the Win2000/NT age etc.

After six plus years, the present day BOINC is creaking and I think someone, and I have no idea who, needs to look and see if better ways can be found to use present day hardware and find better solutions to some of the band-aids that have been applied to the present version.
ID: 974991 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 974993 - Posted: 1 Mar 2010, 8:33:11 UTC

Good news, I found the improved rewritten version for anyone who voted Yes.

From BoincMgr, hit: Ctrl+Shift+V

ID: 974993 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 975014 - Posted: 1 Mar 2010, 13:11:55 UTC
Last modified: 1 Mar 2010, 13:32:04 UTC

While the Americans are asleep, I wonder if I can make a slightly calmer post? (All the best flame wars seem to break out while I'm in bed :P )

BOINC (the organisation) is a curious animal. It has two objectives: to deliver science to the citizen, and to deliver citizens to perform the science. Citizens like to have lots of extra features, like message boards and feedback: scientists (I may be over-generalising here) don't. They'd just like their science back, please: preferably right first time, quickly, and with the minimum of fuss. That way they can keep their databases lean and mean, minimal-maintenance, and get on with the important task of writing their Nobel-winning discovery paper.

The other problem for BOINC is that it's infrastructure. Nobody loves (or loves to fund) infrastructure. The best ones are completely invisible (how many people here could write or debug a USB device driver? And how many of you think about that when you plug in your latest and greatest toy? Surprisingly often, the infrastructure - just works). So David is never going to get a rush of volunteers to run BOINC, or fund it, or develop it - he just gets people rushing past, picking up a copy as a necessary evil on the way to running the project that they're really interested in. Like USB, the vast majority won't think about the infrastructure at all: and those who do, will only do so when it appears not to meet their preconceptions about how they'd like their copy to work.

That leads on to my main difficulty with the opening premise of this thread: "a complete code overhaul or ground up design". Who's going to do that?

As I see it, there are three possibilities:

1) The existing project management and programmers
2) The existing project management, but a fresh programming team
3) A completely fresh project, such as Synecdoche are attempting.

Think about each of those in turn. Really think about them.

(1) is really hard to do. Instinctively, a programmer already 'knows' how something should be done: they've encountered the problem before, worked out how to do it, and - eventually - got it working and moved on. That same solution will immediately pop into their head as they reach that stage of the job: it's really hard, and counter-productive, to deliberately forget the previous version. After all, computer programs have only become as sophisticated as they have in the sixty years since LEO moved out of the research lab and into business, because code and coding techniques have been shared, circulated and reused. Ground-up without preconceptions is hard.

2) would be hard in any environment, as the examples posted by previous contributors make clear. Do you sack the previous team, and re-start with no preconceptions at all? Or do you keep them on, perhaps maintaining the obsolescent version, so that you can pick their brains for the core design features but resolutely reject the implementation they've already sweated blood over? That's not just hard, it's cruel. And expensive: you'd have to triple the budget, at least, to do it in a reasonable timescale.

3) is the hardest of all. The new start-up has to win the trust and loyalty of both communities, the scientists and the citizens. I don't see much sign that Synecdoche, with 24 downloads and three users, has found a way of doing this yet. Perhaps the only viable candidate with sufficient reputation and finance would be IBM, with their existing interest in WCG and PtP. Would you really want citizen-science to move from an academic base to a commercial one?

I mentioned finance just now. Perhaps some background would reinforce the point. As a freelance, with a background in both computing and charity work, I was for about 15 years the principal programmer for FunderFinder, a small UK charity providing information, advice and software for organisations (and later individuals) seeking to raise money. I wrote the DOS version, and migrated it to Windows: but I retired from writing the Internet upgrade, largely for the reasons listed in (1) above - I wouldn't have been any good at it. During my time with FunderFinder, we worked closely with the Directory of Social Change, the leading UK publisher in this field: I upgraded their flat-file database to relational, so the two organisations could work more closely together.

So believe me when I say that I can't see how you could write a credible funding proposal to implement either (2) or (3). They aren't going to happen.

So we're left with (1). And the way to achieve that is not a petition signed by 14/324,279, or 0.004% of the active user base. Re-writing is out: we're left with incremental change, like it or lump it.

That's not to say I'm all in favour of the current development process. I think it's got major flaws, and I'll list some of them. But they can only be overcome by working constructively with the team (the only team we've got): they can't be overcome by alienating them.

* The citizen-scientist dichotomy. I think too much emphasis has been put on recruiting citizens recently, at the expense of recruiting/retaining scientists. Constant change means that there isn't the simple, secure base that the scientists need. And without projects, there is no BOINC. And without projects, and robust software, the citizens who sign up drop out again. Far better to get the core infrastructure working again, and then recruit volunteers on the strength of that.

* Version control. It's crazy that the 'recommended' version is still v6.10.18 (almost four months old), while the development version has reached v6.10.35 already. We got within a few hours of a recommended release of v6.10.29, but I had to squash that because of a newly-introduced bug which David himself agreed was "critical". They must stop introducing new features for long enough to get a new 'best so far' version debugged and released.

* Mind-set. (And here I do come close to agreeing with Luke's call for a re-write). It's clear, I think, that the developers haven't completely taken on board and fully internalised all the implications of the switch from "1 project = 1 application + 1 CPU" to multi-resource, multi-core, multi-threading, multi-application, multi-everything projects. The bug in v6.10.34/.35 (which David has corrected overnight - watch out for v6.10.36) clearly arose from a subconscious assumption that an app_info.xml file would only ever contain one app_version, and hence use one resource. They have to train themselves out of that mind-set, and perhaps one way to do it would be a a thorough code review (not re-write), concentrating on areas where the entity-scopes are wrong (project-level DCF being a much-quoted case in point).

I think that's enough: I'd better shut up now.

PS One I prepared earlier: please read mailing list bug management as part of this post.
ID: 975014 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 975039 - Posted: 1 Mar 2010, 15:58:04 UTC - in response to Message 975014.  

Thank you Richard for your rational discussion of the matter.
Some of us apply more emotion than logic to the situation, and then things degrade to personal attacks when others do not agree with our thoughts or opinions.
Which does little to advance any resolution to the subject at hand.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 975039 · Report as offensive
Profile Bill Walker
Avatar

Send message
Joined: 4 Sep 99
Posts: 3868
Credit: 2,697,267
RAC: 0
Canada
Message 975045 - Posted: 1 Mar 2010, 16:16:32 UTC - in response to Message 975039.  

Thank you Richard for your rational discussion of the matter.
Some of us apply more emotion than logic to the situation, and then things degrade to personal attacks when others do not agree with our thoughts or opinions.
Which does little to advance any resolution to the subject at hand.


OK, who hacked into SETI user accounts and stole Mark's ID?

ID: 975045 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 975047 - Posted: 1 Mar 2010, 16:22:44 UTC - in response to Message 975045.  

OK, who hacked into SETI user accounts and stole Mark's ID?

LOL!

It wasn't me, honest
ID: 975047 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · Next

Message boards : Number crunching : BOINC needs a overhaul


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.