To add some personal data, I've counted the wu's listed in my client file. It appears there are 1168 unique _0-files, 248 _1-files, and 45 _2-files. They each have estimated run times of 1 hr.

So this snap shot indicates that 3.1% of the work given to me right now is because of a failure of somekind else where, and will require 45h of compute time, making 45h further from finding ET.

I don't know what variation to expect because this is a snap shot and seti doesn't publish data in this area.

[quote]Maybe I am wrong but I believe the re-issues can be calculated quite easily from the "Workunits waiting for db purging" and the "Results waiting for db purging" numbers.
[quote]

I think I mentioned that scarecrow tried to plot these numbers for months but recently quit. I think the formula was in error or the indicators were inconsistent. It was a nice try, though.

To add some personal data, I've counted the wu's listed in my client file. It appears there are 1168 unique _0-files, 248 _1-files, and 45 _2-files. They each have estimated run times of 1 hr.

So this snap shot indicates that 3.1% of the work given to me right now is because of a failure of somekind else where, and will require 45h of compute time, making 45h further from finding ET.

I don't know what variation to expect because this is a snap shot and seti doesn't publish data in this area.

OK, 3$ divided amongst:
1) The recent bug in the test version of the BOINC client that would not stop downloading WUs.
2) Abandoned tasks. (User quit)
3) Equipment failure.
4) Equipment retired with BOINC still on it.
5) Ghost WUs.
6) Over heating.
7) Over ambitious overclocking.
8) Queues set too large.
9) Machine not on enough of the time.
10) Code bugs.
11) Out of HD space.
...
Note that MOST of these causes are not under the control of the project.
There is a fix for the Ghost WU problem, however, it eats so much server resources that it was turned off.

---

BOINC WIKI

Ok, we could make suggestions as to how to reduce the effect of each of these, but how would we know it is working? My point has been that there is no published sensitive statistical measure, and so we don't know much.

Nevertheless, off the top of my il-educated head:
1) New software should be better tested before being released. Sorry, that's a fact. It is also a motherhood statement and therefore doesn't say much. We would need to understand the boinc development process better to make suggestions here, I suppose.
2) Minimize the number of tasks issued to any user so that when he quits the effect is small. (This is related to cache size below)
3) Same as #2
4) Same as #2
5) Never understood what ghost wu's are; couldn't keep up with those endless threads at the time they were a hot item.
6) Issue tasks based on previous success rate for a given host. If the guy is too agressive with overclocking, then he isn't helping the project much and he won't be missed.
7) Same as #6
8) Take the cache sizing out of the users hands, at least a bit, by making it a dynamic parameter that the system controls. The user has no skin in the game if he can download large caches willy-nilly. Also, a smaller cache size may be beneficial for the servers.
9) If this is high on the paredo, then consider some sort of pinging. Also, I don't understand the deadline process (at least without going back and reviewing the message boards). It seems odd that on a given day I get some wu's that are needed 2 weeks from now and some of equal length that are due in a few days. But that happens and probably has some impact with machines that aren't available much.
10) See 1.
11) I assume you mean the servers. If so, #8 may help.
...

Again, my point is not that there are failures or inadequacies. My point is that it would be nice to see the consequences of the development program more quantitatively/publically. The process of change matters to me, and I would hope others.

1) New software should be better tested before being released. Sorry, that's a fact. It is also a motherhood statement and therefore doesn't say much. We would need to understand the boinc development process better to make suggestions here, I suppose.

Go on then, join the Alpha testers group, bring all your friends, the more, the merrier. If you want to just let others do the testing, do not make comments like that.

Again, my point is not that there are failures or inadequacies. My point is that it would be nice to see the consequences of the development program more quantitatively/publically. The process of change matters to me, and I would hope others.

It's probably a good idea for people to stop posting about the newest Alpha version on these forums, as when things go wrong with the client, it'll usually crash and delete all your work. Got comments about the latest alpha? Post them on the Alpha email list, not on the BOINC Dev forums, not in a thread on these forums. The BOINC developers do not read forums.

---

The BOINC developers do not read forums.

Sad, since they are the ones who decide what features go into the forum software. But probably true.

If I was a Boinc developer I wouldnt want to read the boards. The only time something comes onto these boards is somethings wrong, 3 people are unhappy and 50k dont care.

Besides, AFAIK there 'is' a way to get info to the develpers but its not on the boards. Theres to much 'junk' here to make it worthwhile. Scan 50 threads about 1 issue. Theres 35 different reactions to that 1 issue and a bunch of different ideas on how to fix it.

Besides: Name of this section?
Number crunching
Credit, leaderboards, CPU performance. I dont see 'development' anywhere in there.

Sincerely,
Daysteppr

---

If I was a Boinc developer I wouldnt want to read the boards. The only time something comes onto these boards is somethings wrong, 3 people are unhappy and 50k dont care.

Besides, AFAIK there 'is' a way to get info to the develpers but its not on the boards. Theres to much 'junk' here to make it worthwhile. Scan 50 threads about 1 issue. Theres 35 different reactions to that 1 issue and a bunch of different ideas on how to fix it.

Besides: Name of this section?
Number crunching
Credit, leaderboards, CPU performance. I dont see 'development' anywhere in there.

Sincerely,
Daysteppr

Well gee, you think?

The point is as a developer, silence from the user base generally means you did something right, or at least you didn't break anything relatively important.

However, if you really want to know if what you did accomplished your objective, then you go where the rubber meets the road in practice.

Simply put, you just don't get that in the BOINC dev community, and if you aren't one of the 'gang' your experience trying to give feedback is, shall we say, somewhat less than satisfying.

Alinator

Let me tell you a story. A true story.

On New Year's Day this year, a public holiday in all countries relevant to this story, I noticed a server fault on a BOINC project. It was a small fault, unimportant, and it could have been left until after the weekend and the start of the full working week. But it wasn't going to clear itself: it needed project admin action, and so I posted the error log on a message board so that details would reach admin in due course.

Within 80 minutes, a project administrator had posted in reply:

I'm working on the server now.
Despite having warning scripts in place it's [failed] and gone down without having sent me any warnings, which is a bit of a nuisance. It will take a few hours to [fix] and bring it up again.

In fact, it took less than that, and the server was up and running again less than three hours after my initial post.

There are several morals to be drawn from that story:

1) Project admins are dedicated people too, and are as keen to keep their 'babies' running on public holidays as we are.
2) Relying on automated tools and scripts isn't always enough - the mark one human eyeball is also a useful tool, even when it's installed in a volunteer user.
3) (and probably most significant) - the message board I posted on wasn't available to the general public - it was a restricted board, limited to technical issues only, with access by invitation only. Within that restricted board, there is one thread reserved for server problems. So the project administrator only has one place to monitor, and knows that anything which gets posted there is likely to be reasonably relevant. Which probably explains why the admin in question was reading it on New Year's Day. (They may have been alerted by an email subscription, but I don't know that for certain).

I fully agree with daysteppr that SETI project admins can't be expected to read every word posted on these busy and, would it be fair so say, chaotic, message boards - and in fact, we know that Eric Korpela does read here from time to time and sometimes drops in and adds to the conversation. He's always welcome.

My comment related to BOINC developers, not SETI admins, and I find it much less explicable that BOINC developers don't appear to monitor either their own message boards, or the more formal 'trac' bug reporting and recording database. That frustrates me much more than SETI's limited resources do.

...or the more formal 'trac' bug reporting and recording database.

When you make a ticket, an email of it is sent to you (the maker) and the person you assigned it to. But you have to know that there's lots of stuff on their to-do lists already and they won't drop everything they're busy with to go answer a ticket, unless it's showing a big bug.

They do answer on the BOINC Dev forums, when asked to come take a look at a certain thread or problem. But think, if they would have to read the forums and answer to them, who will program the next BOINC then? BOINC perhaps? ;-)

---

...or the more formal 'trac' bug reporting and recording database.

When you make a ticket, an email of it is sent to you (the maker) and the person you assigned it to. But you have to know that there's lots of stuff on their to-do lists already and they won't drop everything they're busy with to go answer a ticket, unless it's showing a big bug.

They do answer on the BOINC Dev forums, when asked to come take a look at a certain thread or problem. But think, if they would have to read the forums and answer to them, who will program the next BOINC then? BOINC perhaps? ;-)

Well, I'm willing to go along with that in principle.

However, I do follow along in the dev community, and recently there has been a move to use SAH Main as more of an Alpha guinea pig that I can recall in a long time.

Therefore, I don't find it an extraordinary request that some more attention be paid to NC here than might otherwise be given.

Alinator

The BOINC developers do not read forums.

Sad, since they are the ones who decide what features go into the forum software. But probably true.

I'm going to quote the full paragraph from the earlier message:

It's probably a good idea for people to stop posting about the newest Alpha version on these forums, as when things go wrong with the client, it'll usually crash and delete all your work. Got comments about the latest alpha? Post them on the Alpha email list, not on the BOINC Dev forums, not in a thread on these forums. The BOINC developers do not read forums.

Why? Because there was lots of good advice on how to talk to the BOINC developers, and the only part that was quoted was how NOT to talk to the developers.

I won't argue against better testing, and while I'm on the BOINC Alpha project (and the SETI Beta project) I'm probably not as active as I should be.

There are only two ways to do this kind of testing: have an active Alpha or Beta test group, or do all the testing in-house and hope you hit everything.

Sorry if I'm venting, but I currently have a project (non-BOINC) in beta, and I'm frustrated with my beta test group -- for lack of feedback. Better (and more) testers are probably welcome, and BOINC will get better as a result.

Just keep in mind that Alpha and Beta software versions are by definition broken. If you are going to test, you should be prepared for some pain.

---

Just keep in mind that Alpha and Beta software versions are by definition broken. If you are going to test, you should be prepared for some pain.

I hear you, but there lies the problem. When threads come up "6.6.x is out", people will read it and think "hey, I want it now", without realizing it's alpha software.

OK, perhaps that Martin (and whomever else posts these threads here) can post a strong warning about it, but even then there are people who do not read the whole thread, just ask for the link to the software and complain here because it threw out their cache or broke their NNT setting (which, by the way was a scheduler problem on the server).

Then there is the problem that most of the BOINC developers do not crunch for Seti, they have no RAC > 1, so they cannot post here with questions leading to possible answers. They do not have those constraints on the email lists. And it's way easier for them to ask for details there than it is for any of us to play middle man here, ask for the details, get the answers from the other person and send them on to the developers.

---

Just keep in mind that Alpha and Beta software versions are by definition broken. If you are going to test, you should be prepared for some pain.

I hear you, but there lies the problem. When threads come up "6.6.x is out", people will read it and think "hey, I want it now", without realizing it's alpha software.

There are only two solutions:

1) Restrict access to the alpha and/or beta versions.

2) Warn people.

I see a big red "(MAY BE UNSTABLE - USE ONLY FOR TESTING)" next to 6.6.4 on the download page.

When the download page was better hidden, people sought out, and posted links directly to the installer. Make it harder, and some people will post the latest restricted version to their own pages. At least making things a little more public makes that less likely and means that people might read.

... but there comes a point where you have to stop helping and let people do what they're going to do. If someone doesn't want to read, and absolutely has to run the absolute latest, who am I to blow against the wind?

---

Just keep in mind that Alpha and Beta software versions are by definition broken. If you are going to test, you should be prepared for some pain.

I hear you, but there lies the problem. When threads come up "6.6.x is out", people will read it and think "hey, I want it now", without realizing it's alpha software.

OK, perhaps that Martin (and whomever else posts these threads here) can post a strong warning about it, but even then there are people who do not read the whole thread, just ask for the link to the software and complain here because it threw out their cache or broke their NNT setting (which, by the way was a scheduler problem on the server).

Then there is the problem that most of the BOINC developers do not crunch for Seti, they have no RAC > 1, so they cannot post here with questions leading to possible answers. They do not have those constraints on the email lists. And it's way easier for them to ask for details there than it is for any of us to play middle man here, ask for the details, get the answers from the other person and send them on to the developers.

I would have thought that, if the developers use a project, in this case Seti, as their test site, then it is in their interest to also crunch for that project.
They might then just see some of the problems first hand, and assuming they get some credit, might just pop in with the odd msg to say they are on the case.

It might just help if the alpha and beta versions of BOINC were not on the public "index of /dll" page. That they have their own pages, with these only open the alpha and beta testers.

Ok, we could make suggestions as to how to reduce the effect of each of these, but how would we know it is working? My point has been that there is no published sensitive statistical measure, and so we don't know much.

Nevertheless, off the top of my il-educated head:
1) New software should be better tested before being released. Sorry, that's a fact. It is also a motherhood statement and therefore doesn't say much. We would need to understand the boinc development process better to make suggestions here, I suppose.
</quote>This was during a test that the problem was noted. It was not in released code.<quote>
2) Minimize the number of tasks issued to any user so that when he quits the effect is small. (This is related to cache size below)
3) Same as #2
4) Same as #2
5) Never understood what ghost wu's are; couldn't keep up with those endless threads at the time they were a hot item.
</quote>Ghost WUs are easy to explain. The client calls the server for more work, and the server allocates some to the client. The network then goes kabloey and the client never gets word that there was any work assigned (usually times out the network request).<quote>
6) Issue tasks based on previous success rate for a given host. If the guy is too agressive with overclocking, then he isn't helping the project much and he won't be missed.
</quote>This is handled by the daily quota / CPU. However, it can take quite a while to cut work down to 1 task / day. If even 1 in 10 completes successfully, the quota may never drop enough to be significant.<quote>
7) Same as #6
8) Take the cache sizing out of the users hands, at least a bit, by making it a dynamic parameter that the system controls. The user has no skin in the game if he can download large caches willy-nilly. Also, a smaller cache size may be beneficial for the servers.
</quote>That would be fine for always connected users, but not for those planning a disconnected period.<quote>
9) If this is high on the paredo, then consider some sort of pinging. Also, I don't understand the deadline process (at least without going back and reviewing the message boards). It seems odd that on a given day I get some wu's that are needed 2 weeks from now and some of equal length that are due in a few days. But that happens and probably has some impact with machines that aren't available much.
</quote>Pinging the client from the server is not possible.<quote>
10) See 1.
11) I assume you mean the servers. If so, #8 may help.
</quote>No, I meant on the client (I have lost a couple of WUs this way)<quote>
...

Again, my point is not that there are failures or inadequacies. My point is that it would be nice to see the consequences of the development program more quantitatively/publically. The process of change matters to me, and I would hope others.

There are some tasks that are going to fail or not be returned no matter what the project does. A 3% re-issue rate is far better than it used to be (around 30%). There have been improvements, and waiting until it was gold plated before starting would also be a mistake.

---

BOINC WIKI

Winterknight, or was it Waterknight? wrote:

I would have thought that, if the developers use a project, in this case Seti, as their test site, then it is in their interest to also crunch for that project.
They might then just see some of the problems first hand, and assuming they get some credit, might just pop in with the odd msg to say they are on the case.

Correction, "they" don't use this project as their test site. Perhaps that Dr. Anderson puts the newest scheduler and forum software on here and all that. But the clients are all tested in the alpha group, before they're being released as the recommended version for everyone, who hasn't got it yet, to download.

It might just help if the alpha and beta versions of BOINC were not on the public "index of /dll" page. That they have their own pages, with these only open the alpha and beta testers.

That's where they show up first anyway, but the link to it is already public knowledge. It also doesn't say on that site that it's an alpha. No warning at all. The warning is only there on the all downloads and dev=1 links. And even then people do not read that text and complain that it doesn't do what they expect it to do.

The whole 6.6 series has a completely new work fetch and client side CPU scheduler, it has a new way of calculating the long term and short term debt, it has fixed some bugs and added a whole load of new ones. Things you would probably know if you followed the alpha and development email lists, but which aren't public (read: forum) knowledge. And so when the client does things differently than what people are used to, where do they then post their remarks and complaints about it?

Now it may not be perceived as a problem if they only posted it to Seti and the BOINC Dev forums, but there are plenty of threads in the other 75+ project forums as well. Do you really want the developers to go read all those forums as well, crunch some for them in case they have to ask questions?

Common sense says it's undoable. Then again, where is common sense on these forums, where people still wonder why when they play their heavy 3D games while they run CUDA at the same time, that their games are sluggish as heck and/or they're turning out only errors?

Ned Ludd wrote:

... but there comes a point where you have to stop helping and let people do what they're going to do. If someone doesn't want to read, and absolutely has to run the absolute latest, who am I to blow against the wind?

The helpful Luddite next door? ;-)

---

psst...it's the cuda

---

"The most exciting phrase to hear in science, the one that heralds new discoveries, is not Eureka! (I found it!) but rather, 'hmm... that's funny...'" -- Isaac Asimov

There are some tasks that are going to fail or not be returned no matter what the project does. A 3% re-issue rate is far better than it used to be (around 30%). There have been improvements, and waiting until it was gold plated before starting would also be a mistake.

Sun Tzu said "No plan of battle ever survives contact with the enemy."

You can work for decades on what you think BOINC can be, or you can get it out there, gain experience, and watch it evolve into what it should be.

Getting the reissue rate from 30% to 3% is a strong indicator of progress.

---

. . . re-visiting: Develop an "Abundance Mentality"

quote]

An "abundance mentality" is more than having a positive mental attitude,

although a positive mental attitude is very important.

When you have a positive mental attitude, you look at how things can be done

rather than why they can't be done.


You believe that "where there's a will, there's a way."


You look at possibilities and opportunities rather than obstacles and problems.

This mindset is important for success in any endeavor.

< snip >

Abundance starts in your mind. The more you think abundantly,

the more abundance you can enjoy.

The more abundance you enjoy, the more success you will enjoy

[/quote]


~ as an after-thought: Thanks to Lunatics - it works quite well - Thanking Each of You btw ~

---

BOINC Wiki . . .

Science Status Page . . .