Are you sure you're Canadian? you(sic) over simplify(sic) complicated arguements(sic) like a German, or an American.

If I leave my old clunker running 9 to 5 because I'm expecting business related e-mails, and then give the spare CPU cycles to BOINC, there is no net increase in wasted energy, green house gasses(sic), etc. (Bob's comment: In reality, energy use doubles.)

If I build a multi CPU multi GPU speed machine to occaisonally(sic) read those same e-mails, and let it run 24/7 to get RAC to prop up my fragile ego(sick), THAT is Environmentally Irresponsible.

... Not to bash on those who choose to go the extra mile, but the project never expected anyone to build entire farms and shoulder more responsibility (power consumption) than anyone else.

The entire point of Distributed Computing is that it is done by spare CPU cycles around the world, meaning when the computer is powered on doing other things but not using all of its CPU cycles. Turning off your computer when you are done, or when its hot out, or when you go on vacation - its all the same.

This is a down-side to the natural competitiveness that many people have; their yearning to reach the top and collect points and be number 1. In this game, it costs real money to do that. Congrats to those who can. The rest of us have a budget and other problems, especially in this economic troubling time.

Best Wishes on your next project!

@ all

Congratulation! to all on reaching your own personal Milestones!

Byron

I retired the Top Host that served so well for so long. It was a wonderful machine.

Check out my profile here for a picture of the computer, plus some technical data.

Bob

Congrats! That was an amazing host, and your next project seems very ambitious indeed!


Off topic, but is that the same ExecPC that started out as a BBS back in the 80s and 90s? ExecPC was one of my top favorite BBSes to dial into, and it always made me want to start my own BBS because it seemed so fun.

The scheduler in 6.6.36 seems to be totally broken. Starting a machine from scratch last night I've received probably 300 6.03s. Meanwhile my GPUs have been idle for 16 hours. S@H only on the machine in question.

1) AP tasks influencing the anticipated running times of CUDA tasks is a design flaw in BOINC which we're going to have to live with until at least BOINC v6.10

The worst effects can be mitigated by careful fine-tuning of the FLOPs figures in app_info.xml: if you get it right (carefully balanced for the performance bias of your particular hardware), you can avoid EDF almost entirely.

2) Except that then, you wouldn't have discovered the mis-behaviour of CUDA under stress! From what I'm reading, there are problems:

a) When there are two or more CUDA devices in a single host (which rules me out for testing, sadly)

b) When a combination of cache size/DCF/deadlines brings 'shorties' forward in EDF - which to my mind should be flagged as 'High Priority', but doesn't seem to be.

I think someone - not me, I'm single GPU only - is going to have to do some logging with <coproc_debug> (see Client configuration), try and work out what's happening, and report the analysis and logs to boinc_alpha.

Fred said: ... When one EDF CUDA WU finished another started and the partner EDF CUDA WU continued to completion. When a CPU MB WU completed and uploaded, there was no effect on the CUDA WU's in flight. And when I gave up and restored the original settings (including removing the config flags) and allowed the pre-empted CUDA WU's to complete, they did so without error (no -5!!).

The only difference on my host at the moment is that I am running down my cache..

Fred, is there an imbalance in cache size between your 608 vs. 603+AP queue on your system? I mean, do you have an intentional difference in cache size, achieved by filling up one, then going back to a shorter queue for running? On mine there is a full 10 days of AP for the CPU, and half that duration of MB for CUDA. I'm wondering if BOINC is confused by the cache size contrast?

I work on the pricipal that the minimum turn-round time specified for MB tasks is 7-days, so with a 3-day cache I am rarely going to run into EDF. I don't try to micro-manage the cache - I just tell it 3-days (with a 0.1 day connect interval) and leave Boinc to it.
You have a (approx) 5-day cache for MB. If your cache is filled up when your DCF is at or near its minimum value and then the DCF is doubled by a task taking longer than expected (I have watched this happen), then any 7-day return task that was toward the end of your queue is in deadline trouble and you are into EDF. Running a 3 (or less) day cache gives much more headroom in this respect and I have not run out of work at any time recently, even with the major network problems we have experienced. I am pretty sure that halving your cache size would virtually elimate the "waiting to run's".

F.

If it's anything like mine (running v6.6.28, .31, or .33), then once the GPU is in EDF mode (and it doesn't show "high priority" in BM) then for every 2 tasks that start, one will be left in "waiting to run" when the next 2 start until it comes out of EDF mode. I have recently noticed that with v6.6.33, when the GPU is in EDF mode, the completion of a CPU task will put *both* GPU tasks into "wating to run" and start a new CPU tasks and 2 new CUDA tasks. The only way to avoid all this seems to be to stay out of EDF mode on the CUDA which I do manage to do most of the time with my 3-day cache.

F.

Wating to Run issue:

...and it just created 3 more in the past few minutes.

Bob

Bob, you may be thinking of the way they have narrowed the range on the VLARs. I've got a few that would have been -6'd before but are now deemed to be acceptable to run. Raistmer was a bit on the cautious side when he first figured the range.

From Richard:
...
Starting with v6.6.23 (I think they missed a couple of release numbers), CUDA tasks should run in the order they're received from the server, unless there's a deadline problem. Then, they would switch to 'earliest deadline' mode, but should also show a flag for running in "High Priority". Are you seeing that? (You may need to extend the width of the 'status' column to be sure). Fred's screenshot doesn't show High Priority, so the shorty tasks may simply have been the next due to run in FIFO order: unfortunately, with a bespoke sort order set, we can't see the tasks in issue order.

[Tip:
If, like Fred, you have a bespoke sort order set, you can clear it from the registry by clearing these two registry keys:

[HKEY_CURRENT_USER\Software\Space Sciences Laboratory, U.C. Berkeley\BOINC Manager\Tasks]
"SortColumn"=dword:ffffffff
"SortAscending"=dword:00000001

It's the only way I've found.
/Tip]

With a 1.6 day cache, there should never be any need for High Priority running, and hence no EDF. But beware: if you have a full cache, and something goes wrong with your Duration Correction Factor or other time metrics, a 1.6 day cache can suddenly evaluate to seven or more days, and trigger EDF. I've had that happen with a near-VLAR which escaped my rebranding. Look and see if your current BOINC estimates for unstarted tasks seem to be realistic.

If that doesn't throw up any clues, we may be moving into the territory of extended debug logging - see Client configuration. Are any of you up for that? We're probably talking about <coproc_debug>, <cpu_sched> and <cpu_sched_debug>.

I wondered if you'd had so many VLAR kills it had cleared all your tasks out - making it look like you'd had lots of the preempt problems.

The other -5 I mentioned is the memory problem rather than then the pre-empt issue. 1247338546
...
<core_client_version>6.6.31</core_client_version>
<![CDATA[
<message>
- exit code -5 (0xfffffffb)
</message>
<stderr_txt>
...
setiathome_CUDA: Found 2 CUDA device(s):
Work Unit Info:
...............
WU true angle range is : 8.984127
Cuda error 'cudaMemcpy(dev_cx_DataArray, cx_DataArray, NumDataPoints * sizeof(*cx_DataArray), cudaMemcpyHostToDevice)' in file 'd:/BTR/SETI6/SETI_MB_CUDA/client/cuda/cudaAcceleration.cu' in line 262 : unspecified launch failure.
SETI@home error -5 Can't open file
(work_unit.sah) in read_wu_state() errno=2

</stderr_txt>
]]>

From Fred W:
...
So the completion and upload of an EDF WU on one half of my GTX295 causes the WU running on the other half to be pre-empted. I then end up with a whole list of "Waiting to run" WU's which error out with: [-5]

From Richard:
Unfortunately, all my CUDA cards are single-core, so I can't follow this one up with personal observations. But it sure sounds as if there's another bug still waiting to be tracked down.

You also seem to have been unlucky and have snagged a whole bunch of VLARs which have been killed off :-

...
I see one -5 task which looks like the output from the pre-empt problem from this morning at 5:35 which I assume was after you've upgraded to 6.6.31? However other -5's say they are VLAR kills as well.

</stderr_txt>
]]>

Other errors include :-
Cuda error 'cudaMemcpy(dev_cx_DataArray, cx_DataArray, NumDataPoints * sizeof(*cx_DataArray), cudaMemcpyHostToDevice

which seems to point at a memory problem with CUDA - are your machines ever rebooted. The extra waiting tasks may still be in CUDA memory and it just ran out - famous last words - perhaps a reboot will clear the problem.

I have started to get a whole bunch of waiting to runs - I think this is because the preempting is no longer erroring them when the tasks switch - before they just errored and disappeared.

I really abused 6.6.31 when I upgraded it and coouldn't make it -5 once - all with WindowsXPSP3.

John.