I attempted to Update my LHC this afternoon (EST) after sending in completed WU"S. I started receiving more WU's. Seti the contacted the schedualer and started downloading 20 WU's. Both downloads tried to run at the same time causeing only Seti to get "Unrecoverable Errors" on all but 3 of the WU's. I felt this might have happened because LHC started downloading first.

Is there something I am missing...Is there a way to "Pause" one program while the other is doing downloads?

Thank You In Advance!

Have A Great Day And A Better Tomorrow!

Regards,

Rocky Cudd
<img><img> <img>

Rocky,

Judging by all of the other problems with downloading that folks have been talking about today, I'd say it was just cooincidence that you happened to get errors (like everyone else) and also be grabbing a download from another project at the same time.

> I attempted to Update my LHC this afternoon (EST) after sending in completed
> WU"S. I started receiving more WU's. Seti the contacted the schedualer and
> started downloading 20 WU's. Both downloads tried to run at the same time
> causeing only Seti to get "Unrecoverable Errors" on all but 3 of the WU's. I
> felt this might have happened because LHC started downloading first.
>
> Is there something I am missing...Is there a way to "Pause" one program
> while the other is doing downloads?

As far as I know this is a new issue that the developers are aware of. I have no rumors about a fix however. As best as I know, the problem is that if two projects are contacted at the same time there is confusion. On my part, what I did was to turn off network access in the hopes that I can contrl the contacts to one and only one scheduler at a time (probably will tell it to pause processing when I do manual updates).

:(

The bad news is, as best I can tell, few people have taken my advice to be multi-project so that if the well runs dry on one side, the cp.net is there to bask in the sun.

I imagine that once someone figures out what exactly is going wrong a cure will be forth comming. For now, my advice is to drop Piarates/Einstein@Home for now, and also Predictor@Home till they go back on the air. Which looks like it is going to be sometime... they still don't have the new servers in ... a good reason to go with Dell ... :)

(I couldn't resist)

That is the best advice I can give you ...
<p>
For BOINC Documentaion: Click Me!

> Rocky,
>
> Judging by all of the other problems with downloading that folks have been
> talking about today, I'd say it was just cooincidence that you happened to get
> errors (like everyone else) and also be grabbing a download from another
> project at the same time.
>


Then again, in light of Paul's post, I suppose I could be wrong.... you know, there's a first time for everything. ;)

(I hope the trolls can see the twinkle in my eye when I say that)

>
> Then again, in light of Paul's post, I suppose I could be wrong.... you know,
> there's a first time for everything. ;)
>
> (I hope the trolls can see the twinkle in my eye when I say that)

No sweat...

I try very hard to not let anyone get at me... I know I am right and you are wrong and that is good enough for me ...

I did 20 Years in the navy, got a masters degree, own my house, got money in the bank, stock market, etc. And no, I am happily married and besides you would not like me anyway ... :)

Seriously though, I am not sure of anything yet ... doing a single one at a time update seems to fix most of the whonky WU that show up with the wrong project. I am not sure yet if my "cure" does avoid this problem or not.

If you get some Wu in the hung states we may have to get physical with them ... but for the moment I am not going to reset cp.net ... too much work done by now ...

I am going to let my stash build up with a bunch to upload and report ... then I will do an update. But to be sure I am going to need a couple days to a week.

I think we have seen our first (and hopefully the last) major cross-project bug ... My natural autistic negative altitude (another naval aviation insiders joke) says it is not going to be the last though ..
<p>
For BOINC Documentaion: Click Me!

> > Rocky,
> >
> > Judging by all of the other problems with downloading that folks have
> been
> > talking about today, I'd say it was just cooincidence that you happened
> to get
> > errors (like everyone else) and also be grabbing a download from another
> > project at the same time.
> >
>
>
> Then again, in light of Paul's post, I suppose I could be wrong.... you know,
> there's a first time for everything. ;)
>
> (I hope the trolls can see the twinkle in my eye when I say that)
>
>

The same issue occurred the last time that the scheduler was off-line. The failed downloads were very common among many that were running just SETI, including me. I had them the last time and also have been getting them today. It does not happen to every downloaded file. I personally had 8 out of 29 that were downloaded today.

Unrecoverable error for result 06my04aa.15945.1650.417318.122_2 (WU download error: couldn't get input files: MD5 computation error)

I never get these download errors except the last two times that the scheduler was down or turned off, as in the last case.

----------



Join the Overclockers.com SETI Team!

> The same issue occurred the last time that the scheduler was off-line. The
> failed downloads were very common among many that were running just SETI,
> including me. I had them the last time and also have been getting them today.
> It does not happen to every downloaded file. I personally had 8 out of 29
> that were downloaded today.
>
> Unrecoverable error for result 06my04aa.15945.1650.417318.122_2 (WU download
> error: couldn't get input files: MD5 computation error)
>
> I never get these download errors except the last two times that the scheduler
> was down or turned off, as in the last case.

Just for interest's sake, this is a pre-v 4.09 ... unbeknownst to me, I still had one 4.05 BOINC Work Manager ... And it has/had those weird download and cross-project problems ...
<p>
For BOINC Documentaion: Click Me!

Paul,

More info:
I am running v4.09 release on both of my seti rigs.
The are running WinXP Pro - One w/SP1 - Other w/SP2
Both are P4-800fsb-HT enabled
Both have had failed downloads today
I did not have any failed downloads during the scheduler up time except when it first comes back online.

----------



Join the Overclockers.com SETI Team!

> Paul,
>
> More info:
> I am running v4.09 release on both of my seti rigs.
> The are running WinXP Pro - One w/SP1 - Other w/SP2
> Both are P4-800fsb-HT enabled
> Both have had failed downloads today
> I did not have any failed downloads during the scheduler up time except when
> it first comes back online.

Now if I only knew who was a developer so I could tell them ... as a participant, all I can do is write about it... kinda like a reporter ... but I don't know if they buy my paper ... :)

My suspicion about the MD5 is that there is actually two problems. The one is the cross project one, but the server load one is different. What I mean is that we have two problems but the reflection of them is about the same ... I am pretty sure that the one I have been talking about because it is more critical is because of an error when there are more than one conversations goinig on between the BOINC Work Manager and the project site.

The second problem mimics this and shows up with a lot of failed down loads. But this time it can be a corrupt URL error, 0 lenght files, or MD5 failures (or all 3) and that is because of a high server load and the server is losing track of the threads and who asked for what ...

The bad news is with the two showing just about the same face it looks like only one problem. Of course, I could be wrong! :)
<p>
For BOINC Documentaion: Click Me!

This was also my suspicion. When they turn the scheduler back on, all the systems are slamming the server for work and to report. Just don't have what we need at this point to handle the load.

I would hope that they are already aware of these issues or have been made aware by reading some of our posts.
----------



Join the Overclockers.com SETI Team!

Hi All,
Thank you for all your responses.
Paul,
I have WIN XP Home and 4.09 on both machines with joint projects, which are LHC and Seti. I keep the network disabled all the time until I am ready to send in WU's. I think I am confusing myself in what I am asking, or I am looking to deep into your response.
At the time I got all the MD5 Errors, I had 3 completed Seti WU's and 16 Completed LHC wu's. As soon as I re-enable the network access all the WU's transfer. But what I have noticed is that as soon as I do a manual update on LHC, Seti auotmatically had sent a request to the schedualer for WU's. On my transfer screen I had LHC WU's downloading 2 at a time at the top of the screen and 2 SETI WU's downloading inthe middle of the screen. What I noticed is that, the first download that starts on SETI would always complete, however the second WU, if it took longer then 19 seconds to begin, it would fail. This will happen repeatly until the first WU is within the 90% bracket of being complete. So I lose about 4 WU's for every one that downloads.
I I could pause one or the other from contacting the schedualer until one finishes that would be great, and stop the problem. I have had it happen twice today so I have exhausted my Supply of WU's for the day, but tomorrow is another day.

I hope this makes better sense of what I was saying earlier, and when I saw in your answer about Einstein and Pirates, I knew I had neither one.

Thanks for the help. ( Just trying to learn)

Have A Great Day And A Better Tomorrow!

Warm Regards,

Rocky Cudd

Rocky,

> I have WIN XP Home and 4.09 on both machines with joint projects, which are
> LHC and Seti. I keep the network disabled all the time until I am ready to
> send in WU's. I think I am confusing myself in what I am asking, or I am
> looking to deep into your response.

Well, some say I am pretty shallow... so looking too deep may not be a good idea ... :)

> At the time I got all the MD5 Errors, I had 3 completed Seti WU's and 16
> Completed LHC wu's. As soon as I re-enable the network access all the WU's
> transfer. But what I have noticed is that as soon as I do a manual update on
> LHC, Seti auotmatically had sent a request to the schedualer for WU's. On my
> transfer screen I had LHC WU's downloading 2 at a time at the top of the
> screen and 2 SETI WU's downloading inthe middle of the screen. What I noticed
> is that, the first download that starts on SETI would always complete, however
> the second WU, if it took longer then 19 seconds to begin, it would fail. This
> will happen repeatly until the first WU is within the 90% bracket of being
> complete. So I lose about 4 WU's for every one that downloads.

Yes, you are seeing the same behaviors I am ... I too have had download failures with BOTH LHC@Home and SETI@Home. Over time, 4.09 seems to be able to clear out the cross-connected files listed. There was a suggestion that you need to do a detach and attach, re-install, etc. and that is not what I have experienced this morning. I went through the list of computers and enable access and then watched. I did get traffic on both schedulers at the same time, but now there seemed to be no issues. I don't know why, or if it is luck. But I am going to stick, for now, to limiting network access.

WIth the 4 day setting I have "wiggle-room" and there is not pressure to rush in where sailors fear to tread ...

> I I could pause one or the other from contacting the schedualer until one
> finishes that would be great, and stop the problem. I have had it happen twice
> today so I have exhausted my Supply of WU's for the day, but tomorrow is
> another day.

I am now thinking that we may have another clue. Based on one data point, but a potential clue none the less. Since I had a fair stack of pending uploads and so forth, I got immediate connects on both LHC@Home and SETI@Home and they pushed up the work, then I did individual Updates, pausing to let them succeed. LHC@Home had already downloaded some additional work so I did not seem to get contention to fill with SETI@Home (at least not this go around). Though, if I have a continued "draw-down" of SETI@Home I should "run-dry" of those on one or two machines today or tomorrow and we shall see if the resource debt forces a re-balance.

> I hope this makes better sense of what I was saying earlier, and when I saw
> in your answer about Einstein and Pirates, I knew I had neither one.

Yes, good.

My newest suspicion is that we may have an issue with the class of scheduler communications. I am only speculating hoping that someone older, wiser, or just plain old smarter will be able to say that I am off my nut. But, suppose the error manifests itself in this way.

Project "A" is contacted to upload, or download work and the BOINC Work Manager and the scheduler of that project begin to communicate.

While this is happening we get a second thread that detects that it is time to contact the scheduler that belongs to project "B" because the time expired on a now work available?

We now have two different communication styles/modules affected. Each thread/module is smart enough to use a spin lock to prevent multiple requests at the same time, but not smart enough to prevent the other type of communication?

Just guessing, a drowning man with clutch at straw ... :)

> Thanks for the help. ( Just trying to learn)

Me too ... as an innocent (ok, maybe not so innocent, but I can dream can't I?) participant and documenter, I am trying to learn all the time. My shining stars are still JKeck and John McLeod (hi guys); like the old commercial ... when they talk, I listen ... :)

Rocky, you can also goto the books and mouse around in there. All of the wisdom we collected during beta and what I have learned since is in there as the best as I can do. Each day I try to add just a little more (This last week I added about 15 new pages, 4,000 additional links and lots of new examples, a total of 250+ plus items changed or added).

As I find errors or new wisdom, I refine all of what is there. So it is in constant evolution... Hmm, I should put in a page date shouldn't I? Well, if I ever finished, what would I do all day?

:)


<p>
For BOINC Documentaion: Click Me!

Rocky,

Just thinking aloud... is there a reason why you need to limit your network communication to only the times you specify?

The reason that I ask is that it seems you're greatly increasing the chances of having the client communicating with two schedulers at the same time. Useful for exposing bugs? Sure. But not so good if you're trying to avoid having them. ;)

If you're on dial-up (and you don't want the client prompting a connection for whatever reason), then of course I can see the need, but otherwise, I'd just let BOINC talk whenever it wants to.

Paul,

You sound like quite a catch, but I agree that it would never work out. 14 years USAF here... I'm a zoomie and you're a squid. Our parent (services) would never approve. ;) Thanks for a good laugh though!

At least I know I can refer to something in my BOINC client as a complete charlie foxtrot and you and Rocky will understand. ;)

-Thunder

> Just thinking aloud... is there a reason why you need to limit your network
> communication to only the times you specify?

There is that problem, however, in my case I also had (and on some machines still do) multiple projects and two of those were shut-down. So, in my case I was doing normal uploads and downloads along with random connections. So, with the "control" we are trying to exert the concept is to put it into a "sand-box" and to observe and hopefully head off problems at the pass (Hey! A non nautical pun!).

> The reason that I ask is that it seems you're greatly increasing the chances
> of having the client communicating with two schedulers at the same time.
> Useful for exposing bugs? Sure. But not so good if you're trying to avoid
> having them. ;)

As I stated else where, at least in my case. it is more of the idea of the kinds of contacts and get rid of the "pinging" on the various projects. I turned off all Pirates for now, and Predictor@Home on aa couple machines. And I may limit the others by detaching on them too ... in a couple days I am going to try one or two clients with auto-connections to see if that will work in a less random environement. Past experience with computer SNAFUs is to clamp down real hard at first and then let it back up SLOWLY. :)

> If you're on dial-up (and you don't want the client prompting a connection for
> whatever reason), then of course I can see the need, but otherwise, I'd just
> let BOINC talk whenever it wants to.

As above ...

> You sound like quite a catch, but I agree that it would never work out. 14
> years USAF here... I'm a zoomie and you're a squid. Our parent (services)
> would never approve. ;) Thanks for a good laugh though!

"Brown shoe" Please ... us Airdales are sensitive about that ... nary a snipe nor a bee-bee stacker either ...

Worse than that ... it turns out that heavy (and in my case that is true in move than one sense) computer geeks are probably in one of the classes of autism called Asperger's Syndrome. We take autism to new levels because when you couple autism with raving intelligence you get Bill Gates ...

Bottom line, I have only met one or two people in my life that liked me upon meeting me for the first time.

Besides, my wife took away my axe (I mean, if you are mentally ill it is not fair when they take away you toys, I mean, how can I be a good axe murderer if I cannot find my axe?) but that means that she might take umbrage at the "poaching" ...

Anyway, one woman in anyones life is very often one too many ... :)

> At least I know I can refer to something in my BOINC client as a complete
> charlie foxtrot and you and Rocky will understand. ;)

Yes, well, I still say we are doing quite well at this time... even with all the problems we are exploring a brand new frontier ... and I am encouraged that they seem to have found another potential bottleneck and are going to try to resturcture and see if that helps.

If anyone is interested, and don't know what I mean by this ... start a new thread and say Paul, pontificate about why you should move things off a database server and possibly create stripe sets and use multiple spindles to distribute seek load ... :)
<p>
For BOINC Documentaion: Click Me!

Hi All,

Again Thank You Paul and Thunder for your very timely replies.
As for why i stay detached from Network is yes I am on dial up. I found that after a power failure (which we have to to often) the network was by default "Enabled"and staying enabled caused problems I addressed in other posts, mainly that everytime a workunit was through it would try to "Upload" without a network connection, it would eventually fail out after several , several, attempts. After a second WU was ready to "Upload", it would try again to upload. This time it caused every WU waiting to be crunched to go to a "ready to Report" status with 0% crunching time. Thus I was losing entire caches. I was explained by I believe John McLeod that a "Sticky Swith" needed to be installed so that the network default would stay as we set it until such a time as individually we change it. This would not hurt those with Always On connection, because they could "Choose" to leave newtwork access enabled. I believe John said that the Developers were aware of this situation, but it would be addressed at a later time.

I am very happy that Boinc is making the progress that it is. I think we can all see better days on the horizon.

And I say "Horizon" because as an Airdale on the USS kitty Hawk, I saw many things on the Horizon's for many years.

It's funny The Airforce is strictly a "Air Outfit". Why is it that The US Navy has a far far more vast fleet of all types of Aircraft, for all types of situations.
I guess it's The Navy thats gets the "Dirt Done" and then the "Pretty Boy's" of the Air Force,in there never dirtied uniforms, come in to say "Boy We Did Good" ! hahahaha

Have A Great Day and A Better Tomorrow!

Regards,

Rocky Cudd
USN

Heh, some days I wish I was one of those 'pretty boys'. :P I'm in combat weather, which sounds cushy... (what's dangerous about being a weather forecaster, right?) What it really means is that I'm almost always in the field with either army aviation assets or worse, mechanized infantry. :P The only redeeming aspect of it is that I get additional pay for it. You should see the looks on these army guys faces when they realize that I don't pull guard duty, don't have to work KP, spend most of my time working at computers (albeit in dark, musty tents) AND get paid extra for it. ;) I tell them it's our "We're sorry you have to put up with the army bonus."

To see what a weather forecasting computer geek looks like when you give him a gun:
http://www.rival2025.net/~thunder/thunder_saw.jpg

(OFF SUBJECT)

Ok Thunder,
I did not think the Air Force knew what Cammies were, you proved me wrong, But I must add, do the Soldiers around you know that those little black knobs all around your helmet are actually transmitters and Recievers of E.T. Signals? I guess thats "Mobile Seti".

haha


Have A Great Day And A Better Tomorrow!

Regards,

Rocky

>http://www.rival2025.net/~thunder/thunder_saw.jpg

Hi All,

Here is the News that I just recieved off of the Seti Boinc News group on Yahoo. It is As Follows:

It has been discovered that a flaw exists, triggered by LHC and/or
Pirates that causes a complete corruption of the 'client_state.xml'
file.

Boinc Dev is aware of the problem right now and working on the fix.

The symptoms are

a) MD5 checksum errors during download / stuck downloads

b) unable to locate file errors

c) Wrong application attempting to run a WU.

d) All projects in your 'client_state.xml' file are overwritten with
the offending / triggering project name.

It is still uncertain how long this race condition has been there.
The condition is caused when you have 2 or more projects and attempt
to 'update' them at nearly the same time.... there is a Mutex spin
lock to prevent 1 from starting before the other has completed. This
Mutex code is flawed. It is hoped Boinc dev will fix it quickly and
get updates to all projects today as the condition is easily
repeatable.



Once fixed, the repair will be to detach from each project and
reattach, you will lose any work in your queue



Regards,

Rocky

PS.

Thanks Paul!

I got you on that one ...

We did a full-up abandon ship drill once. They even stopped the Kitty Hawk dead in the water ...

we did everything up to jumping into the water...

The part I liked best was the one where the navigator told us our position ... he pointed out that the nearest land in the direction of the prevailing wind was only 4,000 some odd miles away and it was an island believed inhabited by cannabals.

When I was still in I ran across one other funny factoid, the navy does not have the most ships, the air force does not have the most planes ... the army has them both beat ...

And I neveer thought cammies made much sense where I saw most air force types, for some reason they stood out from the office decor ...

<p>
For BOINC Documentaion: Click Me!

Hey Paul,
1. your right the Cammies never went with the office. haha.
The worst time I ever had on the Kitty Hawk was about 9 days after reporting, I followed the flight cheif on deck he sent me behind an F-4 Phantom which I did not know was turning at Military position. (I could not hear it).
Nedless to say, it blew my young butt acroos the flight deck, I am trying to grab padeyes to stop. Went over the side and landed in the 4 foot saftey net.
I think they used a gallon of Iodine to cover all my scrapes from the non skid.
This has become quite funny to look back on, but when I get to laughing real hard I just look down at the scars on my legs for a very sobering reality check.

Have a great Night.

Warm Regards,

Rocky