Look.....the comms problem that creates the ghosts may or may not be solveable.
It may be a problem created by the bandwidth. Until the servers are totally stable, we will not know.

The solution is, for now.....

Keeping enough server power online to allow them to at least try to keep the resend feature online and send the 'unknown tasks' back to whence they came.

It did not seem to have much of an impact last week on 'tasks in progress'...

Maybe this week it will do better, if enabled.

Two points:
* bandwidth as a source of the problem: isn't there a way to buffer things so that bandwidth is not an issue (assuming that a lack of 'peak' bandwidth is the issue)? Why not do it? (Yes, seti's resource prioritization must altered to do the job.)
* until the servers are totally stable: ahh, have they ever been? But kidding aside, why not throttle the available work or something to make them stable and then focus on solving the ghosting. Perhaps the ghosts go away and that would tell them something. (This theme sounds mildly familiar from the past)

Version in small .zip (1.02) works also on Win98





P.S.
I wonder - do this Graphs for gigabitethernet2_3 show that they enabled only uploads and Ghost resends?:
http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=%2Frouter-interfaces%2Finr-250%2Fgigabitethernet2_3;ranges=d%3Aw;view=octets

---

 


- ALF - "Find out what you don't do well ..... then don't do it!" :)
 

Version in small .zip (1.02) works also on Win98

Good to know, thank you BilBg.

P.S.
I wonder - do this Graphs for gigabitethernet2_3 show that they enabled only uploads and Ghost resends?:

Hard to tell for sure, as far as I can figure Ghost re-sends require scheduler requests/replies to succeed. It would seem that replies have gone AWOL.

What that graph does show is an alarmingly high 'average bits out' which usually corresponds to our uploads and completed work reports.

It was strangely high for a day before the servers came up, went even higher a few hours after the outage officially ended and has remained unusually high ever since.

I would have thought that most, if not all, Work Units would have been uploaded by now so the high levels of traffic are probably occurring because clients keep trying to report their completed work over and over and over again....

EDIT: Probably a good time to suspend network activity until this has been fixed.

Hard to tell for sure, as far as I can figure Ghost re-sends require scheduler requests/replies to succeed. It would seem that replies have gone AWOL.

No, not Replies. The Scheduler is not getting the original Requests. They moved the Scheduler from Anakin to Bane, and one of the Download Servers from Bane to Anakin, and in the process something got bollixed and neither of them is responding to Requests from us.

What that graph does show is an alarmingly high 'average bits out' which usually corresponds to our uploads and completed work reports.

It was strangely high for a day before the servers came up, went even higher a few hours after the outage officially ended and has remained unusually high ever since.

I would have thought that most, if not all, Work Units would have been uploaded by now so the high levels of traffic are probably occurring because clients keep trying to report their completed work over and over and over again....

EDIT: Probably a good time to suspend network activity until this has been fixed.

I concur, and have done so.

---

Donald
Infernal Optimist / Submariner, retired

Possible bug report:

Please check your code for finding the "In progress" tasks -
in browser I see 3 "In progress" tasks
http://setiathome.berkeley.edu/results.php?hostid=3412467&offset=0&show_names=0&state=1

I checked this before and after running GhostDet but it shows:

HostID = 3412467

      8 WU 'In Progress'
      3 WU on board
------------------------
      5 Ghost Work Units

---

 


- ALF - "Find out what you don't do well ..... then don't do it!" :)
 

...
Keeping enough server power online to allow them to at least try to keep the resend feature online and send the 'unknown tasks' back to whence they came.

It did not seem to have much of an impact last week on 'tasks in progress'...

Maybe this week it will do better, if enabled.

The limited resend feature isn't optional, they'd have to roll back to an older version of the Scheduler or patch the current code and rebuild to disable it.

For top hosts it won't help a lot because only 20 can be resent at a time, but that may be enough for most hosts. Every little bit helps.

The full resend feature is an option, if it were enabled but the Feeder disabled so no new work could be sent, perhaps the database could stand the load. It seems worth trying to me at the Friday startup, but I wouldn't be surprised if they had to stop the experiment fairly quickly.

                                                               Joe

Well, something sure mucked up the works........was that it?

---

"Freedom is just Chaos, with better lighting." Alan Dean Foster

Possible bug report:

Please check your code for finding the "In progress" tasks -
in browser I see 3 "In progress" tasks
http://setiathome.berkeley.edu/results.php?hostid=3412467&offset=0&show_names=0&state=1

I checked this before and after running GhostDet but it shows:

HostID = 3412467

      8 WU 'In Progress'
      3 WU on board
------------------------
      5 Ghost Work Units

Well spotted BilBg, you certainly did find a bug, thank you.

I have fixed it. The 'In Progress' count will be 100% accurate when I release the next version.

In the meantime GhostDet might sometimes report that you have up to 8 more 'In Progress' Work Units than is actually the case, sorry about that.

BilBg's example demonstrated a fault in my binary search algorithm's safety mechanism which was intended to prevent endless loops.

It produced exactly the wrong set of circumstances by getting the 'In Progress' page at offset 4, thus the binary search maximum value of 8 was not sufficiently higher than its minimum value, this triggered the safety mechanism causing it to overestimate. Doh!

You are still mucking about with that app, eh?

To what end? Just so somebody can boast about how many ghost tasks the server has allotted to them?

It does not do anything to solve the problem.

Sorry if I am less than amused. And I DO respect anybody that can program.

Just don't see the point in your efforts.

I hope you can respect my opinion on this.......

---

"Freedom is just Chaos, with better lighting." Alan Dean Foster

Says I have 10 ghosts, am I missing something, what do I do now that I know I have them?

I wouldn't do anything, they will be crunched by others once they time out.

Alternatively you could trigger the resending of those by intentionally reporting one result twice by some client_state.xml editing.

---

Says I have 10 ghosts, am I missing something, what do I do now that I know I have them?

I wouldn't do anything, they will be crunched by others once they time out.

Alternatively you could trigger the resending of those by intentionally reporting one result twice by some client_state.xml editing.

I got my Netbook's Ghosts resent (last week) by repeatily starting a request for work, waiting a couple of seconds, then disabled wireless (Function F9 on my Notebook)

It didn't work so well when tried on my Desktop, i just got more Ghosts. :-(

Claggy

Since every client that has requested work in the last 24 hours has failed, does that mean all these clients are primed to get the ghosts resent when comms do succeed?

---

Boinc....Boinc....Boinc....Boinc....

Since every client that has requested work in the last 24 hours has failed, does that mean all these clients are primed to get the ghosts resent when comms do succeed?

Depends if they managed to report any work, neithier your computers or mine have managed to report anything since the 21st, other wise last contact time/date would have updated.

Claggy

I got my Netbook's Ghosts resent (last week) by repeatily starting a request for work, waiting a couple of seconds, then disabled wireless (Function F9 on my Notebook)

It didn't work so well when tried on my Desktop, i just got more Ghosts. :-(

Well, you should NOT request new work on the first report (the intentionally unsuccessful one), as that will for sure generate more ghosts unless the message from server is something like "No work available". Only on the second report you should request work.

---

I got my Netbook's Ghosts resent (last week) by repeatily starting a request for work, waiting a couple of seconds, then disabled wireless (Function F9 on my Notebook)

It didn't work so well when tried on my Desktop, i just got more Ghosts. :-(

Well, you should NOT request new work on the first report (the intentionally unsuccessful one), as that will for sure generate more ghosts unless the message from server is something like "No work available". Only on the second report you should request work.

Good point, lesson learnt.

Claggy

You are still mucking about with that app, eh?
To what end? Just so somebody can boast about how many ghost tasks the server has allotted to them?
It does not do anything to solve the problem.
Sorry if I am less than amused. And I DO respect anybody that can program.
Just don't see the point in your efforts.
I hope you can respect my opinion on this.......

In truth, it is slightly disappointing to see your semi condescending vibe in this thread.. More specifically toward the ghost work unit issue in general.

For someone who has done as much for the project as you have, I'd almost expect you to be leading the charge when it comes to dealing with ghost work units. Especially, knowing that they are creating storage pains and other issues for the project (see Matt's last tech post). In other words, yes, it does matter.

Instead you've given off the I don’t care: "So, it's really a problem for the Seti boyz to unravel.” As if there’s nothing anyone else can do to help the situation.

In reality, there are small simple steps we the users can take to help. No, it may not completely solve the issue, but every little bit may help (sound familiar?) till a smoother resend feature is implemented.


If this tool helps someone figure out they have an excess of ghost work units assigned to their rigs and prompts them to free them. That’ll be one person taking a small step in a beneficial direction.

-[ snip ]-

Its very easy to use so I thought I'd share it with my fellow SETIzens.

If you are interested, you can download it from http://netfolder.in/folder.php?folder_id=iPe7n5V, there you will find 2 Zip files.

-[ snip ]-

Greetings,

I must say, making people jump through hoops and leap over hurdles, just to download a single file, is NOT cool. Having to wait for countdown timers to reach 0 (zero), just to click a link in the end, to finally download the file, SUX! I almost gave up on the download, but curiosity won out. :) I suggest dumping that file hosting site and putting your files somewhere else for instant downloading and not make people wait so long to get the file of their choice.

Oh, and BTW, the "Free download" is NOT slower than "PREMIUM download". I downloaded "free" in less than a second, not 2 minutes as that file hosting site says it will take. Why get the premium if the free is just as fast or faster? Just in case that would be used as an argument against the hoops and hurdles above. BTW, is Netload a pay-per-download file hosting site? Just wondering since one has to "join" to get "faster" downloads. ;)

Just an observation... :)

Keep on BOINCing...! :)

[edit] Pay-per-download question answered after going back to the site... [/edit]

---

CAPT Siran d'Vel'nahr - L L & P _\\//
Winders 11 OS? "What a piece of junk!" - L. Skywalker
"Logic is the cement of our civilization with which we ascend from chaos using reason as our guide." - T'Plana-hath

I've mirrored the 2 files for direct download for the interested :)

GhostDet_v102.zip

GhostDet_Installer_v102.zip

I must say, making people jump through hoops and leap over hurdles, just to download a single file, is NOT cool. Having to wait for countdown timers to reach 0 (zero), just to click a link in the end, to finally download the file, SUX! I almost gave up on the download, but curiosity won out. :) I suggest dumping that file hosting site and putting your files somewhere else for instant downloading and not make people wait so long to get the file of their choice.

Agreed, sorry about that Siran d'Vel'nahr, I was in a rush and chose an annoying host. Later on I did re-upload it to a far better hosting service http://www.mediafire.com/?yyy64lh5953d7.

I've mirrored the 2 files for direct download for the interested :)

Thanks JohnDK much appreciated, you mirroring the files will make it easier for those SETIzens that would like to check it out :)

My......less than enthusiastic attitude towards this was explained in my first post here.......

Being able to look at ghosts does not make then go away......

And, as I said, I am not one who is willing to make then vanish with a detach only to get many more when trying to refill the cache.........

Which does not appear to be happening this weekend anyway.....for some reason.
So, I feel very vindicated now for not having done so. And yes, I am one of the project's biggest poster children, so I realize many look up to what I say.

I am not dissing your effort....I am only saying it makes little difference to the project.

There are reasons for the things that the kittyman does and does not do. Empty kibble bowls do not feed crunchers very well.

Anybody else feeling sheepish about now?

---

"Freedom is just Chaos, with better lighting." Alan Dean Foster

I don't see the big deal, yes it doesn't exactly solve anything, but people can see how many ghosts they have and detach/attach if they feel it's a problem.