Application: Ghost Detector - find out how big your Ghost Army is

Message boards : Number crunching : Application: Ghost Detector - find out how big your Ghost Army is

Previous · 1 · 2 · 3 · 4 · 5 . . . 7 · Next

Author	Message
Geek@Play Volunteer tester Send message Joined: 31 Jul 01 Posts: 2414 Credit: 79,016,596 RAC: 0	Message 1035363 - Posted: 23 Sep 2010, 21:18:57 UTC - in response to Message 1035354. Last modified: 23 Sep 2010, 21:19:30 UTC
	Not interested in which ones are ghosts, only the total of work units "in progress" ____________ Boinc....Boinc....Boinc....Boinc....
	ID: 1035363 ·

bj Send message Joined: 11 Oct 00 Posts: 131 Credit: 47,276,221 RAC: 79,492	Message 1035402 - Posted: 23 Sep 2010, 23:22:47 UTC
	hey.......great program Installed and ran on my five computers. Knew that I had ghost units but not how many. XP1 - 0 XP2 - 28 XP3 - 1 VISTA - 146 WIN7 - 144 Thanks for the program bj ____________
	ID: 1035402 ·

PhonAcq Send message Joined: 14 Apr 01 Posts: 1620 Credit: 20,858,527 RAC: 4,474	Message 1035406 - Posted: 23 Sep 2010, 23:43:36 UTC
	OK. Just checked and I have 190 out of 750 ghosts, or about 25%; I'm running without a GPU. So everybody agrees that Berkeley has a problem, that manifests in as much as 25% of correctable inefficiency (for lack of a better word). But having done that, I'll remove the application to remove the temptation to use it, given that there is nothing I can do about ghosts, except fret. It would be nice if Dr. A(?) or whoever would fix the software structure to get back some server resources. But I suspect such a thought isn't the first, or uniquely mine regarding project.
	ID: 1035406 ·

Bill G Send message Joined: 1 Jun 01 Posts: 241 Credit: 19,000,550 RAC: 24,361	Message 1035412 - Posted: 24 Sep 2010, 0:02:04 UTC - in response to Message 1035340.
	Says I have 10 ghosts, am I missing something, what do I do now that I know I have them? I wouldn't do anything, they will be crunched by others once they time out. Thanks, that is what I will do. ____________
	ID: 1035412 ·

SciManStev Volunteer tester Send message Joined: 20 Jun 99 Posts: 4291 Credit: 62,697,993 RAC: 73,347	Message 1035422 - Posted: 24 Sep 2010, 0:40:06 UTC - in response to Message 1035412.
	According to Matt's post, they are aware of the ghosts, and are seeing what can be done. Steve ____________ Warning, addicted to SETI crunching! Crunching as a member of GPU Users Group. GPUUG Website
	ID: 1035422 ·

BilBg Send message Joined: 27 May 07 Posts: 2019 Credit: 3,010,788 RAC: 6,200	Message 1035441 - Posted: 24 Sep 2010, 2:49:07 UTC - in response to Message 1035286. Last modified: 24 Sep 2010, 3:36:02 UTC
	Version in small .zip (1.02) works on Vista 32 P.S. Version in installer is 1.00 (extracted GhostDet.exe from GhostDet.CAB and checked) Version in small .zip is 1.02 I think this site is most easy for upload and download (but the link changes for every upload): http://www.datafilehost.com/ ____________ - ALF - "Find out what you don't do well ..... then don't do it!" :)
	ID: 1035441 ·

[DPC] hansR Send message Joined: 14 Jul 00 Posts: 35 Credit: 77,623,429 RAC: 59,208	Message 1035470 - Posted: 24 Sep 2010, 5:07:16 UTC Last modified: 24 Sep 2010, 5:08:53 UTC
	It works for me aswell and i'm shocked! HostID = 5382716 3.078 WU 'In Progress' 821 WU on board ------------------------ 2.257 Ghost Work Units That is almost 75 % ____________
	ID: 1035470 ·

msattler Volunteer tester Send message Joined: 9 Jul 00 Posts: 34127 Credit: 342,586,314 RAC: 538,849	Message 1035486 - Posted: 24 Sep 2010, 6:03:06 UTC - in response to Message 1035335. Last modified: 24 Sep 2010, 6:03:54 UTC
	Do know that this application uses page scraping, which adds extra load to the database, which in turn could lead to it crashing. This can lead to your IP address being blocked if you use it too often. If the app is hard on the database, I am not going to pursue it any further. It was more out of curiosity for me than anything, as I am not about to detach/reattach to get rid of whatever ghosts I might have. The problem has been documented well enough by some here, the boyz are painfully aware of it, and Matt has indicated they are going to look into it. They know that unchecked, it is inflating their storage requirements. That's good enough for me at this point. ____________ ****** "Ask not, what your kitty can do for you. Ask what you can do for your kitty." As it is kitten, so shall it be done.
	ID: 1035486 ·

BilBg Send message Joined: 27 May 07 Posts: 2019 Credit: 3,010,788 RAC: 6,200	Message 1035491 - Posted: 24 Sep 2010, 6:42:56 UTC - in response to Message 1035486. Last modified: 24 Sep 2010, 6:57:22 UTC
	If the app is hard on the database, I am not going to pursue it any further. It is only as "hard" as if you look at 5-10 of your result pages - it tries to find the last of the "In progress" pages by (smart) dividing the range (offset=) by 2 Also it has minimum 6 hours delay before the next test (try to press "Detect ..." button again) It will be more "hard on the database" if you manually look ("Next 20", page by page) to get to the end of your "In progress" pages. ____________ - ALF - "Find out what you don't do well ..... then don't do it!" :)
	ID: 1035491 ·

msattler Volunteer tester Send message Joined: 9 Jul 00 Posts: 34127 Credit: 342,586,314 RAC: 538,849	Message 1035493 - Posted: 24 Sep 2010, 6:47:40 UTC - in response to Message 1035491.
	If the app is hard on the database, I am not going to pursue it any further. It is only as "hard" as if you look at 5-10 of your result pages - it tries to find the last of the "In progress" pages by (smart) dividing the range (offset=) by 2 It will be more "hard on the database" if you manually look (page by page) to get to the end of your "In progress" pages. Well, that may be......no matter as far as I am concerned. Those who wish to may continue to play with it. I simply have no reason to. Whatever ghosts I may have out there shall remain there until they time out. Unless the resend feature is still active when the servers come back up tomorrow and start sending them back to me in response to work requests. Which would probably be the best solution to the problem that we have so far. ____________ ****** "Ask not, what your kitty can do for you. Ask what you can do for your kitty." As it is kitten, so shall it be done.
	ID: 1035493 ·

ded1o1 Send message Joined: 29 Sep 07 Posts: 68 Credit: 10,834,919 RAC: 0	Message 1035515 - Posted: 24 Sep 2010, 8:45:05 UTC - in response to Message 1035331.
	Says I have 10 ghosts, am I missing something, what do I do now that I know I have them? Sadly Bill G, there is nothing it can do about the Ghosts it detects. It simply notifies you on how many you have. For those that have a huge army it simply lets them know when/if it is worth running down their cache and doing a detach/re-attach to help the project out by freeing up some server disk space. Do know that this application uses page scraping, which adds extra load to the database, which in turn could lead to it crashing. This can lead to your IP address being blocked if you use it too often. Agreed Ageless, using it to detect Ghosts every few minutes would cause load on the server, perhaps be detectable and could result in IP banning. However, it places far less load on the server than checking by hand would. That is why it will not let you check again manually or automatically (which was only implemented for those with remote hosts) until at least 6 hours have passed, it defaults to 24 hours between checks. In reality, checking once every few days is more than enough and there is no point checking during the 3 day outage. Does it add extra load in comparison to when you check for ghosts manually? Far less than checking by hand JohnDK. Yes, as your browser doesn't check all the pages of your work simultaneously. This app does. Are you talking about the Ghost Detector here Ageless? It does not "check all pages simultaneously", it is very efficient and only needs to request 6 to 10 pages to once every few days achieve its goal. It also enforces a 6 hour minimum refresh interval, people not using this program who check by hand can do it every few minutes if they are so inclined. This routine would be very kind to the data base compared to a manual search. Thanks Geek@Play. hey.......great program Installed and ran on my five computers. Knew that I had ghost units but not how many. Thanks bj, glad it could help you out. Version in installer is 1.00 (extracted GhostDet.exe from GhostDet.CAB and checked) Version in small .zip is 1.02 I think this site is most easy for upload and download (but the link changes for every upload): http://www.datafilehost.com/ Cheers BilBg, good to know it works with Vista 32. Thanks also for the host suggestion, I'll check it out. Will also update the app version in the installer to v1.02 on both hosts now that I know it works, this will make it easier and save people from having to Download both. 2,257 Ghost Work Units Wow, that's one big ghost army you have there [DPC] hansR.
	ID: 1035515 ·

msattler Volunteer tester Send message Joined: 9 Jul 00 Posts: 34127 Credit: 342,586,314 RAC: 538,849	Message 1035516 - Posted: 24 Sep 2010, 8:50:10 UTC Last modified: 24 Sep 2010, 9:00:29 UTC
	Does it really matter??? Not to us users, it doesn't...... Whatever we have on our computers is what we have to crunch. Whatever the servers have showing as assigned to us matters only to the Seti database storage..... So, it's really a problem for the Seti boyz to unravel. Our knowing that we have XXX number of ghosts on the system does not mean much, unless you are willing to run your cache down to zippo and do the detach/reattach dance to set them free, which I am not. ____________ ****** "Ask not, what your kitty can do for you. Ask what you can do for your kitty." As it is kitten, so shall it be done.
	ID: 1035516 ·

PhonAcq Send message Joined: 14 Apr 01 Posts: 1620 Credit: 20,858,527 RAC: 4,474	Message 1035549 - Posted: 24 Sep 2010, 12:27:47 UTC - in response to Message 1035516.
	@msattler, I tried to imply in my response below that it may indeed matter to us. With a 25% ghost contingient army, the servers are having their resources wasted. Unfortunately, Berkeley's excuse for many problems seems to me that they need to acquire more/better servers. I won't debate that more/better hardware helps the project. But, aren't they squandering the volunteer's money and enthusiasm by not fixing this ghost problem, which has been around for years and seems to me to be an architectural thing. If I had a vote, which I don't, I would say that they can't use the recent donations on a new server until the ghost problem was certified solved. A solution would help the project (other projects too, I assume) by using the existing resources more effectively and it possibly may change their perspective on what kind of new server hardware is needed. -Don Quixote.
	ID: 1035549 ·

MadMaC Volunteer tester Send message Joined: 4 Apr 01 Posts: 201 Credit: 44,091,395 RAC: 54,520	Message 1035554 - Posted: 24 Sep 2010, 12:59:25 UTC
	Im sure they will get it sorted eventually, apart from their server resources, there is no real harm done Looks like I have a few, just keep on crunching, they will disappear eventually :-) ____________
	ID: 1035554 ·

perryjay Send message Joined: 20 Aug 02 Posts: 3359 Credit: 9,617,410 RAC: 9,985	Message 1035565 - Posted: 24 Sep 2010, 13:24:59 UTC - in response to Message 1035549.
	Unfortunately, Berkeley's excuse for many problems seems to me that they need to acquire more/better servers. They do have a way to "resend lost tasks" but it is extremely hard on their existing servers so they haven't run it. They have come up with a modified version they have implemented now. I'm not sure exactly what triggers it but I'm sure Joe or someone will explain what I'm referring to soon. But anyway, the point I was trying to make is that it is not an excuse, it is a fact they need more and better servers. The old ones in use now just can't handle the strain. ____________ PROUD MEMBER OF Team Starfire World BOINC
	ID: 1035565 ·

SciManStev Volunteer tester Send message Joined: 20 Jun 99 Posts: 4291 Credit: 62,697,993 RAC: 73,347	Message 1035567 - Posted: 24 Sep 2010, 13:30:18 UTC
	What I have seen with ghosts is that they hinder my ability to load up for an outage. This is true when limits are in place. My rig will be requesting work, and the servers say I have too much already, even though what I have on hand is below the limits. Steve ____________ Warning, addicted to SETI crunching! Crunching as a member of GPU Users Group. GPUUG Website
	ID: 1035567 ·

PhonAcq Send message Joined: 14 Apr 01 Posts: 1620 Credit: 20,858,527 RAC: 4,474	Message 1035584 - Posted: 24 Sep 2010, 14:05:58 UTC - in response to Message 1035554.
	Yes there is harm done, madmac; they are holding in your case about 5K wu's that will need to time out before being issued. This takes disk and database space for sure. Multiply that by some large number of similar users, and you may see the issue. Perryjay: A resend lost tasks function is a bandaid. Get rid of the problem creating lost tasks, or the bulk of them, is the proper approach, don't you think? The bandaid adds loading, to fix a problem that shouldn't be there. I suspect an actual fix would require some proper software engineering, however. Perhaps they aren't staffed for this.
	ID: 1035584 ·

msattler Volunteer tester Send message Joined: 9 Jul 00 Posts: 34127 Credit: 342,586,314 RAC: 538,849	Message 1035596 - Posted: 24 Sep 2010, 14:26:42 UTC - in response to Message 1035584.
	Yes there is harm done, madmac; they are holding in your case about 5K wu's that will need to time out before being issued. This takes disk and database space for sure. Multiply that by some large number of similar users, and you may see the issue. Perryjay: A resend lost tasks function is a bandaid. Get rid of the problem creating lost tasks, or the bulk of them, is the proper approach, don't you think? The bandaid adds loading, to fix a problem that shouldn't be there. I suspect an actual fix would require some proper software engineering, however. Perhaps they aren't staffed for this. Look.....the comms problem that creates the ghosts may or may not be solveable. It may be a problem created by the bandwidth. Until the servers are totally stable, we will not know. The solution is, for now..... Keeping enough server power online to allow them to at least try to keep the resend feature online and send the 'unknown tasks' back to whence they came. It did not seem to have much of an impact last week on 'tasks in progress'... Maybe this week it will do better, if enabled. ____________ ****** "Ask not, what your kitty can do for you. Ask what you can do for your kitty." As it is kitten, so shall it be done.
	ID: 1035596 ·

ded1o1 Send message Joined: 29 Sep 07 Posts: 68 Credit: 10,834,919 RAC: 0	Message 1035599 - Posted: 24 Sep 2010, 14:32:21 UTC - in response to Message 1035554.
	Looks like I have a few, just keep on crunching, they will disappear eventually :-) You surely do have a few there MadMaC :) I see you are using the 1.00 version which sadly has a bug or two and lacks some of the functionality of the 1.02 version. The worst bug causes it to over-estimate the number of Work Units it thinks you have On Board if you crunch for more than one project. Please consider updating, the link is http://www.mediafire.com/?yyy64lh5953d7, all you need to do is download the smaller Zip file then extract it into your install folder.
	ID: 1035599 ·

Josef W. Segur Volunteer developer Volunteer tester Send message Joined: 30 Oct 99 Posts: 3931 Credit: 934,263 RAC: 14	Message 1035634 - Posted: 24 Sep 2010, 15:27:02 UTC - in response to Message 1035596.
	... Keeping enough server power online to allow them to at least try to keep the resend feature online and send the 'unknown tasks' back to whence they came. It did not seem to have much of an impact last week on 'tasks in progress'... Maybe this week it will do better, if enabled. The limited resend feature isn't optional, they'd have to roll back to an older version of the Scheduler or patch the current code and rebuild to disable it. For top hosts it won't help a lot because only 20 can be resent at a time, but that may be enough for most hosts. Every little bit helps. The full resend feature is an option, if it were enabled but the Feeder disabled so no new work could be sent, perhaps the database could stand the load. It seems worth trying to me at the Friday startup, but I wouldn't be surprised if they had to stop the experiment fairly quickly. Joe
	ID: 1035634 ·

Previous · 1 · 2 · 3 · 4 · 5 . . . 7 · Next

Message boards : Number crunching : Application: Ghost Detector - find out how big your Ghost Army is