Message boards :
Number crunching :
disk usage
Message board moderation
Author | Message |
---|---|
David S Send message Joined: 4 Oct 99 Posts: 18352 Credit: 27,761,924 RAC: 12 |
I was rather surprised this morning when both Einstein and Seti told me that one of my rigs had no tasks in progress. Checking the messages, they are both saying I don't have enough room on my hard drive. It's a rather small drive, but I don't store a lot on it and don't use the machine for anything other than BOINC, MagicJack, and my radioreference.com feed. Do Boinc projects ever leave stuff on hard drives that they don't need any more? Is there a way I can clean it out? Or should I be looking elsewhere for a culprit? David Sitting on my butt while others boldly go, Waiting for a message from a small furry creature from Alpha Centauri. |
kittyman Send message Joined: 9 Jul 00 Posts: 51468 Credit: 1,018,363,574 RAC: 1,004 |
I was rather surprised this morning when both Einstein and Seti told me that one of my rigs had no tasks in progress. Checking the messages, they are both saying I don't have enough room on my hard drive. It's a rather small drive, but I don't store a lot on it and don't use the machine for anything other than BOINC, MagicJack, and my radioreference.com feed. In your Boinc manager, click on the 'disk' tab. I believe Einstein is notorious for leaving behind lots of files. "Freedom is just Chaos, with better lighting." Alan Dean Foster |
Mike Send message Joined: 17 Feb 01 Posts: 34253 Credit: 79,922,639 RAC: 80 |
You are totally right Mark. Einstein leaves a lot on the hard drive. With each crime and every kindness we birth our future. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
You are totally right Mark. Yes, that's a deliberate feature called "locality scheduling". You can get a lot of different workunits out of the same set of data files. Once you've downloaded them once, you may not need to download any more for several days. But if you just delete the files, BOINC will download them again at the next restart. Search the Einstein forums (or ask Gary Roberts) how to clean your client_state.xml file so you don't waste that download bandwidth. |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
E@H uses a big 8MB or more WU, so if you have a large cache (10 days for example) it will use a lot of HDD space. If you run both projects at the same time the best thing to do is use a small days cache (anyway the limit of 100WU still on SETI) to avoid that. I use 1 day cache on my hosts, that allow SETI DL all the 100WU and keep the number of E@H WU in a confortable margin. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
Note that Einstein has different types of WU, same as we have MB and AP. The WUs for GPUS, 'BRP4' (Binary Radio Pulsar) use 'one time only' data files - a big download (although they've compressed them recently, so less than 16 MB per WU to download), but clean up after themselves. You don't need a big cache at Einstein, they have reliable servers. It's the CPU tasks (Gravitational Wave S6 LineVeto search) which hang on to the data files for re-use. |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
It doesn't just leave a lot on the hard drive, it's keeping the files for when you're returning to running a task with similar sub-sets of data, do you don't have to download all that data again. Don't forget that Einstein uses Locality Scheduling which is to minimize the amount of data transfer to hosts by preferentially sending jobs to hosts that already have some or all of the input files required by those jobs. |
HAL9000 Send message Joined: 11 Sep 99 Posts: 6534 Credit: 196,805,888 RAC: 57 |
Sometimes I do find stray files from tasks that BOINC has run & for whatever reason not deleted once finished. Normally I only look on machines when they fall back to one of their backup projects. At most it only turns out to be a few tasks. I would probably just check the size of the BOINC folder on that drive & see if it seems larger than expected. SETI@home classic workunits: 93,865 CPU time: 863,447 hours Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[ |
MarkJ Send message Joined: 17 Feb 08 Posts: 1139 Credit: 80,854,192 RAC: 5 |
The quickest and easiest way to clean up Einstein is to detach and reattach (or as its now known remove and then add). Quite often they have moved through files but won't clean up until you do the above or the entire run finishes. BOINC blog |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
Quite often they have moved through files but won't clean up until you do the above or the entire run finishes. Quite often? Weird then that I have never have seen this happen in all the years I've run Einstein. Got links to all the many threads on their forums where you and others complain about it and moderators/admins answer that they know it happens quite often? Or is the Quite Often just a personal feeling (because you don't understand LS)? Edit: it may have happened once, when they switched from the S5 search to the S6 search, but since all searches since then have all been a form or continuation of the S6 search, the data files have been the same. Thus no need to remove them when e.g. S6Bucket stops and S6LV starts. |
MarkJ Send message Joined: 17 Feb 08 Posts: 1139 Credit: 80,854,192 RAC: 5 |
Quite often they have moved through files but won't clean up until you do the above or the entire run finishes. The WU are for a particular frequency which as far as I understand they work their way up the band. Even though they may be at the upper end of the band the data files for the lower ones will still be left on your machine. As I said these would not normally be deleted unless the search finished or the user were to detach/reattach. While there may be no need to remove them the question was asked how to. I don't need to remove them I have plenty of disk space. I understand how locality scheduling works so I have no reason to complain about this, I was merely answering the question. BOINC blog |
rob smith Send message Joined: 7 Mar 03 Posts: 22161 Credit: 416,307,556 RAC: 380 |
Sorry to say Ageless Einsein is still "bad" at not clearing up after itself. Particularly so when it suffers time-outs. One cruncher on which I ran Einstein as a "priority 0" back up had over 2Gb of time-out files, and another has over 200Mb of them. Both these were after all visible work had completed, with "No New Tasks" set and so should have been down to the barest minimum, not hundreds of Mb of "rubbish" left lying around. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
SciManStev Send message Joined: 20 Jun 99 Posts: 6651 Credit: 121,090,076 RAC: 0 |
This thread prompted me to look at my Einstein work. I had completed all my tasks a while back, but had about 6.5 Gig of files still on my rig. I detached, and reattached, and they cleared. I don't know what they were, and I have plenty of hard drive space, so I really didn't care. I was surprised that there was 6.5 Gig. Steve Warning, addicted to SETI crunching! Crunching as a member of GPU Users Group. GPUUG Website |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
I take a look at my E@H data and nothing like this noticed. Maybe something else is in place there. |
Wedge009 Send message Joined: 3 Apr 99 Posts: 451 Credit: 431,396,357 RAC: 553 |
Very rarely, I've seen a scheduler request to the Einstein@Home servers result in a 'request from the server' to delete some data files, but they will only be very old ones. In general, they seem to keep data around for future search runs. They also seem very reluctant to delete old application versions. For the record, I currently have 1.53 GiB in my Einstein directory accumulated since May 2012, compared with 213.61 MiB for SETI@home. I don't process Einstein WUs very much. Soli Deo Gloria |
David S Send message Joined: 4 Oct 99 Posts: 18352 Credit: 27,761,924 RAC: 12 |
I detached and reattached Einstein, with no noticeable difference in the space used on my drive. It also did not solve my problem. It turns out that, despite my earlier statement, I am storing a lot of files on it. I identified some I could safely delete and did so, then clicked Update for both projects. Seti deferred me and opportunistic Einstein downloaded over 100 tasks. Seti eventually got 100 as well. David Sitting on my butt while others boldly go, Waiting for a message from a small furry creature from Alpha Centauri. |
kittyman Send message Joined: 9 Jul 00 Posts: 51468 Credit: 1,018,363,574 RAC: 1,004 |
I detached and reattached Einstein, with no noticeable difference in the space used on my drive. It also did not solve my problem. Don't forget to empty the trash bin if you have not already done so. I also usually find a number of good sized log and dump files that Windows writes to disk after a crash. You can turn this off. I used to have a nifty little program that would graphically display the entire hard drive disk usage, but I cannot find it now. "Freedom is just Chaos, with better lighting." Alan Dean Foster |
Highlander Send message Joined: 5 Oct 99 Posts: 167 Credit: 37,987,668 RAC: 16 |
You mean this one? TreeSize Free Portable -> http://portableapps.com/apps/utilities/treesize-free-portable - Performance is not a simple linear function of the number of CPUs you throw at the problem. - |
David S Send message Joined: 4 Oct 99 Posts: 18352 Credit: 27,761,924 RAC: 12 |
Don't forget to empty the trash bin if you have not already done so. That was the first and last thing I did. I also usually find a number of good sized log and dump files that Windows writes to disk after a crash. You can turn this off. I also ran disk cleanup, but it didn't make a lot of difference either. David Sitting on my butt while others boldly go, Waiting for a message from a small furry creature from Alpha Centauri. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.