Can't update or reset project. TIMEOUT

Message boards : Number crunching : Can't update or reset project. TIMEOUT
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile woodyrox
Volunteer tester

Send message
Joined: 7 Apr 01
Posts: 34
Credit: 16,069,169
RAC: 0
United States
Message 1304046 - Posted: 9 Nov 2012, 16:37:49 UTC

I mostly run SETI when temperatures get cold and I can use the waste heat to help heat the house. So I'm starting SETI tasks on my computers and am having problems with one machine. Last spring, it was running Lunatics, but when I tried to get tasks, the system said I didn't have a valid SETI app. So I uninstalled Lunatics and installed the original SETI software. I've tried several versions, and they all behave the same. Scheduler requests timeout. Resetting has no apparent effect.

I was able to report completed tasks once, but that's it. I've tried detaching and reattaching. No significant improvement. I'm able to get CUDA-FERMI tasks for the GPU, completed results upload, but I can't report them. I detached, lost a bunch of completed work units re-attached, got some new work units, but again can't report them. All my other computers work fine. In fact, they are real snappy when updating. They are still running the old Lunatics version.

One more thing, in the process of bringing this machine on task, my NVIDIA GT240 graphic card fan failed, so I replaced it with a GT430. The drivers recognized the card & GPU, SETI loaded the appropriate CUDA-Fermi app, and the GPU is crunching tasks and completing without errors. I have yet to get a CPU task, and the CPU hasn't changed. Here's what is logged:

Fri 09 Nov 2012 11:33:00 AM EST Project communication failed: attempting access to reference site
Fri 09 Nov 2012 11:33:00 AM EST SETI@home Scheduler request failed: Timeout was reached
Fri 09 Nov 2012 11:33:01 AM EST Internet access OK - project servers may be temporarily down.


The problem computer is here:

http://setiathome.berkeley.edu/results.php?hostid=6109088

If anyone has any ideas where I should look for problems, help would be greatly appreciated.

Thanks
ID: 1304046 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1304047 - Posted: 9 Nov 2012, 16:40:11 UTC - in response to Message 1304046.  

Most likely not a problem on your end.
There have been server availability problems for the last week or so.
Peruse the Panic Mode thread for more information.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1304047 · Report as offensive
Profile Fred J. Verster
Volunteer tester
Avatar

Send message
Joined: 21 Apr 04
Posts: 3252
Credit: 31,903,643
RAC: 0
Netherlands
Message 1304062 - Posted: 9 Nov 2012, 17:12:28 UTC - in response to Message 1304046.  

I mostly run SETI when temperatures get cold and I can use the waste heat to help heat the house. So I'm starting SETI tasks on my computers and am having problems with one machine. Last spring, it was running Lunatics, but when I tried to get tasks, the system said I didn't have a valid SETI app. So I uninstalled Lunatics and installed the original SETI software. I've tried several versions, and they all behave the same. Scheduler requests timeout. Resetting has no apparent effect.

I was able to report completed tasks once, but that's it. I've tried detaching and reattaching. No significant improvement. I'm able to get CUDA-FERMI tasks for the GPU, completed results upload, but I can't report them. I detached, lost a bunch of completed work units re-attached, got some new work units, but again can't report them. All my other computers work fine. In fact, they are real snappy when updating. They are still running the old Lunatics version.

One more thing, in the process of bringing this machine on task, my NVIDIA GT240 graphic card fan failed, so I replaced it with a GT430. The drivers recognized the card & GPU, SETI loaded the appropriate CUDA-Fermi app, and the GPU is crunching tasks and completing without errors. I have yet to get a CPU task, and the CPU hasn't changed. Here's what is logged:

Fri 09 Nov 2012 11:33:00 AM EST Project communication failed: attempting access to reference site
Fri 09 Nov 2012 11:33:00 AM EST SETI@home Scheduler request failed: Timeout was reached
Fri 09 Nov 2012 11:33:01 AM EST Internet access OK - project servers may be temporarily down.


The problem computer is here:

http://setiathome.berkeley.edu/results.php?hostid=6109088

If anyone has any ideas where I should look for problems, help would be greatly appreciated.

Thanks



I see apps for CUDA 2.3 (6.09), also CUDA_FERMI (6.10) and the stock AstroPulse v6 v6.04 (cuda_opencl_100) version.

Since you're using a GT430 (FERMI), the 6.10 (CUDA_FERMI)-app. is the right
choice.

You've aborted all your work, maybe by RE-Attaching SETI, on this host,
(while you're other host with a GT430 works OK), also have Errors on
all CUDA 2.3 (6.09) work.

Install the same driver used by your other GT430 host and check for older and
or driver remnants.
A new install of BOINC may also prove usefull, only 1 CUDA_FERMI (6.10) app.
should be installed. And 1 for AstroPulse work.

TaskManager or ProcessExplorer can be used to see the differences between
the 2 GT430 hosts.

Your host with the 9600GT/GSO GPUs run probably best on CUDA 2.3 (6.09), IMHO.




ID: 1304062 · Report as offensive
Profile Fred E.
Volunteer tester

Send message
Joined: 22 Jul 99
Posts: 768
Credit: 24,140,697
RAC: 0
United States
Message 1304066 - Posted: 9 Nov 2012, 17:20:52 UTC

As the kitty man says, the problem is at the server end and we all see what you see. The project's having serious problems with the server that schedules work. Failed work requests that timeout are common now. Here's a few tips:

1) no need to detach/reattach, won't do any good and you abandoned a lot of tasks that will be hard to replace in this environment.

2) Reporting completed work has a higher probability of acceptance if you go on No New Tasks and don't ask for work. After you get them reported, Allow New Tasks. A work request that is not reporting completions seems to have a little better chance of success.

3) Scheduler fills the most capable device first, up to your cache settings. It won't send cpu work while BOINC is asking for gpu work. Lower your cache settings and you may have better luck. There are temporary task limits in place, so a large cache setting doesn't help anyway. You can also turn off gpu work fetch in your Seti@Home prefs on the website.

4) Are you looking for help to fix the Lunatics install? The installer is not available for download now, but if you have it, a reinstall might work.

5) Have patience. They are working on it.


Another Fred
Support SETI@home when you search the Web with GoodSearch or shop online with GoodShop.
ID: 1304066 · Report as offensive
Profile woodyrox
Volunteer tester

Send message
Joined: 7 Apr 01
Posts: 34
Credit: 16,069,169
RAC: 0
United States
Message 1304638 - Posted: 10 Nov 2012, 21:26:54 UTC - in response to Message 1304066.  

2) Reporting completed work has a higher probability of acceptance if you go on No New Tasks and don't ask for work. After you get them reported, Allow New Tasks. A work request that is not reporting completions seems to have a little better chance of success.


When I turned off new tasks, the update completed instantly. I tried the other tricks and still had no luck getting CPU tasks. Could not find the "gpu fetch" option on the master account preferences page.

I can't find my original copy of the Lunatics installer on my network so don't need help trying to install it until the download becomes available. When I last installed it, I thought it gave about 10% better performance. The Lunatics install also made it possible for me to run 6 tasks simultaneously -- 4 CPU and 2 GPU on another machine... the one with the quadcore and the 9600 GSO video card.

BTW, that 2x 9600 GSO host really isn't. It's a 9600 GSO and a GeForce 8400 GS. I'm hesitant to mess with the drivers because it's not supposed to work unless they're the same graphic cards. I had a spare 8400 PCI card so I plugged it in and it started crunching along with the 9600. That box only has one PCIe slot so I was happy to get another GPU card running in in.

I'll just wait a while and hopefully the server problem gets resolved. None of my other computers were timing out, so I thought it was a problem with that machine.

Thanks everyone for your suggestions.

ID: 1304638 · Report as offensive
Profile Fred E.
Volunteer tester

Send message
Joined: 22 Jul 99
Posts: 768
Credit: 24,140,697
RAC: 0
United States
Message 1304643 - Posted: 10 Nov 2012, 21:39:41 UTC

Could not find the "gpu fetch" option on the master account preferences page.


On the website's SETI@Home project prefs, it's "Use Nvidia GPU" (or ATI). That pref doesn't stop crunching what you have. On the next update, Scheduler will tell BOINC to stop asking for gpu work. I usually keep a browser tab open to that page to remind me to reset it when I get the cpu tasks.

Good luck - work fetch is tough these days.
Another Fred
Support SETI@home when you search the Web with GoodSearch or shop online with GoodShop.
ID: 1304643 · Report as offensive
Profile woodyrox
Volunteer tester

Send message
Joined: 7 Apr 01
Posts: 34
Credit: 16,069,169
RAC: 0
United States
Message 1304952 - Posted: 11 Nov 2012, 15:40:42 UTC - in response to Message 1304638.  
Last modified: 11 Nov 2012, 15:41:50 UTC

Good news, the updates started working on my faulty box last night. Since then, I've received CPU tasks, and completed tasks are reported very quickly.

I'm in need of fixing my Lunatics install on another machine that I'm bringing online. Again, it's complaining about unusable versions. Here's the log:

SETI@home, 2012-11-11 10:11:05, Message from server: Your app_info.xml file doesn't have a usable version of SETI@home Enhanced.
SETI@home, 2012-11-11 10:11:05, Message from server: Your app_info.xml file doesn't have a usable version of AstroPulse v6.
SETI@home, 2012-11-11 10:11:05, Message from server: This computer has finished a daily quota of 1 tasks
SETI@home, 2012-11-11 10:16:10, Sending scheduler request: To fetch work.
SETI@home, 2012-11-11 10:16:10, Requesting new tasks for CPU
SETI@home, 2012-11-11 10:16:18, Scheduler request completed: got 0 new tasks
SETI@home, 2012-11-11 10:16:18, Message from server: No tasks sent
SETI@home, 2012-11-11 10:16:18, Message from server: Your app_info.xml file doesn't have a usable version of SETI@home Enhanced.
SETI@home, 2012-11-11 10:16:18, Message from server: This computer has finished a daily quota of 1 tasks

Here is the content of my app_info.xml file:

<app_info> 
    <app>
        <name>setiathome_enhanced</name>
    </app>
    <file_info>
        <name>AK_v8b_win_SSE3_AMD.exe</name>
        <executable/>
    </file_info>
    <app_version>
        <app_name>setiathome_enhanced</app_name>
        <version_num>603</version_num>
        <file_ref>
           <file_name>AK_v8b_win_SSE3_AMD.exe</file_name>
            <main_program/>
        </file_ref>
    </app_version>
    <app>
        <name>astropulse_v505</name>
    </app>
    <file_info>
        <name>ap_5.05r409_SSE.exe</name>
        <executable/>
    </file_info>
    <app_version>
        <app_name>astropulse_v505</app_name>
        <version_num>505</version_num>
        <file_ref>
            <file_name>ap_5.05r409_SSE.exe</file_name>
            <main_program/>
        </file_ref>
    </app_version>    
    <app>
        <name>setiathome_enhanced</name>
    </app>
    <file_info>
        <name>Lunatics_x38g_win32_cuda32.exe</name>
        <executable/>
    </file_info>
    <file_info>
        <name>cudart32_32_16.dll</name>
        <executable/>
    </file_info>
    <file_info>
        <name>cufft32_32_16.dll</name>
        <executable/>
    </file_info>
    <app_version>
        <app_name>setiathome_enhanced</app_name>
        <version_num>610</version_num>
        <plan_class>cuda_fermi</plan_class>
        <avg_ncpus>0.040000</avg_ncpus>
        <max_ncpus>0.040000</max_ncpus>
        <coproc>
            <type>CUDA</type>
            <count>1</count>
        </coproc>
        <file_ref>
            <file_name>Lunatics_x38g_win32_cuda32.exe</file_name>
            <main_program/>
         </file_ref>
        <file_ref>
	       <file_name>cudart32_32_16.dll</file_name>
        </file_ref>
        <file_ref>
            <file_name>cufft32_32_16.dll</file_name>
        </file_ref>
    </app_version>
    <app_version>
        <app_name>setiathome_enhanced</app_name>
        <version_num>609</version_num>
        <plan_class>cuda23</plan_class>
        <avg_ncpus>0.040000</avg_ncpus>
        <max_ncpus>0.040000</max_ncpus>
        <coproc>
            <type>CUDA</type>
            <count>1</count>
        </coproc>
        <file_ref>
            <file_name>Lunatics_x38g_win32_cuda32.exe</file_name>
            <main_program/>
         </file_ref>
        <file_ref>
	       <file_name>cudart32_32_16.dll</file_name>
        </file_ref>
        <file_ref>
            <file_name>cufft32_32_16.dll</file_name>
        </file_ref>
    </app_version>
    <app_version>
        <app_name>setiathome_enhanced</app_name>
        <version_num>608</version_num>
        <plan_class>cuda</plan_class>
        <avg_ncpus>0.040000</avg_ncpus>
        <max_ncpus>0.040000</max_ncpus>
        <coproc>
            <type>CUDA</type>
            <count>1</count>
        </coproc>
        <file_ref>
            <file_name>Lunatics_x38g_win32_cuda32.exe</file_name>
            <main_program/>
         </file_ref>
        <file_ref>
	       <file_name>cudart32_32_16.dll</file_name>
        </file_ref>
        <file_ref>
            <file_name>cufft32_32_16.dll</file_name>
        </file_ref>
    </app_version>
</app_info> 


Any ideas what the problem might be and how to fix it? I could uninstall Lunatics and start over like I did on the other box. This machine has a quad core CPU and only 1 GPU so it would work with a standard SETI install, albeit at slightly lower performance. If possible, I would like to keep the Lunatics install.

Many thanks in advance.
ID: 1304952 · Report as offensive
Profile woodyrox
Volunteer tester

Send message
Joined: 7 Apr 01
Posts: 34
Credit: 16,069,169
RAC: 0
United States
Message 1304964 - Posted: 11 Nov 2012, 16:24:16 UTC - in response to Message 1304952.  

I compared my app_info.xml file on the failing machine with another flawlessly crunching, AMD 9850 quad core and they are identical. All of the applications listed in the two app_info.xml files also exist in the seti projects folder:

C:\Documents and Settings\All Users\Application Data\BOINC\projects\setiathome.berkeley.edu


So I'm confused why one machine works fine and the other complains about an unusable app version when everything looks identical between the two.

Has anyone seen this before?
ID: 1304964 · Report as offensive
Profile woodyrox
Volunteer tester

Send message
Joined: 7 Apr 01
Posts: 34
Credit: 16,069,169
RAC: 0
United States
Message 1304966 - Posted: 11 Nov 2012, 16:39:35 UTC - in response to Message 1304964.  
Last modified: 11 Nov 2012, 16:39:55 UTC

OK, the astropulse v6 message is probably because that's the only astropulse work that's currently available. The AMD Lunatics install on the functional box isn't getting any Astropulse work either. But it's not complaining about it.

The unusable enhanced app problem I still don't get.

Is it possible to edit the app info file to add a v6 astropulse app and manually download the corresponding app?

I don't want to change the install on my 6-processor box because the SETI app had the gpu + cpu ratio set such that only 5 processors would run simultaneously.

Thanks.
ID: 1304966 · Report as offensive

Message boards : Number crunching : Can't update or reset project. TIMEOUT


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.