Panic Mode On (109) Server Problems?

Message boards : Number crunching : Panic Mode On (109) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 36 · Next

AuthorMessage
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1907364 - Posted: 16 Dec 2017, 1:59:15 UTC
Last modified: 16 Dec 2017, 2:14:51 UTC

The more I read through some of the code, the more I realize the code is not keeping up with current hardware and simply punts when it can't handle the hardware.

For example in the benchmark.cpp module, if you complete the benchmark in less than 2 seconds .... punt .... benchmark is invalidated and you get a default benchmark that is no way representative of your processing capabilities. The problem with this is the benchmark is used to directly calculate things like p_fpops for example.

// If a benchmark is nonzero, keep it. Otherwise use default value
//
void CLIENT_STATE::cpu_benchmarks_set_defaults() {
if (!host_info.p_fpops) host_info.p_fpops = DEFAULT_FPOPS;
if (!host_info.p_iops) host_info.p_iops = DEFAULT_IOPS;
if (!host_info.p_membw) host_info.p_membw = DEFAULT_MEMBW;
if (!host_info.m_cache) host_info.m_cache = DEFAULT_CACHE;


and this one particularly is alarming.

    // Windows: do integer benchmark only on CPU zero.
    // There's a mysterious bug/problem that gives wildly
    // differing benchmarks on multi-CPU and multi-core machines,
    // if you use all the CPUs at once.
    //



What with all the latest processors being multi-cpu and multi-core and running north of 5 Ghz for Intel cpu's for example, how can we trust that BOINC is utilizing our hardware correctly.


Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1907364 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11416
Credit: 29,581,041
RAC: 66
United States
Message 1907373 - Posted: 16 Dec 2017, 2:34:59 UTC - in response to Message 1907364.  

how can we trust that BOINC is utilizing our hardware correctly.

If the tasks are validating then Boinc is using the hardware correctly. Boinc may not and is probably not using the hardware efficiently.
ID: 1907373 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13854
Credit: 208,696,464
RAC: 304
Australia
Message 1907376 - Posted: 16 Dec 2017, 2:43:43 UTC

I just lost all of what I was going to post.

In short, Dec 28 2016 there was the weekly outage, and that's when all the issues began.


ReiAyanami Posted: 29 Dec 2016, 21:57:42 UTC
No tasks available?

Bruce Posted: 30 Dec 2016, 3:34:11 UTC
Is anybody getting any work? I have not received any downloads since about 1:30pm Eastern Standard Time. Nothing is stuck or suspended, but I am just about of work units. Do we need to make any changes at our end because of the changes in the encryption? Just wondering if something wrong at my end.

Brent Norman Posted: 30 Dec 2016, 4:11:44 UTC
I seen a problem where I had to tick off SETI@home v8 "YES" to get work on my Ubuntu box, even though it had Accept Other Work and was working fine before.

Bruce Posted: 30 Dec 2016, 5:53:19 UTC
Thanks Brent. Saying yes to SETI@Home v8 fixed the problem and started the work flowing. I wonder if they might have broken the Accept Other Work switch during the outage?
Anyway it works now.

ReiAyanami Posted: 30 Dec 2016
Thank you, Brent.
It fixed my problem.

etc, etc, etc.

kittyman Posted: 30 Dec 2016, 18:09:18 UTC
Eric has responded, and he is looking at things.
I can get some sleep now.

kittyman Posted: 31 Dec 2016, 22:33:48 UTC
For what it's worth, the little problem with the v8 preference switch has not been addressed yet. Just turned it off for about 15 minutes, and sure enough, the rigs on that preference stopped getting new work.
Turned it back on, and they started to fill right back up on the next round of work requests.

kittyman Posted: 4 Jan 2017, 17:37:52 UTC
Well, it appears that there has been no fix for the v8 checkbox bug yet.
Tested it again this morning. Unclicked v8, dropped 40 some tasks from cache. Reclicked it and in a couple of rounds of work requests cache bumped right back up.
Have sent Eric and Jeff a reminder message about the situation.

kittyman Posted: 5 Jan 2017, 8:07:05 UTC
Whist I was away at work, Eric replied that he is still trying to figure out the v8 checkbox bug.
Said it looks like it should be working, but obviously is not.


And this is where I found out about the issue of trying to get MB work, without having AP installed.
Grant (SSSF) Posted: 14 Jan 2017, 22:28:03 UTC
Server Status page shows everything's good, but the cache on my main cruncher is running down again. Most requests for work result in "Project has no tasks available" messages and those that do get work don't get enough to make up the developing deficit.


kittyman Posted: 16 Jan 2017, 18:49:36 UTC
Still no problems here with cache. Pendings are up a bit, but within usual levels.
In progress (2600) · Validation pending (5224) · Validation inconclusive (116) · Valid (3870) · Invalid (0) · Error (0)

Although I am sure he is aware of and working on the scheduler bug, I sent Eric another reminder message on it.
I suggested that it might help if he would change the checkbox for 'accept work from other applications' from a simple 'Yes' to 'Hell, Yes'.........................LOL.

Meow.

EDIT:
And Eric replied already.
He has not figured the problem out just yet, and hopes he can get back to working on it Thursday or so.
SOOOOOOOOOOO.....
Those of you having problems are going to have to do your best with workarounds until 'at least' the end of the week.

Couple more weeks, and we will have our 1st anniversary of the strat of this issue.

All info from the Panic Mode On (104) Server Problems? thread.
https://setiathome.berkeley.edu/forum_thread.php?id=80573&sort_style=6&start=0
Grant
Darwin NT
ID: 1907376 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1907388 - Posted: 16 Dec 2017, 3:15:26 UTC

To continue with some of my code reading discoveries. There is code that no gpu app can process faster than 10 times the time it takes a cpu app or it gets thrown out.

The special app has thrown a monkey wrench completely into the works for that criteria. Again, showing the project is not keeping up with the changes in both hardware and software.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1907388 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13854
Credit: 208,696,464
RAC: 304
Australia
Message 1907413 - Posted: 16 Dec 2017, 5:07:37 UTC

The Assimilation, Deletion & Validation backlogs have all finally cleared.
However the Replica continues to fall further & further behind.
Grant
Darwin NT
ID: 1907413 · Report as offensive
Kiska
Volunteer tester

Send message
Joined: 31 Mar 12
Posts: 302
Credit: 3,067,762
RAC: 0
Australia
Message 1907416 - Posted: 16 Dec 2017, 5:13:04 UTC - in response to Message 1907360.  

...the server code will be very close to the master code at https://github.com/BOINC/boinc. The client and manager code is in subdirectories 'client' and 'clientgui', respectively: the bit of the server code we're interested in here is in subdirectory 'sched'.
That was an easy solution. There is no directory https://github.com/BOINC/boinc/client/sched, not that I can find anyway. Just by chance I did open home/tbar/Downloads/boinc-master/client/scheduler_op.cpp and search for Project has no tasks available which has to be somewhere in the Server code since it is printed on the replies. There isn't a Project has no tasks available anywhere in the code, so, not the right code.

Just by another chance I decided to change my preferences to what Grant suggested, AP Yes, MB No, If no work for selected applications is available, accept work from other application Yes
I suddenly stopped receiving MB tasks even though If no work for selected applications is available, accept work from other application? is YES
Perhaps it's just a matter of the Server losing track of just what Tasks it does have available and assuming it doesn't have MB work for a particular Host. By forcing it to look at the available tasks again, it realizes it does have MB tasks, until it again loses track of which tasks it has available and needs to be reminded. Anyway, I should be receiving MB tasks with my current settings, but, I'm Not. I am receiving APs though;
**Omitted**


Try these files:
sched/sched_send.cpp
client/cpu_sched.cpp
client/client_state.cpp
client/work_fetch.cpp
sched/handle_request.cpp

These have mention of "no task available", I have skimmed the code, so I probably have listed a few too many files. But those are the main ones I can see
ID: 1907416 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1907452 - Posted: 16 Dec 2017, 8:03:12 UTC - in response to Message 1907413.  

The Assimilation, Deletion & Validation backlogs have all finally cleared.
However the Replica continues to fall further & further behind.

And the db results and work unit purging have fallen from their highs of yesterday but they have plateaued out at 4M and are still doubly high from historical levels.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1907452 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14679
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1907459 - Posted: 16 Dec 2017, 8:56:25 UTC - in response to Message 1907360.  

...the server code will be very close to the master code at https://github.com/BOINC/boinc. The client and manager code is in subdirectories 'client' and 'clientgui', respectively: the bit of the server code we're interested in here is in subdirectory 'sched'.
That was an easy solution. There is no directory https://github.com/BOINC/boinc/client/sched, not that I can find anyway. Just by chance I did open home/tbar/Downloads/boinc-master/client/scheduler_op.cpp and search for Project has no tasks available which has to be somewhere in the Server code since it is printed on the replies. There isn't a Project has no tasks available anywhere in the code, so, not the right code.
You won't find server messages in a client folder. Here you go:

https://github.com/BOINC/boinc/blob/master/sched/handle_request.cpp#L1437
ID: 1907459 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1907472 - Posted: 16 Dec 2017, 10:25:42 UTC - in response to Message 1907459.  
Last modified: 16 Dec 2017, 10:34:26 UTC

Considering my experience with the Project Preferences I'd say they are screwed up to the point the Server doesn't know which tasks it should be sending and even if it has tasks to send.

Here's the results from the third machine. The "No tasks are available for AstroPulse v7" is the same reply I receive if I select AP = YES, MB = NO and "If no work for selected applications is available, accept work from other applications?" YES
It's pretty obvious the Server thinks it doesn't have any of the tasks it is trying to send, try to determine why the Server thinks, g_wreq->no_jobs_available
Since the Preferences had been known about for a Year, I'd start there.

The Preferences here were set to AP = YES, MB = YES If no work for selected applications is available, accept work from other applications? = NO

Sat 16 Dec 2017 02:42:13 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 02:42:13 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 02:42:13 AM EST | SETI@home | Reporting 1 completed tasks
Sat 16 Dec 2017 02:42:13 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 02:42:13 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 02:42:13 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 151619.29 seconds; 0.00 devices
Sat 16 Dec 2017 02:42:15 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 02:42:15 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 02:42:15 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 02:42:15 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 02:42:15 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 02:47:21 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 02:47:21 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 02:47:21 AM EST | SETI@home | Reporting 3 completed tasks
Sat 16 Dec 2017 02:47:21 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 02:47:21 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 02:47:21 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 152480.58 seconds; 0.00 devices
Sat 16 Dec 2017 02:47:24 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 02:47:24 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 02:47:24 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 02:47:24 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 02:47:24 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 02:52:31 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 02:52:31 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 02:52:31 AM EST | SETI@home | Reporting 3 completed tasks
Sat 16 Dec 2017 02:52:31 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 02:52:31 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 02:52:31 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 153703.40 seconds; 0.00 devices
Sat 16 Dec 2017 02:52:39 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 02:52:39 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 02:52:39 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 02:52:39 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 02:52:39 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 02:57:45 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 02:57:45 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 02:57:45 AM EST | SETI@home | Reporting 3 completed tasks
Sat 16 Dec 2017 02:57:45 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 02:57:45 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 02:57:45 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 154708.21 seconds; 0.00 devices
Sat 16 Dec 2017 02:57:51 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 02:57:51 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 02:57:51 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 02:57:51 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 02:57:51 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 03:02:57 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 03:02:57 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 03:02:57 AM EST | SETI@home | Reporting 3 completed tasks
Sat 16 Dec 2017 03:02:57 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 03:02:57 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 03:02:57 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 155620.29 seconds; 0.00 devices
Sat 16 Dec 2017 03:03:05 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 03:03:05 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 03:03:05 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 03:03:05 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 03:03:05 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 03:08:11 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 03:08:11 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 03:08:11 AM EST | SETI@home | Reporting 1 completed tasks
Sat 16 Dec 2017 03:08:11 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 03:08:11 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 03:08:11 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 156503.30 seconds; 0.00 devices
Sat 16 Dec 2017 03:08:19 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 03:08:19 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 03:08:19 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 03:08:19 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 03:08:19 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 03:13:26 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 03:13:26 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 03:13:26 AM EST | SETI@home | Reporting 2 completed tasks
Sat 16 Dec 2017 03:13:26 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 03:13:26 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 03:13:26 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 157335.56 seconds; 0.00 devices
Sat 16 Dec 2017 03:13:33 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 03:13:33 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 03:13:33 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 03:13:33 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 03:13:33 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 03:18:39 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 03:18:39 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 03:18:39 AM EST | SETI@home | Reporting 1 completed tasks
Sat 16 Dec 2017 03:18:39 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 03:18:39 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 03:18:39 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 158080.98 seconds; 0.00 devices
Sat 16 Dec 2017 03:18:48 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 03:18:48 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 03:18:48 AM EST | SETI@home | Project has no tasks available
Sat 16 Dec 2017 03:18:48 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 03:23:54 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 03:23:54 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 03:23:54 AM EST | SETI@home | Reporting 2 completed tasks
Sat 16 Dec 2017 03:23:54 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 03:23:54 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 03:23:54 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 159017.68 seconds; 0.00 devices
Sat 16 Dec 2017 03:24:00 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 03:24:00 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 03:24:00 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 03:24:00 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 03:24:00 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 03:29:06 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 03:29:06 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 03:29:06 AM EST | SETI@home | Reporting 1 completed tasks
Sat 16 Dec 2017 03:29:06 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 03:29:06 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 03:29:06 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 160211.76 seconds; 0.00 devices
Sat 16 Dec 2017 03:29:14 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 03:29:14 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 03:29:14 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 03:29:14 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 03:29:14 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 03:34:21 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 03:34:21 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 03:34:21 AM EST | SETI@home | Reporting 2 completed tasks
Sat 16 Dec 2017 03:34:21 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 03:34:21 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 03:34:21 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 160880.07 seconds; 0.00 devices
Sat 16 Dec 2017 03:34:28 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 03:34:28 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 03:34:28 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 03:34:28 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 03:34:28 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 03:39:34 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 03:39:34 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 03:39:34 AM EST | SETI@home | Reporting 2 completed tasks
Sat 16 Dec 2017 03:39:34 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 03:39:34 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 03:39:34 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 161846.94 seconds; 0.00 devices
Sat 16 Dec 2017 03:39:42 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 03:39:42 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 03:39:42 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 03:39:42 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 03:39:42 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 03:44:48 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 03:44:48 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 03:44:48 AM EST | SETI@home | Reporting 4 completed tasks
Sat 16 Dec 2017 03:44:48 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 03:44:48 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 03:44:48 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 163081.62 seconds; 0.00 devices
Sat 16 Dec 2017 03:44:56 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 03:44:56 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 03:44:56 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 03:44:56 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 03:44:56 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 03:50:07 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 03:50:07 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 03:50:07 AM EST | SETI@home | Reporting 3 completed tasks
Sat 16 Dec 2017 03:50:07 AM EST | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Sat 16 Dec 2017 03:50:07 AM EST | SETI@home | [sched_op] CPU work request: 255.65 seconds; 0.00 devices
Sat 16 Dec 2017 03:50:07 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 164182.63 seconds; 0.00 devices
Sat 16 Dec 2017 03:50:10 AM EST | SETI@home | Scheduler request completed: got 1 new tasks
Sat 16 Dec 2017 03:50:10 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 03:50:10 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 03:50:10 AM EST | SETI@home | [sched_op] estimated total CPU task duration: 7788 seconds
Sat 16 Dec 2017 03:50:10 AM EST | SETI@home | [sched_op] estimated total NVIDIA GPU task duration: 0 seconds
Sat 16 Dec 2017 03:50:10 AM EST | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task 21mr07ai.20610.481.5.32.166_2
Sat 16 Dec 2017 03:50:10 AM EST | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task 05mr07ab.18465.375384.5.32.31_1
Sat 16 Dec 2017 03:50:10 AM EST | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task 15fe07ac.29224.9479.15.42.38_1
Sat 16 Dec 2017 03:50:10 AM EST | SETI@home | [sched_op] Deferring communication for 00:05:03
Sat 16 Dec 2017 03:50:10 AM EST | SETI@home | [sched_op] Reason: requested by project
Sat 16 Dec 2017 03:50:12 AM EST | SETI@home | Started download of 16fe07ab.1887.20931.13.40.231.vlar
Sat 16 Dec 2017 03:50:17 AM EST | SETI@home | Finished download of 16fe07ab.1887.20931.13.40.231.vlar
Sat 16 Dec 2017 03:55:16 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 03:55:16 AM EST | SETI@home | Sending scheduler request: To fetch work.
Sat 16 Dec 2017 03:55:16 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 03:55:16 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 03:55:16 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 165081.80 seconds; 0.00 devices
Sat 16 Dec 2017 03:55:23 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 03:55:23 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 03:55:23 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 03:55:23 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 03:55:23 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:00:30 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:00:30 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:00:30 AM EST | SETI@home | Reporting 5 completed tasks
Sat 16 Dec 2017 04:00:30 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 04:00:30 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 04:00:30 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 166682.40 seconds; 0.00 devices
Sat 16 Dec 2017 04:00:34 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 04:00:34 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:00:34 AM EST | SETI@home | Project has no tasks available
Sat 16 Dec 2017 04:00:34 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:05:45 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:05:45 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:05:45 AM EST | SETI@home | Reporting 3 completed tasks
Sat 16 Dec 2017 04:05:45 AM EST | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Sat 16 Dec 2017 04:05:45 AM EST | SETI@home | [sched_op] CPU work request: 236.04 seconds; 0.00 devices
Sat 16 Dec 2017 04:05:45 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 168101.69 seconds; 0.00 devices
Sat 16 Dec 2017 04:05:49 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 04:05:49 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:05:49 AM EST | SETI@home | Project has no tasks available
Sat 16 Dec 2017 04:05:49 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:10:56 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:10:56 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:10:56 AM EST | SETI@home | Reporting 3 completed tasks
Sat 16 Dec 2017 04:10:56 AM EST | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Sat 16 Dec 2017 04:10:56 AM EST | SETI@home | [sched_op] CPU work request: 512.87 seconds; 0.00 devices
Sat 16 Dec 2017 04:10:56 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 169477.68 seconds; 0.00 devices
Sat 16 Dec 2017 04:11:00 AM EST | SETI@home | Scheduler request completed: got 1 new tasks
Sat 16 Dec 2017 04:11:00 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:11:00 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:11:00 AM EST | SETI@home | [sched_op] estimated total CPU task duration: 7788 seconds
Sat 16 Dec 2017 04:11:00 AM EST | SETI@home | [sched_op] estimated total NVIDIA GPU task duration: 0 seconds
Sat 16 Dec 2017 04:11:00 AM EST | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task 28mr07aj.30456.59338.10.37.16_1
Sat 16 Dec 2017 04:11:00 AM EST | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task 05mr07ab.18465.375384.5.32.21_1
Sat 16 Dec 2017 04:11:00 AM EST | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task 05mr07ab.18465.375384.5.32.35_1
Sat 16 Dec 2017 04:11:00 AM EST | SETI@home | [sched_op] Deferring communication for 00:05:03
Sat 16 Dec 2017 04:11:00 AM EST | SETI@home | [sched_op] Reason: requested by project
Sat 16 Dec 2017 04:11:02 AM EST | SETI@home | Started download of 16fe07ab.24649.8661.16.43.230.vlar
Sat 16 Dec 2017 04:11:04 AM EST | SETI@home | Finished download of 16fe07ab.24649.8661.16.43.230.vlar
Sat 16 Dec 2017 04:16:06 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:16:06 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:16:06 AM EST | SETI@home | Reporting 1 completed tasks
Sat 16 Dec 2017 04:16:06 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 04:16:06 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 04:16:06 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 170841.20 seconds; 0.00 devices
Sat 16 Dec 2017 04:16:13 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 04:16:13 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:16:13 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 04:16:13 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 04:16:13 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:21:20 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:21:20 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:21:20 AM EST | SETI@home | Reporting 3 completed tasks
Sat 16 Dec 2017 04:21:20 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 04:21:20 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 04:21:20 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 172227.69 seconds; 0.00 devices
Sat 16 Dec 2017 04:21:27 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 04:21:27 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:21:27 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 04:21:27 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 04:21:27 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:26:33 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:26:33 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:26:33 AM EST | SETI@home | Reporting 2 completed tasks
Sat 16 Dec 2017 04:26:33 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 04:26:33 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 04:26:33 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 173261.59 seconds; 0.00 devices
Sat 16 Dec 2017 04:26:41 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 04:26:41 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:26:41 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 04:26:41 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 04:26:41 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:31:48 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:31:48 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:31:48 AM EST | SETI@home | Reporting 2 completed tasks
Sat 16 Dec 2017 04:31:48 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 04:31:48 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 04:31:48 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 174173.15 seconds; 0.00 devices
Sat 16 Dec 2017 04:31:52 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 04:31:52 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:31:52 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 04:31:52 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 04:31:52 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:36:58 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:36:58 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:36:58 AM EST | SETI@home | Reporting 1 completed tasks
Sat 16 Dec 2017 04:36:58 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 04:36:58 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 04:36:58 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 174828.94 seconds; 0.00 devices
Sat 16 Dec 2017 04:37:02 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 04:37:02 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:37:02 AM EST | SETI@home | Project has no tasks available
Sat 16 Dec 2017 04:37:02 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:42:08 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:42:08 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:42:08 AM EST | SETI@home | Reporting 2 completed tasks
Sat 16 Dec 2017 04:42:08 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 04:42:08 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 04:42:08 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 175778.76 seconds; 0.00 devices
Sat 16 Dec 2017 04:42:10 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 04:42:10 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:42:10 AM EST | SETI@home | Project has no tasks available
Sat 16 Dec 2017 04:42:10 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:47:16 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:47:16 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:47:16 AM EST | SETI@home | Reporting 3 completed tasks
Sat 16 Dec 2017 04:47:16 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 04:47:16 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 04:47:16 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 176973.87 seconds; 0.00 devices
Sat 16 Dec 2017 04:47:22 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 04:47:22 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:47:22 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 04:47:22 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 04:47:22 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:52:29 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:52:29 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:52:29 AM EST | SETI@home | Reporting 1 completed tasks
Sat 16 Dec 2017 04:52:29 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 04:52:29 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 04:52:29 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 178023.16 seconds; 0.00 devices
Sat 16 Dec 2017 04:52:38 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 04:52:38 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:52:38 AM EST | SETI@home | Project has no tasks available
Sat 16 Dec 2017 04:52:38 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 04:57:44 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 04:57:44 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 04:57:44 AM EST | SETI@home | Reporting 4 completed tasks
Sat 16 Dec 2017 04:57:44 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 04:57:44 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 04:57:44 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 179812.06 seconds; 0.00 devices
Sat 16 Dec 2017 04:57:50 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 04:57:50 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 04:57:50 AM EST | SETI@home | No tasks sent
Sat 16 Dec 2017 04:57:50 AM EST | SETI@home | No tasks are available for AstroPulse v7
Sat 16 Dec 2017 04:57:50 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 05:02:57 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 05:02:57 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 05:02:57 AM EST | SETI@home | Reporting 3 completed tasks
Sat 16 Dec 2017 05:02:57 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 05:02:57 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 05:02:57 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 180832.68 seconds; 0.00 devices
Sat 16 Dec 2017 05:03:06 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 05:03:06 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 05:03:06 AM EST | SETI@home | Project has no tasks available
Sat 16 Dec 2017 05:03:06 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 05:13:26 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 05:13:26 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 05:13:26 AM EST | SETI@home | Reporting 5 completed tasks
Sat 16 Dec 2017 05:13:26 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 05:13:26 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 05:13:26 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 183494.00 seconds; 0.00 devices
Sat 16 Dec 2017 05:13:34 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 05:13:34 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 05:13:34 AM EST | SETI@home | Project has no tasks available
Sat 16 Dec 2017 05:13:34 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 05:18:41 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 05:18:41 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 05:18:41 AM EST | SETI@home | Reporting 1 completed tasks
Sat 16 Dec 2017 05:18:41 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 05:18:41 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 05:18:41 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 184147.36 seconds; 0.00 devices
Sat 16 Dec 2017 05:18:45 AM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sat 16 Dec 2017 05:18:45 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 05:18:45 AM EST | SETI@home | Project has no tasks available
Sat 16 Dec 2017 05:18:45 AM EST | SETI@home | Project requested delay of 303 seconds

+++++++++++++++++++

I just changed the Preferences to AP = YES, MB = YES, and If no work for selected applications is available, accept work from other applications? = YES, and the Server finally Figured out What tasks to send...for a little while.

Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | Reporting 5 completed tasks
Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 185362.12 seconds; 0.00 devices
Sat 16 Dec 2017 05:24:41 AM EST | SETI@home | Scheduler request completed: got 72 new tasks
Sat 16 Dec 2017 05:24:41 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 05:24:41 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 05:24:41 AM EST | SETI@home | [sched_op] estimated total CPU task duration: 0 seconds
Sat 16 Dec 2017 05:24:41 AM EST | SETI@home | [sched_op] estimated total NVIDIA GPU task duration: 17776 seconds
ID: 1907472 · Report as offensive
Profile David@home
Volunteer tester
Avatar

Send message
Joined: 16 Jan 03
Posts: 755
Credit: 5,040,916
RAC: 28
United Kingdom
Message 1907484 - Posted: 16 Dec 2017, 12:39:22 UTC

Cache size falling, event log getting "no tasks are available"

"I have a bad feeling about this"
ID: 1907484 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1907561 - Posted: 16 Dec 2017, 20:41:29 UTC - in response to Message 1907484.  

Cache size falling, event log getting "no tasks are available"

"I have a bad feeling about this"


. . It's funny, after the 1st big crash 2 weeks ago my caches would fill AOK. For the rest of that week I didn't have to play "kick the server". But after the second unscheduled outage it's back to business as usual and I have to kick the servers a couple of times a day to keep them refilling.

. . Life goes on. But it is problem that would be better cured.

Stephen

:(
ID: 1907561 · Report as offensive
Profile Stargate (SA)
Volunteer tester
Avatar

Send message
Joined: 4 Mar 10
Posts: 1854
Credit: 2,258,721
RAC: 0
Australia
Message 1907562 - Posted: 16 Dec 2017, 20:47:45 UTC - in response to Message 1907561.  

Rut row :(
ID: 1907562 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1907736 - Posted: 17 Dec 2017, 17:25:47 UTC - in response to Message 1907472.  

I just changed the Preferences to AP = YES, MB = YES, and If no work for selected applications is available, accept work from other applications? = YES, and the Server finally Figured out What tasks to send...for a little while.

Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | [sched_op] Starting scheduler request
Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | Reporting 5 completed tasks
Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sat 16 Dec 2017 05:24:38 AM EST | SETI@home | [sched_op] NVIDIA GPU work request: 185362.12 seconds; 0.00 devices
Sat 16 Dec 2017 05:24:41 AM EST | SETI@home | Scheduler request completed: got 72 new tasks
Sat 16 Dec 2017 05:24:41 AM EST | SETI@home | [sched_op] Server version 707
Sat 16 Dec 2017 05:24:41 AM EST | SETI@home | Project requested delay of 303 seconds
Sat 16 Dec 2017 05:24:41 AM EST | SETI@home | [sched_op] estimated total CPU task duration: 0 seconds
Sat 16 Dec 2017 05:24:41 AM EST | SETI@home | [sched_op] estimated total NVIDIA GPU task duration: 17776 seconds
Seems this machine is still getting starved. So, I'm going to try the same setting that the Mac has been using since it's about the only one that hasn't had any real trouble getting Work. It still gets the Project has no tasks available ever so often, but has been better than the other machines. Pretty simple settings;
AP = NO, MB = YES, If no work for selected applications is available, accept work from other applications? = NO

So far so good,
Sun 17 Dec 2017 12:09:01 PM EST | SETI@home | Sending scheduler request: To fetch work.
Sun 17 Dec 2017 12:09:01 PM EST | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Sun 17 Dec 2017 12:09:01 PM EST | SETI@home | [sched_op] CPU work request: 1271.63 seconds; 0.00 devices
Sun 17 Dec 2017 12:09:01 PM EST | SETI@home | [sched_op] NVIDIA GPU work request: 215041.36 seconds; 0.00 devices
Sun 17 Dec 2017 12:09:04 PM EST | SETI@home | Scheduler request completed: got 113 new tasks
Sun 17 Dec 2017 12:09:04 PM EST | SETI@home | [sched_op] Server version 707
Sun 17 Dec 2017 12:09:04 PM EST | SETI@home | Project requested delay of 303 seconds
Sun 17 Dec 2017 12:09:04 PM EST | SETI@home | [sched_op] estimated total CPU task duration: 6857 seconds
Sun 17 Dec 2017 12:09:04 PM EST | SETI@home | [sched_op] estimated total NVIDIA GPU task duration: 48019 seconds
Sun 17 Dec 2017 12:09:04 PM EST | SETI@home | [sched_op] Deferring communication for 00:05:03
ID: 1907736 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1907777 - Posted: 17 Dec 2017, 22:38:04 UTC

Since switching over to just MB the cache is staying near full, but, I'm seeing a new type a message. Now, it doesn't even give a reason, it just says, No Task Sent. I suppose it can't even come up with an excuse anymore.

Sun 17 Dec 2017 05:08:21 PM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sun 17 Dec 2017 05:08:21 PM EST | SETI@home | Reporting 1 completed tasks
Sun 17 Dec 2017 05:08:21 PM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sun 17 Dec 2017 05:08:21 PM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sun 17 Dec 2017 05:08:21 PM EST | SETI@home | [sched_op] NVIDIA GPU work request: 159656.95 seconds; 0.00 devices
Sun 17 Dec 2017 05:08:22 PM EST | SETI@home | Started upload of blc04_2bit_guppi_57903_55026_HIP21272_0023.11137.818.17.26.31.vlar_1_r222659153_0
Sun 17 Dec 2017 05:08:23 PM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sun 17 Dec 2017 05:08:23 PM EST | SETI@home | [sched_op] Server version 707
Sun 17 Dec 2017 05:08:23 PM EST | SETI@home | No tasks sent
Sun 17 Dec 2017 05:08:23 PM EST | SETI@home | Project requested delay of 303 seconds
Sun 17 Dec 2017 05:13:29 PM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sun 17 Dec 2017 05:13:29 PM EST | SETI@home | Reporting 4 completed tasks
Sun 17 Dec 2017 05:13:29 PM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sun 17 Dec 2017 05:13:29 PM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sun 17 Dec 2017 05:13:29 PM EST | SETI@home | [sched_op] NVIDIA GPU work request: 161166.29 seconds; 0.00 devices
Sun 17 Dec 2017 05:13:35 PM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sun 17 Dec 2017 05:13:35 PM EST | SETI@home | [sched_op] Server version 707
Sun 17 Dec 2017 05:13:35 PM EST | SETI@home | No tasks sent
Sun 17 Dec 2017 05:13:35 PM EST | SETI@home | Project requested delay of 303 seconds
Sun 17 Dec 2017 05:18:42 PM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sun 17 Dec 2017 05:18:42 PM EST | SETI@home | Reporting 2 completed tasks
Sun 17 Dec 2017 05:18:42 PM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sun 17 Dec 2017 05:18:42 PM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sun 17 Dec 2017 05:18:42 PM EST | SETI@home | [sched_op] NVIDIA GPU work request: 162115.50 seconds; 0.00 devices
Sun 17 Dec 2017 05:18:50 PM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sun 17 Dec 2017 05:18:50 PM EST | SETI@home | [sched_op] Server version 707
Sun 17 Dec 2017 05:18:50 PM EST | SETI@home | No tasks sent
Sun 17 Dec 2017 05:18:50 PM EST | SETI@home | Project requested delay of 303 seconds
Sun 17 Dec 2017 05:23:56 PM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sun 17 Dec 2017 05:23:56 PM EST | SETI@home | Reporting 3 completed tasks
Sun 17 Dec 2017 05:23:56 PM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sun 17 Dec 2017 05:23:56 PM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sun 17 Dec 2017 05:23:56 PM EST | SETI@home | [sched_op] NVIDIA GPU work request: 163310.64 seconds; 0.00 devices
Sun 17 Dec 2017 05:24:04 PM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sun 17 Dec 2017 05:24:04 PM EST | SETI@home | [sched_op] Server version 707
Sun 17 Dec 2017 05:24:04 PM EST | SETI@home | No tasks sent
Sun 17 Dec 2017 05:24:04 PM EST | SETI@home | Project requested delay of 303 seconds
Sun 17 Dec 2017 05:29:11 PM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sun 17 Dec 2017 05:29:11 PM EST | SETI@home | Reporting 4 completed tasks
Sun 17 Dec 2017 05:29:11 PM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sun 17 Dec 2017 05:29:11 PM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sun 17 Dec 2017 05:29:11 PM EST | SETI@home | [sched_op] NVIDIA GPU work request: 164456.90 seconds; 0.00 devices
Sun 17 Dec 2017 05:29:18 PM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sun 17 Dec 2017 05:29:18 PM EST | SETI@home | [sched_op] Server version 707
Sun 17 Dec 2017 05:29:18 PM EST | SETI@home | No tasks sent
Sun 17 Dec 2017 05:29:18 PM EST | SETI@home | Project requested delay of 303 seconds
Sun 17 Dec 2017 05:34:24 PM EST | SETI@home | Sending scheduler request: To report completed tasks.
Sun 17 Dec 2017 05:34:24 PM EST | SETI@home | Reporting 1 completed tasks
Sun 17 Dec 2017 05:34:24 PM EST | SETI@home | Requesting new tasks for NVIDIA GPU
Sun 17 Dec 2017 05:34:24 PM EST | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Sun 17 Dec 2017 05:34:24 PM EST | SETI@home | [sched_op] NVIDIA GPU work request: 165385.76 seconds; 0.00 devices
Sun 17 Dec 2017 05:34:31 PM EST | SETI@home | Scheduler request completed: got 0 new tasks
Sun 17 Dec 2017 05:34:31 PM EST | SETI@home | [sched_op] Server version 707
Sun 17 Dec 2017 05:34:31 PM EST | SETI@home | No tasks sent
Sun 17 Dec 2017 05:34:31 PM EST | SETI@home | Project requested delay of 303 seconds
ID: 1907777 · Report as offensive
Profile Stargate (SA)
Volunteer tester
Avatar

Send message
Joined: 4 Mar 10
Posts: 1854
Credit: 2,258,721
RAC: 0
Australia
Message 1907782 - Posted: 17 Dec 2017, 23:01:47 UTC - in response to Message 1907777.  

Maybe it's gone on strike!

:)
ID: 1907782 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 36781
Credit: 261,360,520
RAC: 489
Australia
Message 1907786 - Posted: 17 Dec 2017, 23:39:06 UTC

No problems at all to report here.

Cheers.
ID: 1907786 · Report as offensive
Profile Stargate (SA)
Volunteer tester
Avatar

Send message
Joined: 4 Mar 10
Posts: 1854
Credit: 2,258,721
RAC: 0
Australia
Message 1907789 - Posted: 17 Dec 2017, 23:58:58 UTC

Working fine here also..

Steve
ID: 1907789 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1907791 - Posted: 18 Dec 2017, 0:14:30 UTC
Last modified: 18 Dec 2017, 0:26:08 UTC

It's back to normal here, for now. I think it's perfectly clear from the Logs that at times the Server chooses to simply Ignore some Work requests. I don't see how anyone can call it a "conspiracy theory" when the logs show it to be fact. We'll see how it continues, so far the other two machines haven't had any trouble asking for MB & AP.

BTW, I doubt someone who just goes through a dozen or so tasks a day would notice a problem if the server stopped sending tasks for an hour or so. Certainly not the way a host returning 3 or 4 tasks every 5 minutes would anyway.
ID: 1907791 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13854
Credit: 208,696,464
RAC: 304
Australia
Message 1907823 - Posted: 18 Dec 2017, 6:43:07 UTC
Last modified: 18 Dec 2017, 6:52:51 UTC

Well, the web site is up and down, mostly down at the moment. And the forums are re-defining slow & unresponsive.
And the Scheduler hasn't responded for about 20min now.

EDIT- managed to pick up some work after posting this, and the following request also managed to get work- but it took over 30 seconds. Usually it's 3-4 seconds.

And a couple of the Server Status values aren't updating- WU waiting purge, AP & MB.
Replica continues to fall further behind.

And next request it's back to normal- able to report work, but it won't give me any.
Triple update time.
Grant
Darwin NT
ID: 1907823 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1907849 - Posted: 18 Dec 2017, 13:16:16 UTC - in response to Message 1907736.  
Last modified: 18 Dec 2017, 14:01:33 UTC

...I'm going to try the same setting that the Mac has been using since it's about the only one that hasn't had any real trouble getting Work. It still gets the Project has no tasks available ever so often, but has been better than the other machines...
I decided to see what was happening to the Mac Requests at the same time the Linux Requests were being Ignored. It seems the Server was treating the Mac requests normally and replacing the Reported tasks with replacements. Both machines are connected to the same Ethernet switch, the Mac is using 24 hour time.

17-Dec-2017 17:00:56 [SETI@home] Reporting 6 completed tasks
17-Dec-2017 17:00:57 [SETI@home] Scheduler request completed: got 6 new tasks
17-Dec-2017 17:06:09 [SETI@home] Reporting 4 completed tasks
17-Dec-2017 17:06:10 [SETI@home] Scheduler request completed: got 4 new tasks
17-Dec-2017 17:11:23 [SETI@home] Reporting 2 completed tasks
17-Dec-2017 17:11:24 [SETI@home] Scheduler request completed: got 2 new tasks
17-Dec-2017 17:16:37 [SETI@home] Reporting 5 completed tasks
17-Dec-2017 17:16:41 [SETI@home] Scheduler request completed: got 4 new tasks
17-Dec-2017 17:21:53 [SETI@home] Reporting 3 completed tasks
17-Dec-2017 17:21:54 [SETI@home] Scheduler request completed: got 3 new tasks
17-Dec-2017 17:27:07 [SETI@home] Reporting 5 completed tasks
17-Dec-2017 17:27:08 [SETI@home] Scheduler request completed: got 5 new tasks
17-Dec-2017 17:32:21 [SETI@home] Reporting 5 completed tasks
17-Dec-2017 17:32:25 [SETI@home] Scheduler request completed: got 5 new tasks
17-Dec-2017 17:37:33 [SETI@home] Reporting 4 completed tasks
17-Dec-2017 17:37:35 [SETI@home] Scheduler request completed: got 0 new tasks
17-Dec-2017 17:37:35 [SETI@home] No tasks sent
17-Dec-2017 17:42:48 [SETI@home] Reporting 5 completed tasks
17-Dec-2017 17:42:49 [SETI@home] Scheduler request completed: got 9 new tasks
* corrected for 24 hr time

Why would it Ignore one Host and just miss a single beat on the other Host sitting next to it?
ID: 1907849 · Report as offensive
Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 36 · Next

Message boards : Number crunching : Panic Mode On (109) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.