Panic Mode On (111) Server Problems?

Message boards : Number crunching : Panic Mode On (111) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 3 · 4 · 5 · 6 · 7 · 8 · 9 . . . 31 · Next

AuthorMessage
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1927476 - Posted: 31 Mar 2018, 9:53:03 UTC
Last modified: 31 Mar 2018, 10:32:29 UTC

Things are looking better. I went to sleep last night right after I started getting the Project is Down message. This morning the caches were Empty and No tasks were being sent.

Just Now, I'm starting to get New Work.

Oh well,
A couple of machine got a few dozen tasks, then nothing since.
Now they are out again, and are receiving;
Sat Mar 31 06:29:48 2018 | SETI@home | Sending scheduler request: To report completed tasks.
Sat Mar 31 06:29:48 2018 | SETI@home | Reporting 6 completed tasks
Sat Mar 31 06:29:48 2018 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Sat Mar 31 06:29:48 2018 | SETI@home | [sched_op] CPU work request: 166313.25 seconds; 0.00 devices
Sat Mar 31 06:29:48 2018 | SETI@home | [sched_op] NVIDIA GPU work request: 281457.27 seconds; 0.00 devices
Sat Mar 31 06:29:50 2018 | SETI@home | Scheduler request completed: got 0 new tasks
Sat Mar 31 06:29:50 2018 | SETI@home | Project has no tasks available

blah...
ID: 1927476 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13854
Credit: 208,696,464
RAC: 304
Australia
Message 1927480 - Posted: 31 Mar 2018, 10:31:24 UTC - in response to Message 1927476.  

Just Now, I'm starting to get New Work.

You must be one of the very few.
I haven't picked up any work for hours, V8 In-progress numbers continue to decline. Eric posted in the News thread, more database issues.
Grant
Darwin NT
ID: 1927480 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1927482 - Posted: 31 Mar 2018, 11:31:15 UTC - in response to Message 1927480.  
Last modified: 31 Mar 2018, 11:31:44 UTC

I haven't picked up any work for hours, V8 In-progress numbers continue to decline. Eric posted in the News thread, more database issues.

The crunching den is so quiet, can't even hear the fans, CPU at 32C Gpus 30-34C, running out of gas.
All requests for UL/DL from my side ends on:
Sat 31 Mar 2018 06:23:19 AM EST | SETI@home | Scheduler request failed: HTTP internal server error

ID: 1927482 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1927483 - Posted: 31 Mar 2018, 11:34:53 UTC - in response to Message 1927482.  

Uploads seem to be working fine here.
But work requests result in no joy.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1927483 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1927484 - Posted: 31 Mar 2018, 11:45:50 UTC - in response to Message 1927483.  
Last modified: 31 Mar 2018, 11:48:42 UTC

Sat 31 Mar 2018 06:43:47 AM EST | SETI@home | update requested by user
Sat 31 Mar 2018 06:43:52 AM EST | SETI@home | Sending scheduler request: Requested by user.
Sat 31 Mar 2018 06:43:52 AM EST | SETI@home | Reporting 67 completed tasks
Sat 31 Mar 2018 06:43:52 AM EST | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Sat 31 Mar 2018 06:44:35 AM EST | SETI@home | Scheduler request failed: HTTP internal server error

Interesting: Why you can UL and I no?
Could be related to the Boinc version we use?
Pause for some coffee & aspirins.
ID: 1927484 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1927485 - Posted: 31 Mar 2018, 11:48:21 UTC - in response to Message 1927484.  

Sat 31 Mar 2018 06:43:47 AM EST | SETI@home | update requested by user
Sat 31 Mar 2018 06:43:52 AM EST | SETI@home | Sending scheduler request: Requested by user.
Sat 31 Mar 2018 06:43:52 AM EST | SETI@home | Reporting 67 completed tasks
Sat 31 Mar 2018 06:43:52 AM EST | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Sat 31 Mar 2018 06:44:35 AM EST | SETI@home | Scheduler request failed: HTTP internal server error

Interesting: Why you can UL and I no?

That is not an upload. That is a reporting/work request.
The 67 WUs it is trying to report have already been uploaded.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1927485 · Report as offensive
Profile Stargate (SA)
Volunteer tester
Avatar

Send message
Joined: 4 Mar 10
Posts: 1854
Credit: 2,258,721
RAC: 0
Australia
Message 1927486 - Posted: 31 Mar 2018, 11:48:22 UTC

Could be a long day tomorrow..
ID: 1927486 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14679
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1927490 - Posted: 31 Mar 2018, 12:00:33 UTC - in response to Message 1927485.  

Sat 31 Mar 2018 06:43:47 AM EST | SETI@home | update requested by user
Sat 31 Mar 2018 06:43:52 AM EST | SETI@home | Sending scheduler request: Requested by user.
Sat 31 Mar 2018 06:43:52 AM EST | SETI@home | Reporting 67 completed tasks
Sat 31 Mar 2018 06:43:52 AM EST | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Sat 31 Mar 2018 06:44:35 AM EST | SETI@home | Scheduler request failed: HTTP internal server error

Interesting: Why you can UL and I no?
That is not an upload. That is a reporting/work request.
The 67 WUs it is trying to report have already been uploaded.
As we discussed in this thread a couple pf days ago, a large report (and 67 tasks is large, in this context) may be triggering the internal server error. Check out the posts from the people who accepted the advice to make the reports smaller.
ID: 1927490 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1927491 - Posted: 31 Mar 2018, 12:08:02 UTC - in response to Message 1927490.  
Last modified: 31 Mar 2018, 12:08:33 UTC

As we discussed in this thread a couple pf days ago, a large report (and 67 tasks is large, in this context) may be triggering the internal server error. Check out the posts from the people who accepted the advice to make the reports smaller.

Sat 31 Mar 2018 07:04:31 AM EST | SETI@home | Sending scheduler request: Requested by user.
Sat 31 Mar 2018 07:04:31 AM EST | SETI@home | Reporting 32 completed tasks
Sat 31 Mar 2018 07:04:31 AM EST | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Sat 31 Mar 2018 07:05:15 AM EST | SETI@home | Scheduler request failed: HTTP internal server error

Down to 32 and the answer is the same. The question could be: What the number of WU to report who is not large?
ID: 1927491 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1927492 - Posted: 31 Mar 2018, 12:12:21 UTC - in response to Message 1927491.  

Just keep reducing it until it works.
Try 10.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1927492 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1927493 - Posted: 31 Mar 2018, 12:19:15 UTC - in response to Message 1927492.  
Last modified: 31 Mar 2018, 12:31:04 UTC

Just keep reducing it until it works.
Try 10.

Will try but normaly my host has more than 10WU to report.
<edit> Down to 8 and no joy, the same error.
Will shutdown the host until Seti recovers, at least i save power.
ID: 1927493 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1927494 - Posted: 31 Mar 2018, 12:23:28 UTC - in response to Message 1927493.  

Just keep reducing it until it works.
Try 10.

Will try but normaly my host has more than 10WU to report.

Well, this is only temporary. Things are not working normally right now.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1927494 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1927495 - Posted: 31 Mar 2018, 12:34:14 UTC - in response to Message 1927494.  

Well, this is only temporary. Things are not working normally right now.

Down to 2 and not even works.
Was you able to report your work or you get the same error? To see if the Boinc version makes difference.
ID: 1927495 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1927496 - Posted: 31 Mar 2018, 12:40:13 UTC - in response to Message 1927495.  

Well, this is only temporary. Things are not working normally right now.

Down to 2 and not even works.
Was you able to report your work or you get the same error? To see if the Boinc version makes difference.

It should not. But I am reporting 5 or 6 at a time and it is working. Just getting no new work back in return.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1927496 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1927497 - Posted: 31 Mar 2018, 13:06:29 UTC - in response to Message 1927495.  

Well, this is only temporary. Things are not working normally right now.

Down to 2 and not even works.
Was you able to report your work or you get the same error? To see if the Boinc version makes difference.

No Problems with this Ubuntu machine Reporting work. If you look at it, it received a New task as recent as 31 Mar 2018, 11:54:07 UTC.
It reported it as completed 5 minutes later, all perfectly normal. It just can't get enough to matter, <core_client_version>7.4.44</core_client_version>
ID: 1927497 · Report as offensive
Profile Chris904395093209d Project Donor
Volunteer tester

Send message
Joined: 1 Jan 01
Posts: 112
Credit: 29,923,129
RAC: 6
United States
Message 1927498 - Posted: 31 Mar 2018, 13:08:44 UTC

Eric mentioned he may have to rebuild an index that is used for task requests, if the reboots didn't work. My cache is down 50% so I'm guessing he will have to rebuild the index. That will mean downtime for our machines - so get those backup projects ready or save a little electricity :-)

Over a million work units ready to send, so when everything is said and done, we should be able to fill our caches quickly enough.
~Chris

ID: 1927498 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1927504 - Posted: 31 Mar 2018, 14:01:56 UTC - in response to Message 1927495.  

Down to 2 and not even works.
♫♫ One is the loneliest number that you'll ever do ♫♫

31/03/2018 15:54:02 | SETI@home | Sending scheduler request: To report completed tasks.
31/03/2018 15:54:02 | SETI@home | Reporting 1 completed tasks
31/03/2018 15:54:02 | SETI@home | Not requesting tasks: don't need (CPU: ; AMD/ATI GPU: job cache full)
31/03/2018 15:54:07 | SETI@home | Scheduler request completed
31/03/2018 15:54:07 | SETI@home | Project requested delay of 303 seconds

31/03/2018 16:01:03 | SETI@home | Sending scheduler request: To report completed tasks.
31/03/2018 16:01:03 | SETI@home | Reporting 1 completed tasks
31/03/2018 16:01:03 | SETI@home | Not requesting tasks: don't need (CPU: ; AMD/ATI GPU: job cache full)
31/03/2018 16:01:07 | SETI@home | Scheduler request completed

:)
ID: 1927504 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1927507 - Posted: 31 Mar 2018, 14:28:48 UTC - in response to Message 1927506.  

Much better:

SETI@home 2018-03-31 16:09:32 Reporting 2 completed tasks
SETI@home 2018-03-31 16:09:32 Requesting new tasks for NVIDIA GPU
SETI@home 2018-03-31 16:09:38 Scheduler request completed: got 34 new tasks

Well, 'tain't happnin' on a regular basis yet, for shore. Lucky you be.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1927507 · Report as offensive
mrchips
Avatar

Send message
Joined: 12 Dec 04
Posts: 17
Credit: 26,590,842
RAC: 8
United States
Message 1927515 - Posted: 31 Mar 2018, 15:58:47 UTC - in response to Message 1927509.  

I got 12 GPU tasks an hour ago, but NO CPU tasks.
ID: 1927515 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1927520 - Posted: 31 Mar 2018, 16:50:34 UTC
Last modified: 31 Mar 2018, 16:55:14 UTC

Ops! Just get the #1 WU since yesterday!
ID: 1927520 · Report as offensive
Previous · 1 . . . 3 · 4 · 5 · 6 · 7 · 8 · 9 . . . 31 · Next

Message boards : Number crunching : Panic Mode On (111) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.