Scheduler request failed: Error 403

Questions and Answers : Windows : Scheduler request failed: Error 403
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile startrekforever

Send message
Joined: 5 Dec 99
Posts: 36
Credit: 17,772,420
RAC: 0
United States
Message 974279 - Posted: 27 Feb 2010, 2:45:31 UTC

The issue of getting "Scheduler request failed: Error 403' has been recurring on two of my systems about once a month or so and the only way I could get around this has been to reset the project. The problem usually occurs if I have several hundred tasks to report. Current system with the problem has 691 completed tasks. I am running the latest client. (This was previously posted, but no working fix was obtained at that time, and setting up a cc_config per one of the responders to the following:
<cc_config>
<options>
<http_1_0>1</http_1_0>
</options>
</cc_config>
did no good for my proxied servers.)

The machines with the problem are running Win2008 Standard or Win2008 Standard R2 and are using a domain account. They both only do CUDA workunits.

Any suggestions?

Thanks.

ID: 974279 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 974320 - Posted: 27 Feb 2010, 3:38:03 UTC - in response to Message 974279.  

What version of BOINC?


BOINC WIKI
ID: 974320 · Report as offensive
Profile startrekforever

Send message
Joined: 5 Dec 99
Posts: 36
Credit: 17,772,420
RAC: 0
United States
Message 974328 - Posted: 27 Feb 2010, 3:44:48 UTC

6.10.18.
ID: 974328 · Report as offensive
Profile Gundolf Jahn

Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 446,358
RAC: 0
Germany
Message 974389 - Posted: 27 Feb 2010, 8:04:00 UTC - in response to Message 974279.  

The more tasks are stored with your BOINC client, the bigger the sched_request*.xml files get. So, perhaps your proxy has some size limit set, since HTTP error 403 is "forbidden".

If that's the case, lowering your cache size (Additional work buffer) could help.

Gruß,
Gundolf
Computer sind nicht alles im Leben. (Kleiner Scherz)

SETI@home classic workunits 3,758
SETI@home classic CPU time 66,520 hours
ID: 974389 · Report as offensive
Profile startrekforever

Send message
Joined: 5 Dec 99
Posts: 36
Credit: 17,772,420
RAC: 0
United States
Message 974613 - Posted: 27 Feb 2010, 23:25:11 UTC - in response to Message 974389.  

The current sched_request*.xml file is about 2 MB, and I have no problems downloading larger files through HTTP, but maybe I do have an upload limit I am not aware of.

Lowering the cache size does sound like something I would want to do, as the system in question has two NVidia 260s and can usually do over 300 units a day. The current setting for these machines is a 4 day cache.

But, you are correct, 403 is a 'forbidden', so I may have to lower my cache to 1 or 2 days.

Why is the scheduler request file so large though? Any workable workaround other than reducing my cache size?

Thanks.



ID: 974613 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 974631 - Posted: 28 Feb 2010, 1:18:48 UTC - in response to Message 974613.  

The current sched_request*.xml file is about 2 MB, and I have no problems downloading larger files through HTTP, but maybe I do have an upload limit I am not aware of.

Lowering the cache size does sound like something I would want to do, as the system in question has two NVidia 260s and can usually do over 300 units a day. The current setting for these machines is a 4 day cache.

But, you are correct, 403 is a 'forbidden', so I may have to lower my cache to 1 or 2 days.

Why is the scheduler request file so large though? Any workable workaround other than reducing my cache size?

Thanks.



Each task adds some data to the scheduler_request file to do the report of the task.

I believe that some change was made in the 6.10 series someplace that limited the number of tasks reported during any single request. Try 6.10.35


BOINC WIKI
ID: 974631 · Report as offensive
Profile startrekforever

Send message
Joined: 5 Dec 99
Posts: 36
Credit: 17,772,420
RAC: 0
United States
Message 975034 - Posted: 1 Mar 2010, 15:24:30 UTC

6.10.35 did not fix the 403 error. I'm leaving that version on though, and also not resetting the project just yet. Any other possible suggestions?

Thanks.

ID: 975034 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 975229 - Posted: 2 Mar 2010, 3:45:38 UTC - in response to Message 975034.  

6.10.35 did not fix the 403 error. I'm leaving that version on though, and also not resetting the project just yet. Any other possible suggestions?

Thanks.

OK, could you turn on some debugging flags for us?

Please go to this page read the instructions and turn on the following debug flags: <task>, <file_xfer>, <sched_ops>, <cpu_sched>, <file_xfer_debug>, <http_debug>, <http_xfer_debug>, <proxy_debug>, and <sched_op_debug>.

After this is done, report the messages from one update.


BOINC WIKI
ID: 975229 · Report as offensive
Profile startrekforever

Send message
Joined: 5 Dec 99
Posts: 36
Credit: 17,772,420
RAC: 0
United States
Message 975313 - Posted: 2 Mar 2010, 15:41:05 UTC - in response to Message 975229.  

02-Mar-2010 10:30:15 [SETI@home] Reporting 691 completed tasks, not requesting new tasks
02-Mar-2010 10:30:15 [SETI@home] [sched_op_debug] CPU work request: 0.00 seconds; 0.00 idle CPUs
02-Mar-2010 10:30:15 [SETI@home] [sched_op_debug] NVIDIA GPU work request: 0.00 seconds; 0.00 idle GPUs
02-Mar-2010 10:30:15 [---] [http_debug] HTTP_OP::init_post(): http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi
02-Mar-2010 10:30:15 [---] [http_debug] HTTP_OP::libcurl_exec(): ca-bundle set
02-Mar-2010 10:30:15 [---] [proxy_debug] HTTP_OP::no_proxy_for_url(): http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi
02-Mar-2010 10:30:15 [---] [proxy_debug] returning false
02-Mar-2010 10:30:15 [---] [proxy_debug]: setting up proxy 10.98.20.23:8080
02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Info: Re-using existing connection! (#0) with host 10.98.20.23
02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Info: Connected to 10.98.20.23 (10.98.20.23) port 8080 (#0)
02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Sent header to server: POST http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi HTTP/1.1

02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Sent header to server: User-Agent: BOINC client (windows_x86_64 6.10.35)

02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Sent header to server: Host: setiboinc.ssl.berkeley.edu

02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Sent header to server: Accept: */*

02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Sent header to server: Accept-Encoding: deflate, gzip

02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Sent header to server: Proxy-Connection: Keep-Alive

02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Sent header to server: Content-Type: application/x-www-form-urlencoded

02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Sent header to server: Content-Length: 2296942

02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Sent header to server: Expect: 100-continue

02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Sent header to server:

02-Mar-2010 10:30:16 [---] [http_debug] [ID#1] Info: Expire cleared
02-Mar-2010 10:30:17 [---] [http_debug] [ID#1] Info: Done waiting for 100-continue
02-Mar-2010 10:30:17 [---] [http_debug] [ID#1] Received header from server: HTTP/1.1 403 Forbidden

02-Mar-2010 10:30:17 [---] [http_debug] [ID#1] Received header from server: Content-Length: 1333

02-Mar-2010 10:30:17 [---] [http_debug] [ID#1] Received header from server: Content-Type: text/html; charset=ISO-8859-1

02-Mar-2010 10:30:17 [---] [http_debug] [ID#1] Received header from server: Pragma: no-cache

02-Mar-2010 10:30:17 [---] [http_debug] [ID#1] Received header from server: Proxy-Connection: keep-alive

02-Mar-2010 10:30:17 [---] [http_debug] [ID#1] Received header from server: Via: 1.1 webwasher (Webwasher 6.8.6.6257)

02-Mar-2010 10:30:17 [---] [http_debug] [ID#1] Received header from server: X-Error-Name: parametertoobig

02-Mar-2010 10:30:17 [---] [http_debug] [ID#1] Received header from server:

02-Mar-2010 10:30:17 [---] [http_xfer_debug] [ID#1] HTTP: wrote 1333 bytes
02-Mar-2010 10:30:18 [---] [http_debug] [ID#1] Info: Expire cleared
02-Mar-2010 10:30:18 [---] [http_debug] [ID#1] Info: Connection #0 to host 10.98.20.23 left intact
02-Mar-2010 10:30:20 [SETI@home] Scheduler request failed: Error 403
02-Mar-2010 10:30:20 [SETI@home] [sched_op_debug] Deferring communication for 1 min 0 sec
02-Mar-2010 10:30:20 [SETI@home] [sched_op_debug] Reason: Scheduler request failed

ID: 975313 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 975417 - Posted: 3 Mar 2010, 4:05:02 UTC

It looks like there's an Apache config parameter that limits request size. I think this is set to 1 MB, which would reject your request message. I'll crank it up to several MB tomorrow.
-- David
ID: 975417 · Report as offensive
Profile startrekforever

Send message
Joined: 5 Dec 99
Posts: 36
Credit: 17,772,420
RAC: 0
United States
Message 975553 - Posted: 3 Mar 2010, 19:39:11 UTC - in response to Message 975417.  
Last modified: 3 Mar 2010, 19:41:51 UTC

Thanks, David, fingers crossed. Will keep the server down / suspended until I hear back from you that the cache has been bumped.
ID: 975553 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 975574 - Posted: 3 Mar 2010, 21:25:22 UTC - in response to Message 975417.  
Last modified: 3 Mar 2010, 21:29:25 UTC

Tsk... account hijacker. ;-)

I did not make that post.
ID: 975574 · Report as offensive
Profile David Anderson
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 13 Feb 99
Posts: 173
Credit: 502,653
RAC: 0
Message 975587 - Posted: 3 Mar 2010, 22:26:45 UTC

Oops!! I had logged into Jord's account for some other problem, and forgot to log out. Mea culpa! -- David

BTW: We've changed our Apache configuration to allow large request messages. Let me know if it's working now.
ID: 975587 · Report as offensive
Profile startrekforever

Send message
Joined: 5 Dec 99
Posts: 36
Credit: 17,772,420
RAC: 0
United States
Message 975603 - Posted: 4 Mar 2010, 0:16:32 UTC - in response to Message 975587.  

Nope. Maybe it is my companies 'WebWasher' proxy that is the culprit?

03-Mar-2010 19:11:10 [---] Re-reading cc_config.xml
03-Mar-2010 19:11:10 [---] Re-read config file
03-Mar-2010 19:11:10 [---] log flags: file_xfer, sched_ops, task, cpu_sched, file_xfer_debug, http_debug
03-Mar-2010 19:11:10 [---] log flags: http_xfer_debug, proxy_debug, sched_op_debug
03-Mar-2010 19:11:18 [SETI@home] update requested by user
03-Mar-2010 19:11:24 [SETI@home] [sched_op_debug] Starting scheduler request
03-Mar-2010 19:11:24 [SETI@home] Sending scheduler request: Requested by user.
03-Mar-2010 19:11:24 [SETI@home] Reporting 691 completed tasks, not requesting new tasks
03-Mar-2010 19:11:24 [SETI@home] [sched_op_debug] CPU work request: 0.00 seconds; 0.00 idle CPUs
03-Mar-2010 19:11:24 [SETI@home] [sched_op_debug] NVIDIA GPU work request: 0.00 seconds; 0.00 idle GPUs
03-Mar-2010 19:11:24 [---] [http_debug] HTTP_OP::init_post(): http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi
03-Mar-2010 19:11:24 [---] [http_debug] HTTP_OP::libcurl_exec(): ca-bundle set
03-Mar-2010 19:11:24 [---] [proxy_debug] HTTP_OP::no_proxy_for_url(): http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi
03-Mar-2010 19:11:24 [---] [proxy_debug] returning false
03-Mar-2010 19:11:24 [---] [proxy_debug]: setting up proxy 10.98.20.23:8080
03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Info: Connection #0 seems to be dead!
03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Info: Closing connection #0
03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Info: timeout on name lookup is not supported
03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Info: About to connect() to proxy 10.98.20.23 port 8080 (#0)
03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Info: Trying 10.98.20.23...
03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Info: Connected to 10.98.20.23 (10.98.20.23) port 8080 (#0)
03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Sent header to server: POST http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi HTTP/1.103-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Sent header to server: User-Agent: BOINC client (windows_x86_64 6.10.35)03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Sent header to server: Host: setiboinc.ssl.berkeley.edu03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Sent header to server: Accept: */*03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Sent header to server: Accept-Encoding: deflate, gzip03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Sent header to server: Proxy-Connection: Keep-Alive03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Sent header to server: Content-Type: application/x-www-form-urlencoded03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Sent header to server: Content-Length: 229694203-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Sent header to server: Expect: 100-continue03-Mar-2010 19:11:25 [---] [http_debug] [ID#1] Sent header to server: 03-Mar-2010 19:11:26 [---] [http_debug] [ID#1] Info: Done waiting for 100-continue
03-Mar-2010 19:11:27 [---] [http_debug] [ID#1] Received header from server: HTTP/1.1 403 Forbidden03-Mar-2010 19:11:27 [---] [http_debug] [ID#1] Received header from server: Content-Length: 133303-Mar-2010 19:11:27 [---] [http_debug] [ID#1] Received header from server: Content-Type: text/html; charset=ISO-8859-103-Mar-2010 19:11:27 [---] [http_debug] [ID#1] Received header from server: Pragma: no-cache03-Mar-2010 19:11:27 [---] [http_debug] [ID#1] Received header from server: Proxy-Connection: keep-alive03-Mar-2010 19:11:27 [---] [http_debug] [ID#1] Received header from server: Via: 1.1 webwasher (Webwasher 6.8.6.6257)03-Mar-2010 19:11:27 [---] [http_debug] [ID#1] Received header from server: X-Error-Name: parametertoobig03-Mar-2010 19:11:27 [---] [http_debug] [ID#1] Received header from server: 03-Mar-2010 19:11:27 [---] [http_xfer_debug] [ID#1] HTTP: wrote 1333 bytes
03-Mar-2010 19:11:27 [---] [http_debug] [ID#1] Info: Expire cleared
03-Mar-2010 19:11:27 [---] [http_debug] [ID#1] Info: Connection #0 to host 10.98.20.23 left intact
03-Mar-2010 19:11:30 [SETI@home] Scheduler request failed: Error 403
03-Mar-2010 19:11:30 [SETI@home] [sched_op_debug] Deferring communication for 1 min 0 sec
03-Mar-2010 19:11:30 [SETI@home] [sched_op_debug] Reason: Scheduler request failed

ID: 975603 · Report as offensive
Profile startrekforever

Send message
Joined: 5 Dec 99
Posts: 36
Credit: 17,772,420
RAC: 0
United States
Message 975797 - Posted: 5 Mar 2010, 0:41:19 UTC

Was the Apache 'request' size increased, or did I need a 'response' size increased on your end?

If the sizes are large enough I guess it must be my company's WebWasher proxy that is causing the issue.

The issue still persists and I will not be making any changes to my server (resetting the project) until Monday.

Thanks.

ID: 975797 · Report as offensive
Profile startrekforever

Send message
Joined: 5 Dec 99
Posts: 36
Credit: 17,772,420
RAC: 0
United States
Message 976787 - Posted: 8 Mar 2010, 19:58:19 UTC

I have reset my project, assume issue is not solvable with company's WebWasher, thanks for trying. Closing case.
ID: 976787 · Report as offensive

Questions and Answers : Windows : Scheduler request failed: Error 403


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.