Panic Mode On (70) Server problems?


log in

Advanced search

Message boards : Number crunching : Panic Mode On (70) Server problems?

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 11 · Next
Author Message
zoom314Project donor
Avatar
Send message
Joined: 30 Nov 03
Posts: 46519
Credit: 36,860,761
RAC: 5,144
United States
Message 1202829 - Posted: 6 Mar 2012, 3:29:55 UTC - in response to Message 1202828.
Last modified: 6 Mar 2012, 3:30:34 UTC

Using 10 does how work for 6.12.34 and it would have to work several times to send all 189!

06/03/2012 03:12:55 | SETI@home | Reporting 189 completed tasks, requesting new tasks for CPU
06/03/2012 03:12:58 | SETI@home | Scheduler request failed: HTTP internal server error
06/03/2012 03:13:40 | SETI@home | Reporting 189 completed tasks, requesting new tasks for CPU
06/03/2012 03:13:44 | SETI@home | Scheduler request failed: HTTP internal server error
06/03/2012 03:13:59 | SETI@home | Reporting 10 completed tasks, requesting new tasks for CPU
06/03/2012 03:14:04 | SETI@home | Scheduler request failed: HTTP internal server error
06/03/2012 03:14:14 | SETI@home | Reporting 10 completed tasks, requesting new tasks for CPU
06/03/2012 03:14:17 | SETI@home | Scheduler request failed: HTTP internal server error

Well in between My last post and this one I had to reinstall My video driver from safe mode, that is I went from 275.50 to 275.33 as I'd boot up into Win7 Pro x64 and then get to the desktop and then bam, BSOD, twice, possibly a corrupted 275.50 driver, maybe. Weird.

Of course I still get the HTTP internal server error...
____________
My Facebook, War Commander, 2015

Profile Brother Frank
Send message
Joined: 10 Dec 11
Posts: 26
Credit: 15,142,410
RAC: 0
United States
Message 1202877 - Posted: 6 Mar 2012, 5:58:37 UTC - in response to Message 1202829.

I've got the same problems as you are reporting on my fastest machine. The one right next to it, a desktop core duo is having no problem at all uploading completed jobs. Everything was going so well until sometime this afternoon and now I get the error you are reporting to: with 166 completed jobs that can't be sent in. I hope SETI staff are aware of this. You aren't alone I'm sure in having this issue.

Brother Frank


Scheduler request failed: HTTP internal server error

Profile Raistmer
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 16 Jun 01
Posts: 3502
Credit: 47,730,279
RAC: 46,964
Russia
Message 1202878 - Posted: 6 Mar 2012, 6:01:37 UTC

06/03/2012 09:20:28 SETI@home Sending scheduler request: To fetch work.
06/03/2012 09:20:28 SETI@home Reporting 342 completed tasks, requesting new tasks for GPU
06/03/2012 09:20:32 SETI@home Scheduler request failed: HTTP internal server error

Profile Jim_SProject donor
Avatar
Send message
Joined: 23 Feb 00
Posts: 4526
Credit: 18,826,127
RAC: 8,669
United States
Message 1202879 - Posted: 6 Mar 2012, 6:02:23 UTC - in response to Message 1202877.

I've got the same problems as you are reporting on my fastest machine. The one right next to it, a desktop core duo is having no problem at all uploading completed jobs. Everything was going so well until sometime this afternoon and now I get the error you are reporting to: with 166 completed jobs that can't be sent in. I hope SETI staff are aware of this. You aren't alone I'm sure in having this issue.

Brother Frank


Scheduler request failed: HTTP internal server error

Same here...Scheduler request failed: HTTP internal server error.
____________

I Desire Peace and Justice, Jim Scott (Mod-Ret.)

Profile Gatekeeper
Avatar
Send message
Joined: 14 Jul 04
Posts: 887
Credit: 176,479,616
RAC: 0
United States
Message 1202880 - Posted: 6 Mar 2012, 6:04:15 UTC

As of now (10PM PST), I have over 1700 WU's to report on my 3 rigs. Still no change in the "internal server error" message. 2 of my 3 rigs will run out of GPU work before Tuesday maintenance starts. I just KNEW mentioning to my wife that all my rigs had been at their "limits" for almost a week was a bad thing to do.
____________

Profile Anthony Arbuzoff
Volunteer tester
Avatar
Send message
Joined: 6 Apr 00
Posts: 204
Credit: 2,547,453
RAC: 1,348
Russia
Message 1202881 - Posted: 6 Mar 2012, 6:15:31 UTC - in response to Message 1202878.

06/03/2012 09:20:28 SETI@home Sending scheduler request: To fetch work.
06/03/2012 09:20:28 SETI@home Reporting 342 completed tasks, requesting new tasks for GPU
06/03/2012 09:20:32 SETI@home Scheduler request failed: HTTP internal server error


То ли от провайдера зависит, то ли от числа готовых заданий, то ли ошибка идёт если одновременно с готовыми идёт запрос новых заданий. У меня готовые задания отлично рапортуются.


06.03.2012 9:52:22 | SETI@home | Computation for task 09my11ad.27402.23385.14.10.21_2 finished
06.03.2012 9:52:22 | SETI@home | Starting task 02oc11ab.21470.30066.11.10.135_2 using setiathome_enhanced version 608
06.03.2012 9:52:24 | SETI@home | Started upload of 09my11ad.27402.23385.14.10.21_2_0
06.03.2012 9:52:35 | SETI@home | Finished upload of 09my11ad.27402.23385.14.10.21_2_0
06.03.2012 9:52:35 | SETI@home | Sending scheduler request: To report completed tasks.
06.03.2012 9:52:35 | SETI@home | Reporting 1 completed tasks, not requesting new tasks
06.03.2012 9:52:42 | SETI@home | Scheduler request completed

____________

Profile Anthony Arbuzoff
Volunteer tester
Avatar
Send message
Joined: 6 Apr 00
Posts: 204
Credit: 2,547,453
RAC: 1,348
Russia
Message 1202882 - Posted: 6 Mar 2012, 6:20:45 UTC - in response to Message 1202881.

So, if you can't to report completed tasks, try to checkpoint "don't request new tasks". May be the problem is there?
____________

zoom314Project donor
Avatar
Send message
Joined: 30 Nov 03
Posts: 46519
Credit: 36,860,761
RAC: 5,144
United States
Message 1202884 - Posted: 6 Mar 2012, 6:25:41 UTC - in response to Message 1202882.

So, if you can't to report completed tasks, try to checkpoint "don't request new tasks". May be the problem is there?

The problem is a server problem, this is not a users PC problem, as before the outage this did not happen.
____________
My Facebook, War Commander, 2015

Profile Cyryl
Avatar
Send message
Joined: 12 Oct 05
Posts: 6
Credit: 10,809,904
RAC: 0
United States
Message 1202885 - Posted: 6 Mar 2012, 6:30:21 UTC

Not doin' so hot...

-------------------------------------------------------------------------------------------------------------------------------------
3/6/2012 12:17:26 AM | | Starting BOINC client version 6.12.34 for windows_x86_64
3/6/2012 12:17:26 AM | | log flags: file_xfer, sched_ops, task
3/6/2012 12:17:26 AM | | Libraries: libcurl/7.21.6 OpenSSL/1.0.0d zlib/1.2.5
3/6/2012 12:17:26 AM | | Data directory: D:\BOINC\_Data
3/6/2012 12:17:26 AM | | Running under account Cyryl
3/6/2012 12:17:26 AM | | Processor: 8 GenuineIntel Intel(R) Core(TM) i7 CPU 950 @ 3.70 GHz [Family 6 Model 26 Stepping 5]
3/6/2012 12:17:26 AM | | Processor: 256.00 KB cache
3/6/2012 12:17:26 AM | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 syscall nx lm vmx tm2 popcnt pbe
3/6/2012 12:17:26 AM | | OS: Microsoft Windows 7: Ultimate x64 Edition, Service Pack 1, (06.01.7601.00)
3/6/2012 12:17:26 AM | | Memory: 5.99 GB physical, 11.98 GB virtual
3/6/2012 12:17:26 AM | | Disk: 500.00 GB total, 490.55 GB free
3/6/2012 12:17:26 AM | | Local time is UTC -6 hours
3/6/2012 12:17:26 AM | | NVIDIA GPU 0: GeForce GTX 570 (driver version 29573, CUDA version 4020, compute capability 2.0, 1280MB, 1530 GFLOPS peak)
3/6/2012 12:17:26 AM | | NVIDIA GPU 1: GeForce GTX 570 (driver version 29573, CUDA version 4020, compute capability 2.0, 1280MB, 1530 GFLOPS peak)
3/6/2012 12:17:26 AM | SETI@home | Found app_info.xml; using anonymous platform
3/6/2012 12:17:26 AM | Einstein@Home | URL http://einstein.phys.uwm.edu/; Computer ID 3962129; resource share 100
3/6/2012 12:17:26 AM | SETI@home | URL http://setiathome.berkeley.edu/; Computer ID 6424486; resource share 200
3/6/2012 12:17:26 AM | | General prefs: from http://www.worldcommunitygrid.org/ (last modified 11-Oct-2010 05:08:09)
3/6/2012 12:17:26 AM | | Host location: none
3/6/2012 12:17:26 AM | | General prefs: using your defaults
3/6/2012 12:17:26 AM | | Reading preferences override file
3/6/2012 12:17:26 AM | | Preferences:
3/6/2012 12:17:26 AM | | max memory usage when active: 3067.55MB
3/6/2012 12:17:26 AM | | max memory usage when idle: 5521.60MB
3/6/2012 12:17:26 AM | | max disk usage: 10.00GB
3/6/2012 12:17:26 AM | | suspend work if non-BOINC CPU load exceeds 70 %
3/6/2012 12:17:26 AM | | (to change preferences, visit the web site of an attached project, or select Preferences in the Manager)
3/6/2012 12:17:26 AM | | Not using a proxy
3/6/2012 12:17:27 AM | SETI@home | Restarting task 16dc11ac.13322.5793.5.10.253_0 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 16dc11ac.27435.13973.6.10.242_1 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 27my11ac.25449.288320.10.10.52_2 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 17dc11aa.11150.6202.14.10.38_2 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 18dc11ab.12201.1294.4.10.238_0 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 18dc11ab.12201.1294.4.10.235_1 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 18dc11ab.12201.1294.4.10.224_1 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 16dc11ac.16903.476.7.10.46_0 using setiathome_enhanced version 603
3/6/2012 12:17:59 AM | SETI@home | update requested by user
3/6/2012 12:18:02 AM | SETI@home | Sending scheduler request: Requested by user.
3/6/2012 12:18:02 AM | SETI@home | Reporting 419 completed tasks, requesting new tasks for CPU and NVIDIA GPU
3/6/2012 12:18:05 AM | SETI@home | Scheduler request failed: HTTP internal server error

-------------------------------------------------------------------------------------------------------------------------------------
Been like this all day.

It's been such a roller coaster ride the last year. UP and DOWN. Many times I get to being just short of 30,000 average per day...and sincerely hope to get there. But then the servers have issues or they continually run out of work units and my system goes dry until I'm back down into the sub-10K range. *sighs*

I've recently reached a combined total of 10,000,000. It was a bit of a goal. But once my main gaming rig (shown above) reaches that mark on it's own, I'm thinking of retiring from SETI. It's become a bit of a headache and frustration. Also... It would really help alleviate some power distribution issues I've been experiencing in my office. (Not to mention...save me some money every month. Need to start saving for a new card. LOL!) 10 million is quite a hefty number and I have been very dedicated in my efforts to remain vigilant with my servers here. Seems like a nice, round number to stop at...if I'm going to. :P

Plus... I'm still unaware of the long-term effects of using the GPU's for it. They get very hot. Backplates for the two 570's really helped disperse the temps quite a bit. But I am still concerned about running it at 70C on a constant basis. I can't help but think that it'll shorten the life of the GPU.

Otherwise... I'm not really aware of whether or not this project can really keep on like this... Although I do appreciate the obvious efforts that the staff have put forward. Please don't misunderstand. :)

Opinions anyone?
____________
2U Rackmount Server:
Tyan 2912E, 2 Opteron DC 2220's @ 2.8GHz, 8GB DDR2 667 ECC Reg

Gaming rig: ASUS Sabertooth X58, i7 950, 6GB Corsair DDR3, 2x EVGA GTX 570 Superclocked in SLI (012-P3-1572-AR), Corsair AX1200

Using Lunatics Win64 SETI

Profile Anthony Arbuzoff
Volunteer tester
Avatar
Send message
Joined: 6 Apr 00
Posts: 204
Credit: 2,547,453
RAC: 1,348
Russia
Message 1202886 - Posted: 6 Mar 2012, 6:38:47 UTC - in response to Message 1202884.

So, if you can't to report completed tasks, try to checkpoint "don't request new tasks". May be the problem is there?

The problem is a server problem, this is not a users PC problem, as before the outage this did not happen.


Then why some users can report their completed tasks easily, and some users can't?
____________

Profile Ripping Silk
Send message
Joined: 18 May 99
Posts: 1
Credit: 14,159,367
RAC: 0
New Zealand
Message 1202887 - Posted: 6 Mar 2012, 6:45:36 UTC - in response to Message 1202886.

I have 286 WU's to upload. These have been stacking up for over 12 hours.
What I am noticing now, is that single, just completed WU's are being uploaded successfully while the 286 in the queue remain.
____________

Grant (SSSF)
Send message
Joined: 19 Aug 99
Posts: 5864
Credit: 60,572,549
RAC: 47,675
Australia
Message 1202888 - Posted: 6 Mar 2012, 6:47:08 UTC - in response to Message 1202887.

What I am noticing now, is that single, just completed WU's are being uploaded successfully while the 286 in the queue remain.

There's no problem uploading completed work, the problem is with reporting it.

____________
Grant
Darwin NT.

Cosmic_Ocean
Avatar
Send message
Joined: 23 Dec 00
Posts: 2287
Credit: 8,811,630
RAC: 4,056
United States
Message 1202889 - Posted: 6 Mar 2012, 7:35:41 UTC
Last modified: 6 Mar 2012, 7:37:48 UTC

Having just read through all the issues here.. I'm reporting that I have no issues on my end.

Times are UTC-5:
2012-03-05 16:05:38|SETI@home|Sending scheduler request: To fetch work. Requesting 1838696 seconds of work, reporting 2 completed tasks
2012-03-05 16:05:43|SETI@home|Scheduler request succeeded: got 0 new tasks
2012-03-05 16:05:43|SETI@home|Message from server: Project has no tasks available
2012-03-05 20:39:55|SETI@home|Sending scheduler request: To fetch work. Requesting 1885941 seconds of work, reporting 1 completed tasks
2012-03-05 20:40:00|SETI@home|Scheduler request succeeded: got 0 new tasks
2012-03-05 20:40:00|SETI@home|Message from server: Project has no tasks available
2012-03-05 23:37:26|SETI@home|Sending scheduler request: To fetch work. Requesting 1916024 seconds of work, reporting 1 completed tasks
2012-03-05 23:37:31|SETI@home|Scheduler request succeeded: got 0 new tasks
2012-03-05 23:37:31|SETI@home|Message from server: Project has no tasks available
2012-03-06 02:33:30|SETI@home|Sending scheduler request: To fetch work. Requesting 1949026 seconds of work, reporting 0 completed tasks
2012-03-06 02:33:35|SETI@home|Scheduler request succeeded: got 1 new tasks
2012-03-06 02:33:37|SETI@home|Started download of ap_12oc11ac_B3_P0_00006_20120209_32708.wu
2012-03-06 02:33:58|SETI@home|Finished download of ap_12oc11ac_B3_P0_00006_20120209_32708.wu (~390KB/sec)


Since ~4PM for me, I have zero work requests that failed or had errors, out of the ~30 requests that have been made.
____________

Linux laptop uptime: 1484d 22h 42m
Ended due to UPS failure, found 14 hours after the fact

Highlander
Avatar
Send message
Joined: 5 Oct 99
Posts: 145
Credit: 31,419,917
RAC: 9,451
Germany
Message 1202890 - Posted: 6 Mar 2012, 7:37:24 UTC
Last modified: 6 Mar 2012, 7:46:04 UTC

Strange, i got not the 500 error here but "Timeout was reached" error. So first contact to server and upload of reporting data takes place but no further reaction of schedular happens (from my point of view). Cricket shows this also, low traffic.

€:
Cosmic_Ocean: nice to see, that there is still life in server closet ^^
____________

rob smithProject donor
Volunteer tester
Send message
Joined: 7 Mar 03
Posts: 8535
Credit: 59,378,554
RAC: 84,284
United Kingdom
Message 1202892 - Posted: 6 Mar 2012, 7:45:25 UTC

A quick look at the crickets shows that during yesterday evening there were a couple of hours of "decent" service, but it all went to bits around midnight, and since then downloads have effectively been stalled, while uploads are continuing more or less normally.
I guess we'll have to wait until the tyre kicker gets in and does his rounds, although being Tuesday today it might be a slightly longer wait than that...


Strange - Just a passing thought, midnight in the UK is about 4pm in Berkeley which must be "about time to go home". Did something fail just as they left, or was something turned off overnight??
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Highlander
Avatar
Send message
Joined: 5 Oct 99
Posts: 145
Credit: 31,419,917
RAC: 9,451
Germany
Message 1202897 - Posted: 6 Mar 2012, 8:15:47 UTC

now after midnight berkeley time, i get also "internal server error" for the first time, looks like some automatic scripts/processes had been re/started.
____________

Profile Wiggo
Avatar
Send message
Joined: 24 Jan 00
Posts: 7351
Credit: 96,866,508
RAC: 66,739
Australia
Message 1202898 - Posted: 6 Mar 2012, 8:18:07 UTC - in response to Message 1202895.

Nothing much anyone can do with the problem except wait until tomorrow's outage by which time the guy's in the lab should have it sorted out. ;)

Cheers.
____________

Profile Wiggo
Avatar
Send message
Joined: 24 Jan 00
Posts: 7351
Credit: 96,866,508
RAC: 66,739
Australia
Message 1202900 - Posted: 6 Mar 2012, 8:26:34 UTC - in response to Message 1202899.

At least with my little 9800GT's/GTX+ I'm good for a another couple of days.

Cheers.
____________

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 11 · Next

Message boards : Number crunching : Panic Mode On (70) Server problems?

Copyright © 2014 University of California