Panic Mode On (70) Server problems?

Message boards : Number crunching : Panic Mode On (70) Server problems?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 9 · Next

AuthorMessage
MikeN

Send message
Joined: 24 Jan 11
Posts: 319
Credit: 64,719,409
RAC: 85
United Kingdom
Message 1202814 - Posted: 6 Mar 2012, 1:28:13 UTC - in response to Message 1202771.  

I, too, cannot report 100 MB tasks on the fastest computer.
Others have reported normally. This is an outrage. :@

NNT, even reporting one task at a time doesn't help and "last contact" timestamp doesn't update.


I have just checked my 4 PCs. Two are in my office in the UK and one has managed to connect OK, but the other just gets HTTP internal server error when I try to force to connect via logmein. Uploads and downloads going through fine though. Mt home PC in the UK also cannot connect, but my laptop which is with me in Florida at present connected and reported 5 tasks at its first attempt this evening!

NB the office machine that cannot connect runs 6.10.58 but all my other machines run the latest update of 6.12.??, so the problem does not seem to be site or seti version specific.
ID: 1202814 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65738
Credit: 55,293,173
RAC: 49
United States
Message 1202818 - Posted: 6 Mar 2012, 1:47:21 UTC - in response to Message 1202814.  

I, too, cannot report 100 MB tasks on the fastest computer.
Others have reported normally. This is an outrage. :@

NNT, even reporting one task at a time doesn't help and "last contact" timestamp doesn't update.


I have just checked my 4 PCs. Two are in my office in the UK and one has managed to connect OK, but the other just gets HTTP internal server error when I try to force to connect via logmein. Uploads and downloads going through fine though. Mt home PC in the UK also cannot connect, but my laptop which is with me in Florida at present connected and reported 5 tasks at its first attempt this evening!

NB the office machine that cannot connect runs 6.10.58 but all my other machines run the latest update of 6.12.??, so the problem does not seem to be site or seti version specific.

I use 6.10.58 too.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1202818 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65738
Credit: 55,293,173
RAC: 49
United States
Message 1202826 - Posted: 6 Mar 2012, 2:33:40 UTC

So when is this going to be fixed? I've got 71wu's to report and I can't report, which means no downloads either...

I just get HTTP internal server errors when I try to do an update.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1202826 · Report as offensive
Profile red-ray
Avatar

Send message
Joined: 24 Jun 99
Posts: 308
Credit: 9,029,848
RAC: 0
United Kingdom
Message 1202828 - Posted: 6 Mar 2012, 3:17:59 UTC - in response to Message 1202804.  

Using 10 does how work for 6.12.34 and it would have to work several times to send all 189!

06/03/2012 03:12:55 | SETI@home | Reporting 189 completed tasks, requesting new tasks for CPU
06/03/2012 03:12:58 | SETI@home | Scheduler request failed: HTTP internal server error
06/03/2012 03:13:40 | SETI@home | Reporting 189 completed tasks, requesting new tasks for CPU
06/03/2012 03:13:44 | SETI@home | Scheduler request failed: HTTP internal server error
06/03/2012 03:13:59 | SETI@home | Reporting 10 completed tasks, requesting new tasks for CPU
06/03/2012 03:14:04 | SETI@home | Scheduler request failed: HTTP internal server error
06/03/2012 03:14:14 | SETI@home | Reporting 10 completed tasks, requesting new tasks for CPU
06/03/2012 03:14:17 | SETI@home | Scheduler request failed: HTTP internal server error
ID: 1202828 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65738
Credit: 55,293,173
RAC: 49
United States
Message 1202829 - Posted: 6 Mar 2012, 3:29:55 UTC - in response to Message 1202828.  
Last modified: 6 Mar 2012, 3:30:34 UTC

Using 10 does how work for 6.12.34 and it would have to work several times to send all 189!

06/03/2012 03:12:55 | SETI@home | Reporting 189 completed tasks, requesting new tasks for CPU
06/03/2012 03:12:58 | SETI@home | Scheduler request failed: HTTP internal server error
06/03/2012 03:13:40 | SETI@home | Reporting 189 completed tasks, requesting new tasks for CPU
06/03/2012 03:13:44 | SETI@home | Scheduler request failed: HTTP internal server error
06/03/2012 03:13:59 | SETI@home | Reporting 10 completed tasks, requesting new tasks for CPU
06/03/2012 03:14:04 | SETI@home | Scheduler request failed: HTTP internal server error
06/03/2012 03:14:14 | SETI@home | Reporting 10 completed tasks, requesting new tasks for CPU
06/03/2012 03:14:17 | SETI@home | Scheduler request failed: HTTP internal server error

Well in between My last post and this one I had to reinstall My video driver from safe mode, that is I went from 275.50 to 275.33 as I'd boot up into Win7 Pro x64 and then get to the desktop and then bam, BSOD, twice, possibly a corrupted 275.50 driver, maybe. Weird.

Of course I still get the HTTP internal server error...
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1202829 · Report as offensive
Profile Brother Frank

Send message
Joined: 10 Dec 11
Posts: 26
Credit: 15,142,410
RAC: 0
United States
Message 1202877 - Posted: 6 Mar 2012, 5:58:37 UTC - in response to Message 1202829.  

I've got the same problems as you are reporting on my fastest machine. The one right next to it, a desktop core duo is having no problem at all uploading completed jobs. Everything was going so well until sometime this afternoon and now I get the error you are reporting to: with 166 completed jobs that can't be sent in. I hope SETI staff are aware of this. You aren't alone I'm sure in having this issue.

Brother Frank


Scheduler request failed: HTTP internal server error
ID: 1202877 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1202878 - Posted: 6 Mar 2012, 6:01:37 UTC

06/03/2012 09:20:28 SETI@home Sending scheduler request: To fetch work.
06/03/2012 09:20:28 SETI@home Reporting 342 completed tasks, requesting new tasks for GPU
06/03/2012 09:20:32 SETI@home Scheduler request failed: HTTP internal server error
ID: 1202878 · Report as offensive
Profile Jim_S
Avatar

Send message
Joined: 23 Feb 00
Posts: 4705
Credit: 64,560,357
RAC: 31
United States
Message 1202879 - Posted: 6 Mar 2012, 6:02:23 UTC - in response to Message 1202877.  

I've got the same problems as you are reporting on my fastest machine. The one right next to it, a desktop core duo is having no problem at all uploading completed jobs. Everything was going so well until sometime this afternoon and now I get the error you are reporting to: with 166 completed jobs that can't be sent in. I hope SETI staff are aware of this. You aren't alone I'm sure in having this issue.

Brother Frank


Scheduler request failed: HTTP internal server error

Same here...Scheduler request failed: HTTP internal server error.

I Desire Peace and Justice, Jim Scott (Mod-Ret.)
ID: 1202879 · Report as offensive
Profile Gatekeeper
Avatar

Send message
Joined: 14 Jul 04
Posts: 887
Credit: 176,479,616
RAC: 0
United States
Message 1202880 - Posted: 6 Mar 2012, 6:04:15 UTC

As of now (10PM PST), I have over 1700 WU's to report on my 3 rigs. Still no change in the "internal server error" message. 2 of my 3 rigs will run out of GPU work before Tuesday maintenance starts. I just KNEW mentioning to my wife that all my rigs had been at their "limits" for almost a week was a bad thing to do.
ID: 1202880 · Report as offensive
Profile Belthazor
Volunteer tester
Avatar

Send message
Joined: 6 Apr 00
Posts: 219
Credit: 10,373,795
RAC: 13
Russia
Message 1202881 - Posted: 6 Mar 2012, 6:15:31 UTC - in response to Message 1202878.  

06/03/2012 09:20:28 SETI@home Sending scheduler request: To fetch work.
06/03/2012 09:20:28 SETI@home Reporting 342 completed tasks, requesting new tasks for GPU
06/03/2012 09:20:32 SETI@home Scheduler request failed: HTTP internal server error


То ли от провайдера зависит, то ли от числа готовых заданий, то ли ошибка идёт если одновременно с готовыми идёт запрос новых заданий. У меня готовые задания отлично рапортуются.


06.03.2012 9:52:22 | SETI@home | Computation for task 09my11ad.27402.23385.14.10.21_2 finished
06.03.2012 9:52:22 | SETI@home | Starting task 02oc11ab.21470.30066.11.10.135_2 using setiathome_enhanced version 608
06.03.2012 9:52:24 | SETI@home | Started upload of 09my11ad.27402.23385.14.10.21_2_0
06.03.2012 9:52:35 | SETI@home | Finished upload of 09my11ad.27402.23385.14.10.21_2_0
06.03.2012 9:52:35 | SETI@home | Sending scheduler request: To report completed tasks.
06.03.2012 9:52:35 | SETI@home | Reporting 1 completed tasks, not requesting new tasks
06.03.2012 9:52:42 | SETI@home | Scheduler request completed

ID: 1202881 · Report as offensive
Profile Belthazor
Volunteer tester
Avatar

Send message
Joined: 6 Apr 00
Posts: 219
Credit: 10,373,795
RAC: 13
Russia
Message 1202882 - Posted: 6 Mar 2012, 6:20:45 UTC - in response to Message 1202881.  

So, if you can't to report completed tasks, try to checkpoint "don't request new tasks". May be the problem is there?
ID: 1202882 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 65738
Credit: 55,293,173
RAC: 49
United States
Message 1202884 - Posted: 6 Mar 2012, 6:25:41 UTC - in response to Message 1202882.  

So, if you can't to report completed tasks, try to checkpoint "don't request new tasks". May be the problem is there?

The problem is a server problem, this is not a users PC problem, as before the outage this did not happen.
The T1 Trust, PRR T1 Class 4-4-4-4 #5550, 1 of America's First HST's
ID: 1202884 · Report as offensive
Profile Cyryl
Avatar

Send message
Joined: 12 Oct 05
Posts: 6
Credit: 11,144,664
RAC: 0
United States
Message 1202885 - Posted: 6 Mar 2012, 6:30:21 UTC

Not doin' so hot...

-------------------------------------------------------------------------------------------------------------------------------------
3/6/2012 12:17:26 AM | | Starting BOINC client version 6.12.34 for windows_x86_64
3/6/2012 12:17:26 AM | | log flags: file_xfer, sched_ops, task
3/6/2012 12:17:26 AM | | Libraries: libcurl/7.21.6 OpenSSL/1.0.0d zlib/1.2.5
3/6/2012 12:17:26 AM | | Data directory: D:\BOINC\_Data
3/6/2012 12:17:26 AM | | Running under account Cyryl
3/6/2012 12:17:26 AM | | Processor: 8 GenuineIntel Intel(R) Core(TM) i7 CPU 950 @ 3.70 GHz [Family 6 Model 26 Stepping 5]
3/6/2012 12:17:26 AM | | Processor: 256.00 KB cache
3/6/2012 12:17:26 AM | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 syscall nx lm vmx tm2 popcnt pbe
3/6/2012 12:17:26 AM | | OS: Microsoft Windows 7: Ultimate x64 Edition, Service Pack 1, (06.01.7601.00)
3/6/2012 12:17:26 AM | | Memory: 5.99 GB physical, 11.98 GB virtual
3/6/2012 12:17:26 AM | | Disk: 500.00 GB total, 490.55 GB free
3/6/2012 12:17:26 AM | | Local time is UTC -6 hours
3/6/2012 12:17:26 AM | | NVIDIA GPU 0: GeForce GTX 570 (driver version 29573, CUDA version 4020, compute capability 2.0, 1280MB, 1530 GFLOPS peak)
3/6/2012 12:17:26 AM | | NVIDIA GPU 1: GeForce GTX 570 (driver version 29573, CUDA version 4020, compute capability 2.0, 1280MB, 1530 GFLOPS peak)
3/6/2012 12:17:26 AM | SETI@home | Found app_info.xml; using anonymous platform
3/6/2012 12:17:26 AM | Einstein@Home | URL http://einstein.phys.uwm.edu/; Computer ID 3962129; resource share 100
3/6/2012 12:17:26 AM | SETI@home | URL http://setiathome.berkeley.edu/; Computer ID 6424486; resource share 200
3/6/2012 12:17:26 AM | | General prefs: from http://www.worldcommunitygrid.org/ (last modified 11-Oct-2010 05:08:09)
3/6/2012 12:17:26 AM | | Host location: none
3/6/2012 12:17:26 AM | | General prefs: using your defaults
3/6/2012 12:17:26 AM | | Reading preferences override file
3/6/2012 12:17:26 AM | | Preferences:
3/6/2012 12:17:26 AM | | max memory usage when active: 3067.55MB
3/6/2012 12:17:26 AM | | max memory usage when idle: 5521.60MB
3/6/2012 12:17:26 AM | | max disk usage: 10.00GB
3/6/2012 12:17:26 AM | | suspend work if non-BOINC CPU load exceeds 70 %
3/6/2012 12:17:26 AM | | (to change preferences, visit the web site of an attached project, or select Preferences in the Manager)
3/6/2012 12:17:26 AM | | Not using a proxy
3/6/2012 12:17:27 AM | SETI@home | Restarting task 16dc11ac.13322.5793.5.10.253_0 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 16dc11ac.27435.13973.6.10.242_1 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 27my11ac.25449.288320.10.10.52_2 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 17dc11aa.11150.6202.14.10.38_2 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 18dc11ab.12201.1294.4.10.238_0 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 18dc11ab.12201.1294.4.10.235_1 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 18dc11ab.12201.1294.4.10.224_1 using setiathome_enhanced version 603
3/6/2012 12:17:27 AM | SETI@home | Restarting task 16dc11ac.16903.476.7.10.46_0 using setiathome_enhanced version 603
3/6/2012 12:17:59 AM | SETI@home | update requested by user
3/6/2012 12:18:02 AM | SETI@home | Sending scheduler request: Requested by user.
3/6/2012 12:18:02 AM | SETI@home | Reporting 419 completed tasks, requesting new tasks for CPU and NVIDIA GPU
3/6/2012 12:18:05 AM | SETI@home | Scheduler request failed: HTTP internal server error

-------------------------------------------------------------------------------------------------------------------------------------
Been like this all day.

It's been such a roller coaster ride the last year. UP and DOWN. Many times I get to being just short of 30,000 average per day...and sincerely hope to get there. But then the servers have issues or they continually run out of work units and my system goes dry until I'm back down into the sub-10K range. *sighs*

I've recently reached a combined total of 10,000,000. It was a bit of a goal. But once my main gaming rig (shown above) reaches that mark on it's own, I'm thinking of retiring from SETI. It's become a bit of a headache and frustration. Also... It would really help alleviate some power distribution issues I've been experiencing in my office. (Not to mention...save me some money every month. Need to start saving for a new card. LOL!) 10 million is quite a hefty number and I have been very dedicated in my efforts to remain vigilant with my servers here. Seems like a nice, round number to stop at...if I'm going to. :P

Plus... I'm still unaware of the long-term effects of using the GPU's for it. They get very hot. Backplates for the two 570's really helped disperse the temps quite a bit. But I am still concerned about running it at 70C on a constant basis. I can't help but think that it'll shorten the life of the GPU.

Otherwise... I'm not really aware of whether or not this project can really keep on like this... Although I do appreciate the obvious efforts that the staff have put forward. Please don't misunderstand. :)

Opinions anyone?
2U Rackmount Server:
Tyan 2912E, 2 Opteron DC 2220's @ 2.8GHz, 8GB DDR2 667 ECC Reg

Gaming rig: ASUS Sabertooth X58, i7 950, 6GB Corsair DDR3, 2x EVGA GTX 570 Superclocked in SLI (012-P3-1572-AR), Corsair AX1200

Using Lunatics Win64 SETI
ID: 1202885 · Report as offensive
Profile Belthazor
Volunteer tester
Avatar

Send message
Joined: 6 Apr 00
Posts: 219
Credit: 10,373,795
RAC: 13
Russia
Message 1202886 - Posted: 6 Mar 2012, 6:38:47 UTC - in response to Message 1202884.  

So, if you can't to report completed tasks, try to checkpoint "don't request new tasks". May be the problem is there?

The problem is a server problem, this is not a users PC problem, as before the outage this did not happen.


Then why some users can report their completed tasks easily, and some users can't?
ID: 1202886 · Report as offensive
Profile Ripping Silk

Send message
Joined: 18 May 99
Posts: 1
Credit: 14,159,367
RAC: 0
New Zealand
Message 1202887 - Posted: 6 Mar 2012, 6:45:36 UTC - in response to Message 1202886.  

I have 286 WU's to upload. These have been stacking up for over 12 hours.
What I am noticing now, is that single, just completed WU's are being uploaded successfully while the 286 in the queue remain.
ID: 1202887 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13731
Credit: 208,696,464
RAC: 304
Australia
Message 1202888 - Posted: 6 Mar 2012, 6:47:08 UTC - in response to Message 1202887.  

What I am noticing now, is that single, just completed WU's are being uploaded successfully while the 286 in the queue remain.

There's no problem uploading completed work, the problem is with reporting it.

Grant
Darwin NT
ID: 1202888 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1202889 - Posted: 6 Mar 2012, 7:35:41 UTC
Last modified: 6 Mar 2012, 7:37:48 UTC

Having just read through all the issues here.. I'm reporting that I have no issues on my end.

Times are UTC-5:
2012-03-05 16:05:38|SETI@home|Sending scheduler request: To fetch work. Requesting 1838696 seconds of work, reporting 2 completed tasks
2012-03-05 16:05:43|SETI@home|Scheduler request succeeded: got 0 new tasks
2012-03-05 16:05:43|SETI@home|Message from server: Project has no tasks available
2012-03-05 20:39:55|SETI@home|Sending scheduler request: To fetch work. Requesting 1885941 seconds of work, reporting 1 completed tasks
2012-03-05 20:40:00|SETI@home|Scheduler request succeeded: got 0 new tasks
2012-03-05 20:40:00|SETI@home|Message from server: Project has no tasks available
2012-03-05 23:37:26|SETI@home|Sending scheduler request: To fetch work. Requesting 1916024 seconds of work, reporting 1 completed tasks
2012-03-05 23:37:31|SETI@home|Scheduler request succeeded: got 0 new tasks
2012-03-05 23:37:31|SETI@home|Message from server: Project has no tasks available
2012-03-06 02:33:30|SETI@home|Sending scheduler request: To fetch work. Requesting 1949026 seconds of work, reporting 0 completed tasks
2012-03-06 02:33:35|SETI@home|Scheduler request succeeded: got 1 new tasks
2012-03-06 02:33:37|SETI@home|Started download of ap_12oc11ac_B3_P0_00006_20120209_32708.wu
2012-03-06 02:33:58|SETI@home|Finished download of ap_12oc11ac_B3_P0_00006_20120209_32708.wu (~390KB/sec)


Since ~4PM for me, I have zero work requests that failed or had errors, out of the ~30 requests that have been made.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1202889 · Report as offensive
Highlander
Avatar

Send message
Joined: 5 Oct 99
Posts: 167
Credit: 37,987,668
RAC: 16
Germany
Message 1202890 - Posted: 6 Mar 2012, 7:37:24 UTC
Last modified: 6 Mar 2012, 7:46:04 UTC

Strange, i got not the 500 error here but "Timeout was reached" error. So first contact to server and upload of reporting data takes place but no further reaction of schedular happens (from my point of view). Cricket shows this also, low traffic.

€:
Cosmic_Ocean: nice to see, that there is still life in server closet ^^
- Performance is not a simple linear function of the number of CPUs you throw at the problem. -
ID: 1202890 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22190
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1202892 - Posted: 6 Mar 2012, 7:45:25 UTC

A quick look at the crickets shows that during yesterday evening there were a couple of hours of "decent" service, but it all went to bits around midnight, and since then downloads have effectively been stalled, while uploads are continuing more or less normally.
I guess we'll have to wait until the tyre kicker gets in and does his rounds, although being Tuesday today it might be a slightly longer wait than that...


Strange - Just a passing thought, midnight in the UK is about 4pm in Berkeley which must be "about time to go home". Did something fail just as they left, or was something turned off overnight??
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1202892 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1202895 - Posted: 6 Mar 2012, 8:03:25 UTC
Last modified: 6 Mar 2012, 8:06:52 UTC

Last connect any of my rigs.....that's 9 rigs...has had was 16:04UTC. (Contact with the scheduler to report work, anyway...uploads are still working).
Then it dropped dead for all 9.
Nothing but internal server HTTP errors since.

Meowsigh.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1202895 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 9 · Next

Message boards : Number crunching : Panic Mode On (70) Server problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.