Panic Mode On (83) Server Problems?

Message boards : Number crunching : Panic Mode On (83) Server Problems?

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 22 · Next

AuthorMessage
David SProject Donor
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 17047
Credit: 20,958,109
RAC: 6,149
United States
Message 1357771 - Posted: 17 Apr 2013, 13:23:18 UTC - in response to Message 1357656.  

Someone load more AP tapes to be split.

The MB and AP splitters both work on the same "tapes" at the same time. The AP splitters go a lot faster and get done first, then have to sit there twiddling their electronic thumbs, waiting for the MBs to finish so more tapes can be loaded.

David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.


ID: 1357771 · Report as offensive
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 45949
Credit: 815,444,020
RAC: 124,454
United States
Message 1357791 - Posted: 17 Apr 2013, 14:33:40 UTC - in response to Message 1357771.  

Someone load more AP tapes to be split.

The MB and AP splitters both work on the same "tapes" at the same time. The AP splitters go a lot faster and get done first, then have to sit there twiddling their electronic thumbs, waiting for the MBs to finish so more tapes can be loaded.

Right now, there is AP work available to split, but for some reason all the AP splitters are in disabled status.
Always remember.....kitties are all Angels with fur.

Have made friends in this life.
Most were cats.
ID: 1357791 · Report as offensive
Profile Donald L. Johnson
Avatar

Send message
Joined: 5 Aug 02
Posts: 8205
Credit: 4,367,334
RAC: 5,461
United States
Message 1357827 - Posted: 17 Apr 2013, 16:26:21 UTC - in response to Message 1357791.  

Someone load more AP tapes to be split.

The MB and AP splitters both work on the same "tapes" at the same time. The AP splitters go a lot faster and get done first, then have to sit there twiddling their electronic thumbs, waiting for the MBs to finish so more tapes can be loaded.

Right now, there is AP work available to split, but for some reason all the AP splitters are in disabled status.

As are the ap_assimilators. Only the ap-validator4 seems to be running. This has been the case since Tuesday afternoon. Maybe a problem with the AstroPulse science database. And since Matt did not post his usual post-outage update yesterday, they may still be working on solving that problem....
Donald
Infernal Optimist / Submariner, retired
ID: 1357827 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 7495
Credit: 91,178,764
RAC: 46,088
Australia
Message 1357852 - Posted: 17 Apr 2013, 18:06:16 UTC - in response to Message 1357827.  

Right now, there is AP work available to split, but for some reason all the AP splitters are in disabled status.

As are the ap_assimilators. Only the ap-validator4 seems to be running. This has been the case since Tuesday afternoon. Maybe a problem with the AstroPulse science database. And since Matt did not post his usual post-outage update yesterday, they may still be working on solving that problem....

AP assimilators are still down- things are really building up there.
As it is, the MB assimilators are only slowly getting that backlog under control.
Grant
Darwin NT
ID: 1357852 · Report as offensive
Profile Donald L. Johnson
Avatar

Send message
Joined: 5 Aug 02
Posts: 8205
Credit: 4,367,334
RAC: 5,461
United States
Message 1357931 - Posted: 18 Apr 2013, 3:16:27 UTC - in response to Message 1357852.  

Right now, there is AP work available to split, but for some reason all the AP splitters are in disabled status.

As are the ap_assimilators. Only the ap-validator4 seems to be running. This has been the case since Tuesday afternoon. Maybe a problem with the AstroPulse science database. And since Matt did not post his usual post-outage update yesterday, they may still be working on solving that problem....

AP assimilators are still down- things are really building up there.
As it is, the MB assimilators are only slowly getting that backlog under control.

Looks like Astropulse is back in business.
[As of 18 Apr 2013, 3:00:05 UTC] all AP functions are running, 7 AP splitters splitting, and about 500 results ready to send.


Donald
Infernal Optimist / Submariner, retired
ID: 1357931 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 7495
Credit: 91,178,764
RAC: 46,088
Australia
Message 1358171 - Posted: 18 Apr 2013, 17:58:12 UTC - in response to Message 1357931.  
Last modified: 18 Apr 2013, 17:59:03 UTC

Ready-to-send buffer is shrinking, no new MB work being split.
Server Status page shows all MB splitters offline.
Grant
Darwin NT
ID: 1358171 · Report as offensive
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 45949
Credit: 815,444,020
RAC: 124,454
United States
Message 1358182 - Posted: 18 Apr 2013, 18:10:05 UTC - in response to Message 1358171.  
Last modified: 18 Apr 2013, 18:10:23 UTC

Ready-to-send buffer is shrinking, no new MB work being split.
Server Status page shows all MB splitters offline.

I hope that doesn't mean all my GPUs will be sucking wind whilst I am gone to work today.....
Hopefully da boyz in da lab are just playing about with some thingys and nothing's broken.
Always remember.....kitties are all Angels with fur.

Have made friends in this life.
Most were cats.
ID: 1358182 · Report as offensive
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 45949
Credit: 815,444,020
RAC: 124,454
United States
Message 1358193 - Posted: 18 Apr 2013, 18:24:21 UTC
Last modified: 18 Apr 2013, 18:24:36 UTC

And I see that the last server status page update shows some MB splitters back at work.

Meowsmile.
Always remember.....kitties are all Angels with fur.

Have made friends in this life.
Most were cats.
ID: 1358193 · Report as offensive
Tom*

Send message
Joined: 12 Aug 11
Posts: 126
Credit: 12,013,843
RAC: 3,155
United States
Message 1358194 - Posted: 18 Apr 2013, 18:24:23 UTC
Last modified: 18 Apr 2013, 18:24:40 UTC

Since all MB splitters are disabled but the pfb splitter is running
and the stats show
Results ready to send 217,822 258 8m
Current result creation rate 2.2091/sec 1.6656/sec 5m

Are we currently processing the more sensitive MB's?

if not who is splitting MB's
ID: 1358194 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6108
Credit: 155,330,948
RAC: 50,126
United States
Message 1358210 - Posted: 18 Apr 2013, 18:52:37 UTC

Maybe this is some of that new splitter code that was talked about months ago going into place. At the moment all but 3 splitters are online and they are being generated at 36.9398/sec & 1.7282/sec.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the BP6/VP6 User Group today!
ID: 1358210 · Report as offensive
Profile RottenMutt
Avatar

Send message
Joined: 15 Mar 01
Posts: 1011
Credit: 230,274,184
RAC: 0
United States
Message 1358379 - Posted: 19 Apr 2013, 5:13:25 UTC - in response to Message 1358210.  

queue's are draining, what's up with that...
ID: 1358379 · Report as offensive
rob smithProject Donor
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 13338
Credit: 154,962,148
RAC: 118,093
United Kingdom
Message 1358382 - Posted: 19 Apr 2013, 5:26:17 UTC

The only "production" splitters running are on Lando and Vader, I guess one of those two is having a bit of a problem :-(

Oh well, reserve projects here I come...
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1358382 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 7495
Credit: 91,178,764
RAC: 46,088
Australia
Message 1358394 - Posted: 19 Apr 2013, 6:28:51 UTC - in response to Message 1358382.  
Last modified: 19 Apr 2013, 6:29:18 UTC

Been getting lots of "Project has no tasks" messages. Notice the splitters just aren't cranking up the output as the number that are ready-to-send drops.
Also been getting a few downloads timing out, or taking a while to start downloading.
Grant
Darwin NT
ID: 1358394 · Report as offensive
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 45949
Credit: 815,444,020
RAC: 124,454
United States
Message 1358400 - Posted: 19 Apr 2013, 6:38:00 UTC - in response to Message 1358394.  

Been getting lots of "Project has no tasks" messages. Notice the splitters just aren't cranking up the output as the number that are ready-to-send drops.
Also been getting a few downloads timing out, or taking a while to start downloading.

Yup...
The MB splitters are not keeping up at all for some reason.
Hit and miss here..
My top rig totally ran out of GPU work, which might have taken an hour or so, given the shorties goin' on. And then, it had the fortune of getting a 98 WU hit, which will shore it back up for a while.



Always remember.....kitties are all Angels with fur.

Have made friends in this life.
Most were cats.
ID: 1358400 · Report as offensive
Profile Bernie Vine
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 26 May 99
Posts: 8599
Credit: 43,172,073
RAC: 20,363
United Kingdom
Message 1358401 - Posted: 19 Apr 2013, 6:38:03 UTC

Currently, uploads and reporting failing. Downloads get "projects has no tasks available"
"Sometimes it is the people no one imagines anything of who do the things that no one can imagine."
ID: 1358401 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 665
Credit: 351,299,605
RAC: 139,738
Australia
Message 1358432 - Posted: 19 Apr 2013, 7:12:10 UTC - in response to Message 1358401.  

we're dry ... ready to send was at 115 ...

this is why we need to lift the limits so that we can continue to work through while they sort out issues that arise ...
ID: 1358432 · Report as offensive
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 45949
Credit: 815,444,020
RAC: 124,454
United States
Message 1358438 - Posted: 19 Apr 2013, 7:17:03 UTC - in response to Message 1358432.  

we're dry ... ready to send was at 115 ...

this is why we need to lift the limits so that we can continue to work through while they sort out issues that arise ...

Yeah, I know....
I'd truly love it if they could.
Unless it would cause the DB to crash.
Always remember.....kitties are all Angels with fur.

Have made friends in this life.
Most were cats.
ID: 1358438 · Report as offensive
alan
Avatar

Send message
Joined: 18 Feb 00
Posts: 131
Credit: 401,606
RAC: 0
United Kingdom
Message 1358446 - Posted: 19 Apr 2013, 7:47:28 UTC

The limits are there to prevent issues from arising in the first place. I fail to see how raising them would help the project at all.

The project scope is clearly stated, to use spare cpu cycles that would otherwise be "wasted" running screensavers or other idle tasks. It was never intended to cope with dedicated computers or specially built crunching farms with multiple GPU's and demands for massive caches to sustain them running 24x7.

By constantly demanding that the limits be raised you are making your need to execute as many SETI tasks as possible more important than the well-being of the project itself.

The fix to this problem is larger workunits which will reduce the number of "in progress" entries in the database.
ID: 1358446 · Report as offensive
Profile Bernie Vine
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 26 May 99
Posts: 8599
Credit: 43,172,073
RAC: 20,363
United Kingdom
Message 1358452 - Posted: 19 Apr 2013, 8:00:11 UTC - in response to Message 1358446.  

The limits are there to prevent issues from arising in the first place. I fail to see how raising them would help the project at all.

The project scope is clearly stated, to use spare cpu cycles that would otherwise be "wasted" running screensavers or other idle tasks. It was never intended to cope with dedicated computers or specially built crunching farms with multiple GPU's and demands for massive caches to sustain them running 24x7.

By constantly demanding that the limits be raised you are making your need to execute as many SETI tasks as possible more important than the well-being of the project itself.

The fix to this problem is larger workunits which will reduce the number of "in progress" entries in the database.

+1
"Sometimes it is the people no one imagines anything of who do the things that no one can imagine."
ID: 1358452 · Report as offensive
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 45949
Credit: 815,444,020
RAC: 124,454
United States
Message 1358454 - Posted: 19 Apr 2013, 8:05:13 UTC - in response to Message 1358446.  

The limits are there to prevent issues from arising in the first place. I fail to see how raising them would help the project at all.

The project scope is clearly stated, to use spare cpu cycles that would otherwise be "wasted" running screensavers or other idle tasks. It was never intended to cope with dedicated computers or specially built crunching farms with multiple GPU's and demands for massive caches to sustain them running 24x7.

By constantly demanding that the limits be raised you are making your need to execute as many SETI tasks as possible more important than the well-being of the project itself.

The fix to this problem is larger workunits which will reduce the number of "in progress" entries in the database.


Alan...
Please don't try to remind some of us 'what the original scope of the project' was.
We know that. And we all (some of us, anyway,) know that the capacity of the project has thankfully grown far beyond the original concept.
No, in it's inception, it was never envisioned that vast numbers of like minded individuals would flock to the calling. But we have.
And the project has tried to it's best ability to respond.
They have done rather well, considering the meager funding...in most latter years, by it's own participants.

Matt has indicated that since the move to the colocation facility, we may well be outstripping the capacity to gather data by our massive ability to process it.

I see this as a great thing for the Seti project. And proof that the whole Boinc distributed computing concept has indeed come to fruition. We, as small users, have amassed such an enormous amount of computing power that the project might have some trouble supplying enough data for us to process......who would have dreamed?
Not Eric or Matt, I assure you.

As I have suggested, but am not in the direct loop right now, I suspect that since the relocation of the servers and the ability of the project to supply data to the crunching base, one of the upshots is that the GPUUG fundraisers upcoming might possibly be for a dedicated server to process the accumulated results.

The kitties would still like to get and process as much work as possible to further the initial goal of this project. Which is, of course, to prove that we are not alone.

I believe that every day we get closer to proving that reality.
Always remember.....kitties are all Angels with fur.

Have made friends in this life.
Most were cats.
ID: 1358454 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 22 · Next

Message boards : Number crunching : Panic Mode On (83) Server Problems?


 
©2016 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.