Panic Mode On (108) Server Problems?

Message boards : Number crunching : Panic Mode On (108) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 18 · 19 · 20 · 21 · 22 · 23 · 24 . . . 32 · Next

AuthorMessage
kittyman Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 50087
Credit: 938,731,176
RAC: 248,350
United States
Message 1904091 - Posted: 1 Dec 2017, 13:47:56 UTC - in response to Message 1904085.  

It's all fine. The project will be back in January. Maybe a little earlier than that.
Which January?

Yes.

But in all seriousness.
I can only imagine they meant January for the year 3141.

I do hope it's a bit sooner than that.
Meow!
Happy is the person who shares their life with a cat. (Or two or three or........) =^.^=

Have made friends here.
Most were cats.
ID: 1904091 · Report as offensive
Profile Dr.Diesel Special Project $75 donor

Send message
Joined: 14 May 99
Posts: 37
Credit: 51,050,465
RAC: 204,544
United States
Message 1904095 - Posted: 1 Dec 2017, 14:15:40 UTC - in response to Message 1904091.  

Eric mentioned Informix, which I suspect is IBM's Informix DB, probably waiting on a new build from them. Or at minimum awaiting an undocumented config switch as a temp work-a-round. There was some backend working happening last night, perhaps they gave it a go and ran into further issues.
ID: 1904095 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6530
Credit: 189,872,510
RAC: 15,348
United States
Message 1904101 - Posted: 1 Dec 2017, 14:33:58 UTC - in response to Message 1904095.  
Last modified: 1 Dec 2017, 14:34:48 UTC

Eric mentioned Informix, which I suspect is IBM's Informix DB, probably waiting on a new build from them. Or at minimum awaiting an undocumented config switch as a temp work-a-round. There was some backend working happening last night, perhaps they gave it a go and ran into further issues.

It is that and last I heard we are on an older version of it.
I would hope it is a well know issue they ran into and time, because of the large size of the db, is the only factor.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the BP6/VP6 User Group today!
ID: 1904101 · Report as offensive
juan BFP Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 7132
Credit: 411,849,339
RAC: 280,297
Panama
Message 1904102 - Posted: 1 Dec 2017, 14:34:39 UTC
Last modified: 1 Dec 2017, 14:35:02 UTC

From the Informix site:

Fast, Always-on Transactions

Provides one of the industry’s widest sets of options for keeping data available at all times, including zero downtime for maintenance.

ID: 1904102 · Report as offensive
rob smith Special Project $250 donor
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 16473
Credit: 340,109,527
RAC: 224,376
United Kingdom
Message 1904108 - Posted: 1 Dec 2017, 14:56:59 UTC

Always believe advertising guff....
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1904108 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6530
Credit: 189,872,510
RAC: 15,348
United States
Message 1904110 - Posted: 1 Dec 2017, 14:59:51 UTC - in response to Message 1904102.  

From the Informix site:

Fast, Always-on Transactions

Provides one of the industry’s widest sets of options for keeping data available at all times, including zero downtime for maintenance.

Sure. Under specific conditions, and provided you don't need do anything like lock a table while writing data.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the BP6/VP6 User Group today!
ID: 1904110 · Report as offensive
Grumpy Old Man
Volunteer tester
Avatar

Send message
Joined: 1 Nov 08
Posts: 7269
Credit: 45,283,118
RAC: 1,412
Sweden
Message 1904142 - Posted: 1 Dec 2017, 16:03:46 UTC
Last modified: 1 Dec 2017, 16:05:29 UTC

And when it comes back again, every SETI computer on the planet being bone dry, what will that do to the servers, and the databases?

Poof......
And down we go again.

This is going to be an interesting weekend.
LOL
ID: 1904142 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 8279
Credit: 19,799,779
RAC: 13,211
United States
Message 1904157 - Posted: 1 Dec 2017, 16:33:00 UTC - in response to Message 1904142.  

And when it comes back again, every SETI computer on the planet being bone dry, what will that do to the servers, and the databases?

Poof......
And down we go again.

This is going to be an interesting weekend.
LOL

Thank dog for Einstein
ID: 1904157 · Report as offensive
Iona
Avatar

Send message
Joined: 12 Jul 07
Posts: 783
Credit: 14,481,748
RAC: 17,618
United Kingdom
Message 1904169 - Posted: 1 Dec 2017, 16:51:38 UTC - in response to Message 1904142.  

That's what I thought, so I've kept back a little stash of work (4 hours maybe) for just that reason. Hopefully, it'll be enough to get me through, what may prove to be a difficult re-start. For now, a little more Mass Effect seems in order.
Don't take life too seriously, as you'll never come out of it alive!
ID: 1904169 · Report as offensive
Grumpy Old Man
Volunteer tester
Avatar

Send message
Joined: 1 Nov 08
Posts: 7269
Credit: 45,283,118
RAC: 1,412
Sweden
Message 1904194 - Posted: 1 Dec 2017, 19:41:17 UTC
Last modified: 1 Dec 2017, 19:52:00 UTC

I seriously doubt that the project will come back before they leave the lab for the weekend.
Not that many hours left now.
Monday perhaps, could be a Monday in January 2018 or even 2019 though :-)
But whatever, it's fine with me, and my wallet too.
ID: 1904194 · Report as offensive
Profile Jeff Buck Special Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1904212 - Posted: 1 Dec 2017, 21:10:53 UTC
Last modified: 1 Dec 2017, 21:16:56 UTC

It has been suggested a number of times, by a number of Setizens, that the project's task deadlines are much too long, compared with other projects. That perhaps shortening the deadlines could speed up the turnaround time for the tasks that inevitably time out due to lapsed or drive-by users, who fail to clear their caches before dropping out of the project, as well as the "ghost" tasks that we see all the time. That speeding up the turnaround would also lessen the storage requirements of the master and replica databases, an issue that seems to rear its head quite often.

The shutdown of the scheduling server has given us an opportunity to get an idea as to how much of an impact these deadline timeouts actually have. At the present time, over 25,000 tasks have accumulated in the RTS buffer, all of them likely tasks that have passed their deadlines since the scheduling server was shut down less than 24 hours ago. That's out of 2,453,080 tasks the SSP shows as being "out in the field". So, in less than one day, more than 1% of the outstanding tasks have timed out. Since every one of those tasks has also been causing one (or more) of that WU's wingman's tasks to be held in the database in a pending status, that represents 2% of the outstanding tasks that have been in limbo for the full duration of the original deadline.

Now, I don't know what percentage of the master database is occupied by task data, versus workunit data, account data, host data, and so on, but it seems to me that if 2% of the storage space occupied by the task data could be freed up by just shortening the deadlines by a single day, it's something that should be worth exploring. Does that number scale? Could shortening the deadlines by 5 days reduce the requirements by 10%, or making them 10 days shorter cut 20%? I don't know, but having actual numbers like this to look at should certainly be helpful in making such an assessment. (Of course, the project admins could likely have always extracted such numbers at any time they wanted, with just a simple database query.)

Food for thought.....I hope. :^)

EDIT: The actual total number of tasks ("results") in the database should include the 3,193,665 "returned and awaiting validation", which is where the wingmen's pending tasks would be counted. So, my stated percentages are too high, but still significant, I think.
ID: 1904212 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 8279
Credit: 19,799,779
RAC: 13,211
United States
Message 1904214 - Posted: 1 Dec 2017, 21:24:00 UTC
Last modified: 1 Dec 2017, 21:24:23 UTC

On the same subject of database size. When data is rerun on a newer app do they purge the original result? If not that's a lot of useless baggage.
ID: 1904214 · Report as offensive
moomin
Avatar

Send message
Joined: 21 Oct 17
Posts: 2978
Credit: 20,017
RAC: 54
Sweden
Message 1904216 - Posted: 1 Dec 2017, 21:32:20 UTC

What?
SETI now doesn't accept my already crunched files ready to report.
https://setiathome.berkeley.edu/results.php?userid=10596805
ID: 1904216 · Report as offensive
Profile Advent42
Avatar

Send message
Joined: 23 Mar 17
Posts: 175
Credit: 4,015,683
RAC: 0
Ireland
Message 1904218 - Posted: 1 Dec 2017, 21:36:40 UTC - in response to Message 1904216.  

Have just returned from a leave........so no tasks whatsoever......
Down until the new year...
Too bad....oh well....back to scanning the skies with my telescope...:-)
ID: 1904218 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 5567
Credit: 386,175,853
RAC: 1,034,274
United States
Message 1904225 - Posted: 1 Dec 2017, 21:57:25 UTC - in response to Message 1904212.  

I too have wondered why SETI has stuck with the extremely long deadlines I assess were implemented for the original hardware used on the project. That kind of hardware is 18 years in the past and does not need to continue to be supported. I agree with you Jeff, I would expect the sizes of databases and the strain they put on the project would be greatly lessened if the deadlines were reduced by a month, lets say from the current 2 month deadline.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1904225 · Report as offensive
kittyman Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 50087
Credit: 938,731,176
RAC: 248,350
United States
Message 1904227 - Posted: 1 Dec 2017, 22:00:50 UTC - in response to Message 1904225.  

I too have wondered why SETI has stuck with the extremely long deadlines I assess were implemented for the original hardware used on the project. That kind of hardware is 18 years in the past and does not need to continue to be supported. I agree with you Jeff, I would expect the sizes of databases and the strain they put on the project would be greatly lessened if the deadlines were reduced by a month, lets say from the current 2 month deadline.

As I understand it, the reason there has been no adjustment is because Eric does not wish to disenfranchise anybody from participating in this project.

And that would include folks with very meager hardware resources. Not everybody can afford what some of us are able to.

That is why.
Happy is the person who shares their life with a cat. (Or two or three or........) =^.^=

Have made friends here.
Most were cats.
ID: 1904227 · Report as offensive
Profile Brent Norman Special Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2395
Credit: 273,413,841
RAC: 684,190
Canada
Message 1904229 - Posted: 1 Dec 2017, 22:07:57 UTC - in response to Message 1904212.  

Jeff, I agree that the deadlines are a way too long for tasks. It just makes no sense to me why with a 10 max cache setting, the deadlines are as much as 8 weeks. But your numbers are a way out. What is being flagged as a resend by timing out is part of the normal 4.5M tasks that are normally in the field, not the 2.5 that are left, and yes pendings should be included too. AP tasks usually have shorter deadline around 25 days, MB much longer. I think a reasonable deadline would be 20 days, 30 at most. Not 8 weeks.

But yes, it would help a small amount for the database size.
ID: 1904229 · Report as offensive
rob smith Special Project $250 donor
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 16473
Credit: 340,109,527
RAC: 224,376
United Kingdom
Message 1904230 - Posted: 1 Dec 2017, 22:08:57 UTC

Further to Mark's comment.
SETI has a far lower timout ratio than most other projects because they have deadlines that are demonstrably too short. Why send out 800 hours worth of tasks to a computer that will only complete 600 hours of processing before the dealine is reached.

Actually long deadlines do not impact on the size of the main database but on the size of one of the intermediate databases.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1904230 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 10345
Credit: 138,920,592
RAC: 84,115
Australia
Message 1904233 - Posted: 1 Dec 2017, 22:12:56 UTC - in response to Message 1904227.  

I too have wondered why SETI has stuck with the extremely long deadlines I assess were implemented for the original hardware used on the project. That kind of hardware is 18 years in the past and does not need to continue to be supported. I agree with you Jeff, I would expect the sizes of databases and the strain they put on the project would be greatly lessened if the deadlines were reduced by a month, lets say from the current 2 month deadline.

As I understand it, the reason there has been no adjustment is because Eric does not wish to disenfranchise anybody from participating in this project.

And that would include folks with very meager hardware resources.

Such as phones and low end to mid range tablets.
Take those and pre-Core 2 Duo architectures out of the picture and you could reduce the deadlines by at least 2/3.
Grant
Darwin NT
ID: 1904233 · Report as offensive
kittyman Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 50087
Credit: 938,731,176
RAC: 248,350
United States
Message 1904235 - Posted: 1 Dec 2017, 22:15:11 UTC - in response to Message 1904233.  

I too have wondered why SETI has stuck with the extremely long deadlines I assess were implemented for the original hardware used on the project. That kind of hardware is 18 years in the past and does not need to continue to be supported. I agree with you Jeff, I would expect the sizes of databases and the strain they put on the project would be greatly lessened if the deadlines were reduced by a month, lets say from the current 2 month deadline.

As I understand it, the reason there has been no adjustment is because Eric does not wish to disenfranchise anybody from participating in this project.

And that would include folks with very meager hardware resources.

Such as phones and low end to mid range tablets.
Take those and pre-Core 2 Duo architectures out of the picture and you could reduce the deadlines by at least 2/3.

Well, you could suggest that to Eric, but don't expect him to do anything remotely like that.
Happy is the person who shares their life with a cat. (Or two or three or........) =^.^=

Have made friends here.
Most were cats.
ID: 1904235 · Report as offensive
Previous · 1 . . . 18 · 19 · 20 · 21 · 22 · 23 · 24 . . . 32 · Next

Message boards : Number crunching : Panic Mode On (108) Server Problems?


 
©2018 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.