Panic Mode On (20) Server problems

Message boards : Number crunching : Panic Mode On (20) Server problems
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 15 · Next

AuthorMessage
Profile TCP JESUS
Avatar

Send message
Joined: 19 Jan 03
Posts: 205
Credit: 1,248,845
RAC: 0
Canada
Message 917692 - Posted: 14 Jul 2009, 23:41:55 UTC

The Upload server was taken down prior to Tuesday maintenance.....so I would have expected an update in the 'Technical News' section to accompany it, that is all.

Unfortunately, this basically means that I will not be able to participate this entire week because once I FINALLY get the 400+ WU's uploaded from the machine I am sitting at, it will be time for another outage........and the fun starts again.

This is part of the frustration I feel, as well as others.

I use Seti@Home as a benchmark and stability testing tool.

I am currently considering ABORTING all 400+ outstanding uploads (and growing every minute) that I have so that I can download more new work and continue to 'test' my machine's stability. Is THAT another alternative ?
I am TCP JESUS...The Carpenter Phenom Jesus....and HAMMERING is what I do best!
formerly known as...MC Hammer.
ID: 917692 · Report as offensive
Profile Blurf
Volunteer tester

Send message
Joined: 2 Sep 06
Posts: 8964
Credit: 12,678,685
RAC: 0
United States
Message 917693 - Posted: 14 Jul 2009, 23:43:14 UTC - in response to Message 917692.  

The Upload server was taken down prior to Tuesday maintenance.....so I would have expected an update in the 'Technical News' section to accompany it, that is all.

Unfortunately, this basically means that I will not be able to participate this entire week because once I FINALLY get the 400+ WU's uploaded from the machine I am sitting at, it will be time for another outage........and the fun starts again.

This is part of the frustration I feel, as well as others.

I use Seti@Home as a benchmark and stability testing tool.

I am currently considering ABORTING all 400+ outstanding uploads (and growing every minute) that I have so that I can download more new work and continue to 'test' my machine's stability. Is THAT another alternative ?


Using Seti as a benchmark may not be an entirely accurate measurement tool at this time with the bandwith and other issues we're experiencing.


ID: 917693 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 917702 - Posted: 15 Jul 2009, 0:19:52 UTC - in response to Message 917656.  

I have noticed that BOINC often "forgets" to upload results and I have to do a manual Update to get them to send in. I'll see logs of "finished wu, uploading wu, dowloading wu" and such with no errors. Just will have 6-10 results "Ready for upload" in my tasks. I have read around to see if this is a "normal" thnk to occur, but it happens on all of my hosts.


Uploading and reporting are two different things, you may be referring to that? A wu that has been processed should be uploaded fairly fast (asuming i-net connection to Berkeley/SETI is good).. reporting is done in larger chunks usualy. This is done because the reporting part takes resources and is more efficient done in 'bursts'. An manual reporting action partially circumvents this..

Hope to have (partly?) answerd your question

Eesger


It's actually "Ready to report" I miss typed the message.

I will see say 4 tasks "Ready to report" and will be processing 2. When one of the processing tasks finishes it reports and uploads and the other 4 are still "Ready to report". Manually selecting Update sends them in. Just seems odd to me. Once the upload server is back on I can do some screen shots if that would be more helpful.


Yes, this is perfectly normal. The uploads happen in two stages: when the crunching is finished, it is reported immediately to the project server, but the result is kept on the user's hard drive so that the uploads can be batched together, easing the server's resources (as opposed to some fast machines that finish workunits within 30 minutes, pounding the servers ever half hour with uploads and reports).

Once there has been a backlog of results, typically on the next server request for more work, the backlog gets uploaded as well.

Hitting the Update button forces the server communication and can make things worse for bandwidth-starved, server-overloaded projects like SETI.

I guarantee that if you leave those tasks alone, they will be reported within 24 hours of completion on average.

**Not saying any of this to be mean or to scold anyone, just trying to be helpful.**
ID: 917702 · Report as offensive
Profile Jack Zhang
Volunteer tester
Avatar

Send message
Joined: 2 Jul 06
Posts: 206
Credit: 6,142,449
RAC: 0
Canada
Message 917703 - Posted: 15 Jul 2009, 0:22:37 UTC - in response to Message 917692.  

The Upload server was taken down prior to Tuesday maintenance.....so I would have expected an update in the 'Technical News' section to accompany it, that is all.

Unfortunately, this basically means that I will not be able to participate this entire week because once I FINALLY get the 400+ WU's uploaded from the machine I am sitting at, it will be time for another outage........and the fun starts again.

This is part of the frustration I feel, as well as others.

I use Seti@Home as a benchmark and stability testing tool.

I am currently considering ABORTING all 400+ outstanding uploads (and growing every minute) that I have so that I can download more new work and continue to 'test' my machine's stability. Is THAT another alternative ?


Prime95 is another really good stability testing software. It also stresses the system more so that you can be sure you have safe overclocks for example.
What if Fiction was Fact and Fact was Fiction and vice versa?
ID: 917703 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 917705 - Posted: 15 Jul 2009, 0:24:38 UTC - in response to Message 917692.  

This is part of the frustration I feel, as well as others.


We're trying to work on the focus of that frustration, and divert it elsewhere.

I use Seti@Home as a benchmark and stability testing tool.


Nothing wrong with that, but do know that this is not its intended purpose.

I am currently considering ABORTING all 400+ outstanding uploads (and growing every minute) that I have so that I can download more new work and continue to 'test' my machine's stability. Is THAT another alternative ?


No one can stop you. I'd hate to think that any user is that selfish, but we've had others do just-as-selfish acts and not care about it. We can't dictate to you what your priorities should be, but it is frowned upon (not that it seems like you care about that).

But as someone whom has complained about wasting processing power, such an act would most certainly deem all our efforts on those tasks moot. I can't really imagine you'd want to do that. I think once cooler heads prevail, you'll be glad you didn't - at least I hope you would be.

Remember also that we are all part of this community together. There's no need to focus your frustration on people who are trying to help you - even if it isn't exactly what you want to hear. :)
ID: 917705 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 917706 - Posted: 15 Jul 2009, 0:25:44 UTC - in response to Message 917703.  

The Upload server was taken down prior to Tuesday maintenance.....so I would have expected an update in the 'Technical News' section to accompany it, that is all.

Unfortunately, this basically means that I will not be able to participate this entire week because once I FINALLY get the 400+ WU's uploaded from the machine I am sitting at, it will be time for another outage........and the fun starts again.

This is part of the frustration I feel, as well as others.

I use Seti@Home as a benchmark and stability testing tool.

I am currently considering ABORTING all 400+ outstanding uploads (and growing every minute) that I have so that I can download more new work and continue to 'test' my machine's stability. Is THAT another alternative ?


Prime95 is another really good stability testing software. It also stresses the system more so that you can be sure you have safe overclocks for example.


Agreed, and they even have a "stress testing" option built right into the client so that you don't trash any of their workunits.
ID: 917706 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 917708 - Posted: 15 Jul 2009, 0:35:17 UTC - in response to Message 917702.  

I have noticed that BOINC often "forgets" to upload results and I have to do a manual Update to get them to send in. I'll see logs of "finished wu, uploading wu, dowloading wu" and such with no errors. Just will have 6-10 results "Ready for upload" in my tasks. I have read around to see if this is a "normal" thnk to occur, but it happens on all of my hosts.


Uploading and reporting are two different things, you may be referring to that? A wu that has been processed should be uploaded fairly fast (asuming i-net connection to Berkeley/SETI is good).. reporting is done in larger chunks usualy. This is done because the reporting part takes resources and is more efficient done in 'bursts'. An manual reporting action partially circumvents this..

Hope to have (partly?) answerd your question

Eesger


It's actually "Ready to report" I miss typed the message.

I will see say 4 tasks "Ready to report" and will be processing 2. When one of the processing tasks finishes it reports and uploads and the other 4 are still "Ready to report". Manually selecting Update sends them in. Just seems odd to me. Once the upload server is back on I can do some screen shots if that would be more helpful.


Yes, this is perfectly normal. The uploads happen in two stages: when the crunching is finished, it is reported immediately to the project server, but the result is kept on the user's hard drive so that the uploads can be batched together, easing the server's resources (as opposed to some fast machines that finish workunits within 30 minutes, pounding the servers ever half hour with uploads and reports).

Once there has been a backlog of results, typically on the next server request for more work, the backlog gets uploaded as well.

Hitting the Update button forces the server communication and can make things worse for bandwidth-starved, server-overloaded projects like SETI.

I guarantee that if you leave those tasks alone, they will be reported within 24 hours of completion on average.

**Not saying any of this to be mean or to scold anyone, just trying to be helpful.**


Ah okies. I'll have to watch then.

Normally:
I would come into work in the morning.
See all the machines with loads of "Ready to report" tasks.
Scratch my head.
Shrug.
Click upload.

Thought that perhaps there was some sort of minor bug. I'll just let them be and see once the servers are back to what they like to call "Normal" :)
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 917708 · Report as offensive
HAL

Send message
Joined: 28 Mar 03
Posts: 704
Credit: 870,617
RAC: 0
United States
Message 917712 - Posted: 15 Jul 2009, 0:45:39 UTC

The last time we had upload problems I had one machine that waited almost 3 days to upload. 24 hours isn't even close!
ID: 917712 · Report as offensive
Profile Dirk Sadowski
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 917713 - Posted: 15 Jul 2009, 0:46:48 UTC
Last modified: 15 Jul 2009, 0:48:40 UTC


IIRC, the UL server is now ~ 24 hours offline.



A presumption of Pappa..
Technical News : 'Working as Expected (Jul 13 2009) - Message 917697'


ID: 917713 · Report as offensive
Profile TCP JESUS
Avatar

Send message
Joined: 19 Jan 03
Posts: 205
Credit: 1,248,845
RAC: 0
Canada
Message 917727 - Posted: 15 Jul 2009, 1:12:11 UTC

So, is it fair to assume that since the 'work day' is officially over on the West Coast...and without any updates, that we are going to be left in the dark another night without uploads....and not knowing what is actually going on ?
I am TCP JESUS...The Carpenter Phenom Jesus....and HAMMERING is what I do best!
formerly known as...MC Hammer.
ID: 917727 · Report as offensive
Siddarth P Kumar
Volunteer tester

Send message
Joined: 10 Oct 04
Posts: 2
Credit: 13,126
RAC: 0
India
Message 917730 - Posted: 15 Jul 2009, 1:30:46 UTC - in response to Message 916224.  

Unable to upload from past 30 hours or so and unable to download work from past 12 hours.. is it an issue with my computer or with SETI servers?

Also when i try downloading work from my office the download always fail. Can you help me with this..
ID: 917730 · Report as offensive
| ganja |
Volunteer tester
Avatar

Send message
Joined: 9 Aug 08
Posts: 26
Credit: 2,463,915
RAC: 0
United States
Message 917736 - Posted: 15 Jul 2009, 1:46:41 UTC

I understand having to go down for work for a few hrs
on Tuesday afternoons hell this has been going on for about a week now. I havent uploaded a wu since like 4:30 am est what gives here I think they need a back up plan or server. I currently have about 35 wu tring to upload now.

ganga
ID: 917736 · Report as offensive
Daniel S. Gandolfo

Send message
Joined: 17 Nov 02
Posts: 56
Credit: 2,430,947
RAC: 0
United Kingdom
Message 917739 - Posted: 15 Jul 2009, 1:58:15 UTC

I dont kbow if I've posted in the correct topic but here goes. Over the past few months or so I have noticed that SETI computers are becoming increasingly taxed and often crash.

Could anyone tell me what the problem at the moment is. Ie I cannot upload any of my completed WUs to SETI and they are not issueing any new ones. This has been intermittent for the last few weeks or so and is it likely to be fixed soon?

Nevica
ID: 917739 · Report as offensive
Profile slak

Send message
Joined: 1 Jun 99
Posts: 5
Credit: 2,498,787
RAC: 0
Canada
Message 917742 - Posted: 15 Jul 2009, 2:01:14 UTC - in response to Message 917739.  

I dont kbow if I've posted in the correct topic but here goes. Over the past few months or so I have noticed that SETI computers are becoming increasingly taxed and often crash.

Could anyone tell me what the problem at the moment is. Ie I cannot upload any of my completed WUs to SETI and they are not issueing any new ones. This has been intermittent for the last few weeks or so and is it likely to be fixed soon?

Nevica


This thread should help explain what some of the issues are.

http://setiathome.berkeley.edu/forum_thread.php?id=54631&nowrap=true#917472
ID: 917742 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 917743 - Posted: 15 Jul 2009, 2:02:17 UTC - in response to Message 917739.  

I dont kbow if I've posted in the correct topic but here goes. Over the past few months or so I have noticed that SETI computers are becoming increasingly taxed and often crash.

Could anyone tell me what the problem at the moment is. Ie I cannot upload any of my completed WUs to SETI and they are not issueing any new ones. This has been intermittent for the last few weeks or so and is it likely to be fixed soon?

Nevica

I don't think they've crashed, but there definitely are times when there is so much inbound traffic that you'd have a tough time telling if they were up or down.

Historically, SETI@Home has had some long stretches of smooth running, and some stretches of tough times. This is one of the rough times.

Odds are pretty good that we can all ride it out.
ID: 917743 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 917746 - Posted: 15 Jul 2009, 2:04:52 UTC - in response to Message 917712.  

The last time we had upload problems I had one machine that waited almost 3 days to upload. 24 hours isn't even close!


Other factors can have an effect on things, such as invalid cached DNS in the local resolver, BOINC's random back-off feature and other things.

When things are working "normally", all tasks are reported within 24 hours.
ID: 917746 · Report as offensive
Profile Kai
Volunteer tester
Avatar

Send message
Joined: 30 Jun 09
Posts: 619
Credit: 15,732
RAC: 0
United Kingdom
Message 917747 - Posted: 15 Jul 2009, 2:06:38 UTC - in response to Message 917730.  
Last modified: 15 Jul 2009, 2:09:19 UTC

Unable to upload from past 30 hours or so and unable to download work from past 12 hours.. is it an issue with my computer or with SETI servers?

Also when i try downloading work from my office the download always fail. Can you help me with this..


The Upload server is currently offline. It should be back up soon!! Please be patient this is affecting everyone but work will start to go through again soon.
This is being discussed in many, many, many threads... Unfortunately we crunchers can't do anything ourselves, we just have to stick with it and see what happens over the next week or so.

Click Here For The SETI@home Server Status Page

"Only two things are infinite, the universe and human stupidity, and I'm not sure about the former." - Albert Einstein
Vextor Homepage | Vextor Blog
ID: 917747 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 917748 - Posted: 15 Jul 2009, 2:08:00 UTC - in response to Message 917736.  
Last modified: 15 Jul 2009, 2:09:30 UTC

If you've been reading any of the threads around here for the past week, it explains the problems SETI's been having, and much discussion has been had over what the best "back up" plan to use, but ultimately it is the Admin's decision as to what works best for them and how much money they have to enact said plan.
ID: 917748 · Report as offensive
Profile Blurf
Volunteer tester

Send message
Joined: 2 Sep 06
Posts: 8964
Credit: 12,678,685
RAC: 0
United States
Message 917750 - Posted: 15 Jul 2009, 2:10:51 UTC - in response to Message 917727.  

So, is it fair to assume that since the 'work day' is officially over on the West Coast...and without any updates, that we are going to be left in the dark another night without uploads....and not knowing what is actually going on ?


Yes, the work day usually ends at 5pm PST but the Admins will work on their own time when family needs aren't more pressing-however I am of the firm belief that family time is VERY important.

Regarding Tech News, Matt does this as a favor to the Seti community (for which I am appreciative), however if there's a decision to be made between taking the time to post and fixing a server, I'd rather him delay a post to us and work on the server.

Again the issue here is a severe lack of resources in the Berkeley Seti office. With more money, they could hire another staff person which would free up more resources to give you the updates you (and all of us) want.


ID: 917750 · Report as offensive
Profile Pappa
Volunteer tester
Avatar

Send message
Joined: 9 Jan 00
Posts: 2562
Credit: 12,301,681
RAC: 0
United States
Message 917756 - Posted: 15 Jul 2009, 2:24:12 UTC - in response to Message 917713.  
Last modified: 15 Jul 2009, 2:27:38 UTC


IIRC, the UL server is now ~ 24 hours offline.



A presumption of Pappa..
Technical News : 'Working as Expected (Jul 13 2009) - Message 917697'



I could be a bit glib. Over the years, I remember when the Classic Servers were taken offline for weeks to sort Candidates. So what is 24 hours?

Right, wrong or indifferent. I am also aware of some things hapening in that the Staff are trying to "balance" the problem to return to normal. Many have posted information as to potential reasons and their thoughts.

Seti currently has a 100Megabit connection down the hill to the to the UCB Campus where the Gigabit link comes in. That link will support 12500000 Bytes per second transfer. You can do the math for MultiBeam take 12500000/367000 and look at the number (that is in seconds). That tells you how many downloads can happen at the same time. What makes it more complicated is that when we look at Astropulse. That file size is 8196000, so you can take now take 12500000/8196000 (that is also in seconds). The calculator tells you the rest. Now come up with a mix that will fit in the download bandwidth. That is what the Feeder is supposed to do. Currently there is an imbalance that is causing issues. So the Real Panic is how does the Seti Staff nail down the cause and correct it?

Fast Ethernet, the Rules apply. When you get over 90% of the link capacity things start going bad in a hurry.

If you want, please go to a hardware store and puchase an 8 foot lenght of pipe with an inside diameter of 1/2 inch. Stop at Toy store and purchase a couple of bags of marbles (you need to be able to overfill the 8 foot lenght of pipe). Now when you get home open the bags of marbles and start shoving them though the pipe (downloads). When the pipe is Full, start trying to shove marbles up the other end.

Please take a moment to inform us of the results of the experiment. Yes I know of the flaw, Astropluse Marbles are Larger than 1/2 Inch.

So the presumption that you mention is that Matt is testing how many downloads that can happen at the same time without interfering with Uploads which includes Scheduler Requests (get more work) or the Result that is Patiently waiting... I see nothing wrong with that.

This does not mention that Seti Beta also shares the same Bandwith. Which is TOTALLY Disabled. It all "used to work," now what happened and why?

So while typing this I see that Someone (Matt) has added new images and the splitters are doing both MB and AP

Regards
Please consider a Donation to the Seti Project.

ID: 917756 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 15 · Next

Message boards : Number crunching : Panic Mode On (20) Server problems


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.