Download server glitch

Message boards : News : Download server glitch
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Eric Korpela Project Donor
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 3 Apr 99
Posts: 1342
Credit: 40,232,850
RAC: 60,825
United States
Message 1982953 - Posted: 1 Mar 2019, 21:00:09 UTC

We had a problem with one of our download servers that was slowing down downloads and slowing down work generation. We've fixed the problem, but there are lots of people out there trying to download. It'll probably be a while before our downloads aren't swamped with requests.

Work ready to send has started trending back up, so we shouldn't end up with a work shortage.
@SETIEric

ID: 1982953 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11157
Credit: 160,837,353
RAC: 110,825
Australia
Message 1982973 - Posted: 1 Mar 2019, 21:45:14 UTC

Unfortunately the issue is still occurring, just not as badly.
2-10 retries to get downloads restarting when they go in to backoff, instead of 50+ for no result.
Grant
Darwin NT
ID: 1982973 · Report as offensive
Profile ronssito
Avatar

Send message
Joined: 8 Feb 00
Posts: 14
Credit: 28,897,908
RAC: 57,542
United States
Message 1982976 - Posted: 1 Mar 2019, 21:53:15 UTC

yep day 2 with no boincstats
ID: 1982976 · Report as offensive
Stephen "Heretic" Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 4269
Credit: 124,866,978
RAC: 180,893
Australia
Message 1982981 - Posted: 1 Mar 2019, 22:08:02 UTC - in response to Message 1982973.  
Last modified: 1 Mar 2019, 22:08:56 UTC

Unfortunately the issue is still occurring, just not as badly.
2-10 retries to get downloads restarting when they go in to backoff, instead of 50+ for no result.


. . At this point I am getting that too but I am guessing it is just congestion as Eric suggested.

Stephen

. .
ID: 1982981 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11157
Credit: 160,837,353
RAC: 110,825
Australia
Message 1982985 - Posted: 1 Mar 2019, 22:21:08 UTC - in response to Message 1982981.  
Last modified: 1 Mar 2019, 22:24:09 UTC

Unfortunately the issue is still occurring, just not as badly.
2-10 retries to get downloads restarting when they go in to backoff, instead of 50+ for no result.


. . At this point I am getting that too but I am guessing it is just congestion as Eric suggested.

It may be load related- but I've never seen this issue at any other time after an extended outage (even with a similar deficit of Work in progress). Usually the downloads will start, the timer counting up but nothing actually downloading after 30s to a couple of minutes it will then time out. Here the download times out instantly, or after a second or so.
And the actual download speed when the download does start is much, much slower than usual (especially so considering it's only 1-2 WU at a time, not my usual dozen).
Grant
Darwin NT
ID: 1982985 · Report as offensive
Profile Bernie Vine
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 26 May 99
Posts: 9705
Credit: 67,584,509
RAC: 97,731
United Kingdom
Message 1982989 - Posted: 1 Mar 2019, 22:23:38 UTC

All good here, tasks completing, uploading, new ones downloading.
ID: 1982989 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1281
Credit: 141,423,463
RAC: 169,045
United States
Message 1983032 - Posted: 2 Mar 2019, 0:06:12 UTC

Back to intermittently needing to force retrys, and having the retrys fail complaining of project communication failure. Very different from the "no tasks available" syndrome commonly attributed to traffic after and outage ...
ID: 1983032 · Report as offensive
Profile Wiggo "Democratic Socialist"
Avatar

Send message
Joined: 24 Jan 00
Posts: 16305
Credit: 210,039,037
RAC: 138,798
Australia
Message 1983034 - Posted: 2 Mar 2019, 0:12:11 UTC

All's fine here, but it has been for the last 24hrs.

Cheers.
ID: 1983034 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 8287
Credit: 716,540,386
RAC: 1,722,723
United States
Message 1983040 - Posted: 2 Mar 2019, 0:19:37 UTC

I am in agreement with Grant. This is not showing the typical symptoms of heavy congestion after an outage. I am still being left with hosts unable to report tasks because of stuck downloads. When you do manually unstick them, then you get 98% of the download to download before a few stragglers stick again. The stuck tasks keep pushing their backoffs larger and larger at each countdown attempt. When I do manually get the tasks to download and report, I am so far past the normal 5 minute countdown that the request for work is in the hundreds which increases the likelihood of getting more stuck tasks at the end of the download session.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1983040 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 8287
Credit: 716,540,386
RAC: 1,722,723
United States
Message 1983051 - Posted: 2 Mar 2019, 0:57:17 UTC

Pretty disheartening to watch a brand new download of a dozen tasks on a host go instantly to a 5 hour backoff on first attempt to download. No other tasks were stuck on any other host.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1983051 · Report as offensive
.:krypt08700:.

Send message
Joined: 20 Jul 02
Posts: 4
Credit: 5,993,009
RAC: 33,928
United States
Message 1983073 - Posted: 2 Mar 2019, 2:31:08 UTC

Issue seems to be resolved for me. I did have to retry a few stuck downloads though.
ID: 1983073 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 8287
Credit: 716,540,386
RAC: 1,722,723
United States
Message 1983074 - Posted: 2 Mar 2019, 2:56:25 UTC - in response to Message 1983073.  

Still getting stuck tasks on all hosts. Backoffs start at 10 minutes and rapidly progress to several hours.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1983074 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 11157
Credit: 160,837,353
RAC: 110,825
Australia
Message 1983079 - Posted: 2 Mar 2019, 3:03:58 UTC

Yep, 50+ retries to get a WU or 3, then that's it.
Instant timeouts (1 second gets added to the Elapsed time after it times out) on each attempt for another 50+ retries.
Grant
Darwin NT
ID: 1983079 · Report as offensive
Profile Christian Diekmann

Send message
Joined: 30 Mar 00
Posts: 13
Credit: 45,561,519
RAC: 33,738
Germany
Message 1983117 - Posted: 3 Mar 2019, 13:50:42 UTC

Hello everybody!
Hello SETI-Team!

These failures (and also the maintenance once a week) show, however, that it makes sense to greatly increase the amount of work units for the clients.

The client is with his settings in the point at "Save at least 10 working days" and "Save for an additional 10 working days" absolutely Antediluvian!

That has never worked. Maybe that was once meant for computers that were not that fast. Since 100 work units were sufficient for 1-2 weeks. I do not even have a very fast computer but it's done with 100 work units after 20 hours. So where are the WUs for the next 19 days?

Please take care of the program, so that all users just get more WUs and just really (!) a maximum of 20 days can count to minimize these failures in terms of work performance.

Over the years, too much working time is lost in which no calculation can take place. And the constant failures are not really fun to continue to work for SETI. I've been counting on SETI for over 20 years now and I think I can bring in this objection.

Please, finally, revise the client in the point of the amount of WUs! I think that I speak for many users here.
ID: 1983117 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 17341
Credit: 389,121,755
RAC: 121,887
United Kingdom
Message 1983123 - Posted: 3 Mar 2019, 14:01:26 UTC - in response to Message 1983117.  

Quite the contrary - it would make much more sense to dramatically reduce the amount of work out in the field, thus reducing the stress on the servers until such time as they are stabilized again.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1983123 · Report as offensive
Kissagogo27 Special Project $75 donor

Send message
Joined: 6 Nov 99
Posts: 451
Credit: 6,517,235
RAC: 3,811
France
Message 1983137 - Posted: 3 Mar 2019, 15:01:28 UTC

the " additionnal" is bad named, it's time between 2 asks for Wu .... with 0.01 it's done every 15mn :p
ID: 1983137 · Report as offensive
Profile tullio
Volunteer tester

Send message
Joined: 9 Apr 04
Posts: 7492
Credit: 2,438,873
RAC: 325
Italy
Message 1983138 - Posted: 3 Mar 2019, 15:04:49 UTC

My cache has alwas been two and half day work. But finally I god rid of 38 ghost tasks which should have arrived on January 7 and never arrived. Maybe I was moving a PC to a different room with a WiFi connection to the router and it worked only partially. Now I have retired my old SUN workstation after 11 years of service and brought that PC to my living room, closer to the router.
Tullio
ID: 1983138 · Report as offensive
Profile Gone with the wind (2) Crowdfunding Project Donor*Special Project $75 donor
Volunteer tester

Send message
Joined: 19 Nov 00
Posts: 41571
Credit: 41,951,437
RAC: 18
Message 1983160 - Posted: 3 Mar 2019, 16:43:04 UTC

This will keep happening until the s/w bloat is sorted and the hardware is updated.
ID: 1983160 · Report as offensive
Profile 1080
Avatar

Send message
Joined: 27 Jan 19
Posts: 1
Credit: 973,876
RAC: 7,393
Message 1983181 - Posted: 3 Mar 2019, 18:05:11 UTC - in response to Message 1983160.  

This will keep happening until the s/w bloat is sorted and the hardware is updated.


Wäre es dann nicht günstig, wenn die User aufgefordert würden, ihre Rechner für ein bis mehrere Tage offline zu setzen.
Das wäre in der Regel produktiver, als die Server zu berennen. Wäre diese Meldung auf Seti@home nicht gekommen, wäre
ich noch verzweifelt und hätte meinem Sohn jedwede weitere Nutzung verboten, weil ich davon aus ging, dass er meinen PC
gegen die Wand gefahren hat.

Man kann sie ja jederzeit per E-Mail reaktivieren.

Mit freundlichen Grüßen
Andreas aka 1080
ID: 1983181 · Report as offensive
Manfred Reiff
Avatar

Send message
Joined: 27 Apr 18
Posts: 2
Credit: 1,725,994
RAC: 138
Germany
Message 1983184 - Posted: 3 Mar 2019, 18:07:42 UTC

I had problems to upload finished tasks and problems downloading new tasks since Friday morning (0700 UTC). I ran out of tasks during Friday afternoon local time so I decided to swith to other projects.
Since Sunday afternoon (UTC) downloading new tasks seems to work normally. In the beginning I had still some problems to upload finished tasks. This problem seems to be solved too. Since ~1630 UTC I experienced no problems.
But there are 330+ finished tasks hanging around at Mr. Pen Dings coffee shop... ;-)
ID: 1983184 · Report as offensive
1 · 2 · 3 · Next

Message boards : News : Download server glitch


 
©2019 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.