Download server glitch

Message boards : News : Download server glitch
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Eric Korpela Project Donor
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 3 Apr 99
Posts: 1382
Credit: 54,506,847
RAC: 60
United States
Message 1982953 - Posted: 1 Mar 2019, 21:00:09 UTC

We had a problem with one of our download servers that was slowing down downloads and slowing down work generation. We've fixed the problem, but there are lots of people out there trying to download. It'll probably be a while before our downloads aren't swamped with requests.

Work ready to send has started trending back up, so we shouldn't end up with a work shortage.
@SETIEric@qoto.org (Mastodon)

ID: 1982953 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1982973 - Posted: 1 Mar 2019, 21:45:14 UTC

Unfortunately the issue is still occurring, just not as badly.
2-10 retries to get downloads restarting when they go in to backoff, instead of 50+ for no result.
Grant
Darwin NT
ID: 1982973 · Report as offensive
Profile ronssito
Avatar

Send message
Joined: 8 Feb 00
Posts: 19
Credit: 43,465,609
RAC: 63
United States
Message 1982976 - Posted: 1 Mar 2019, 21:53:15 UTC

yep day 2 with no boincstats
ID: 1982976 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1982981 - Posted: 1 Mar 2019, 22:08:02 UTC - in response to Message 1982973.  
Last modified: 1 Mar 2019, 22:08:56 UTC

Unfortunately the issue is still occurring, just not as badly.
2-10 retries to get downloads restarting when they go in to backoff, instead of 50+ for no result.


. . At this point I am getting that too but I am guessing it is just congestion as Eric suggested.

Stephen

. .
ID: 1982981 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1982985 - Posted: 1 Mar 2019, 22:21:08 UTC - in response to Message 1982981.  
Last modified: 1 Mar 2019, 22:24:09 UTC

Unfortunately the issue is still occurring, just not as badly.
2-10 retries to get downloads restarting when they go in to backoff, instead of 50+ for no result.


. . At this point I am getting that too but I am guessing it is just congestion as Eric suggested.

It may be load related- but I've never seen this issue at any other time after an extended outage (even with a similar deficit of Work in progress). Usually the downloads will start, the timer counting up but nothing actually downloading after 30s to a couple of minutes it will then time out. Here the download times out instantly, or after a second or so.
And the actual download speed when the download does start is much, much slower than usual (especially so considering it's only 1-2 WU at a time, not my usual dozen).
Grant
Darwin NT
ID: 1982985 · Report as offensive
Profile Bernie Vine
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 26 May 99
Posts: 9954
Credit: 103,452,613
RAC: 328
United Kingdom
Message 1982989 - Posted: 1 Mar 2019, 22:23:38 UTC

All good here, tasks completing, uploading, new ones downloading.
ID: 1982989 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1849
Credit: 268,616,081
RAC: 1,349
United States
Message 1983032 - Posted: 2 Mar 2019, 0:06:12 UTC

Back to intermittently needing to force retrys, and having the retrys fail complaining of project communication failure. Very different from the "no tasks available" syndrome commonly attributed to traffic after and outage ...
ID: 1983032 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1983034 - Posted: 2 Mar 2019, 0:12:11 UTC

All's fine here, but it has been for the last 24hrs.

Cheers.
ID: 1983034 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1983040 - Posted: 2 Mar 2019, 0:19:37 UTC

I am in agreement with Grant. This is not showing the typical symptoms of heavy congestion after an outage. I am still being left with hosts unable to report tasks because of stuck downloads. When you do manually unstick them, then you get 98% of the download to download before a few stragglers stick again. The stuck tasks keep pushing their backoffs larger and larger at each countdown attempt. When I do manually get the tasks to download and report, I am so far past the normal 5 minute countdown that the request for work is in the hundreds which increases the likelihood of getting more stuck tasks at the end of the download session.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1983040 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1983051 - Posted: 2 Mar 2019, 0:57:17 UTC

Pretty disheartening to watch a brand new download of a dozen tasks on a host go instantly to a 5 hour backoff on first attempt to download. No other tasks were stuck on any other host.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1983051 · Report as offensive
.:krypt08700:.

Send message
Joined: 20 Jul 02
Posts: 5
Credit: 6,788,712
RAC: 0
United States
Message 1983073 - Posted: 2 Mar 2019, 2:31:08 UTC

Issue seems to be resolved for me. I did have to retry a few stuck downloads though.
ID: 1983073 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1983074 - Posted: 2 Mar 2019, 2:56:25 UTC - in response to Message 1983073.  

Still getting stuck tasks on all hosts. Backoffs start at 10 minutes and rapidly progress to several hours.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1983074 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1983079 - Posted: 2 Mar 2019, 3:03:58 UTC

Yep, 50+ retries to get a WU or 3, then that's it.
Instant timeouts (1 second gets added to the Elapsed time after it times out) on each attempt for another 50+ retries.
Grant
Darwin NT
ID: 1983079 · Report as offensive
Profile Christian Diekmann

Send message
Joined: 30 Mar 00
Posts: 13
Credit: 60,615,800
RAC: 163
Germany
Message 1983117 - Posted: 3 Mar 2019, 13:50:42 UTC

Hello everybody!
Hello SETI-Team!

These failures (and also the maintenance once a week) show, however, that it makes sense to greatly increase the amount of work units for the clients.

The client is with his settings in the point at "Save at least 10 working days" and "Save for an additional 10 working days" absolutely Antediluvian!

That has never worked. Maybe that was once meant for computers that were not that fast. Since 100 work units were sufficient for 1-2 weeks. I do not even have a very fast computer but it's done with 100 work units after 20 hours. So where are the WUs for the next 19 days?

Please take care of the program, so that all users just get more WUs and just really (!) a maximum of 20 days can count to minimize these failures in terms of work performance.

Over the years, too much working time is lost in which no calculation can take place. And the constant failures are not really fun to continue to work for SETI. I've been counting on SETI for over 20 years now and I think I can bring in this objection.

Please, finally, revise the client in the point of the amount of WUs! I think that I speak for many users here.
ID: 1983117 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22158
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1983123 - Posted: 3 Mar 2019, 14:01:26 UTC - in response to Message 1983117.  

Quite the contrary - it would make much more sense to dramatically reduce the amount of work out in the field, thus reducing the stress on the servers until such time as they are stabilized again.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1983123 · Report as offensive
Profile Kissagogo27 Special Project $75 donor
Avatar

Send message
Joined: 6 Nov 99
Posts: 715
Credit: 8,032,827
RAC: 62
France
Message 1983137 - Posted: 3 Mar 2019, 15:01:28 UTC

the " additionnal" is bad named, it's time between 2 asks for Wu .... with 0.01 it's done every 15mn :p
ID: 1983137 · Report as offensive
Profile tullio
Volunteer tester

Send message
Joined: 9 Apr 04
Posts: 8797
Credit: 2,930,782
RAC: 1
Italy
Message 1983138 - Posted: 3 Mar 2019, 15:04:49 UTC

My cache has alwas been two and half day work. But finally I god rid of 38 ghost tasks which should have arrived on January 7 and never arrived. Maybe I was moving a PC to a different room with a WiFi connection to the router and it worked only partially. Now I have retired my old SUN workstation after 11 years of service and brought that PC to my living room, closer to the router.
Tullio
ID: 1983138 · Report as offensive
Profile Rafi
Avatar

Send message
Joined: 27 Jan 19
Posts: 1
Credit: 2,009,067
RAC: 18
Germany
Message 1983181 - Posted: 3 Mar 2019, 18:05:11 UTC - in response to Message 1983160.  

This will keep happening until the s/w bloat is sorted and the hardware is updated.


Wäre es dann nicht günstig, wenn die User aufgefordert würden, ihre Rechner für ein bis mehrere Tage offline zu setzen.
Das wäre in der Regel produktiver, als die Server zu berennen. Wäre diese Meldung auf Seti@home nicht gekommen, wäre
ich noch verzweifelt und hätte meinem Sohn jedwede weitere Nutzung verboten, weil ich davon aus ging, dass er meinen PC
gegen die Wand gefahren hat.

Man kann sie ja jederzeit per E-Mail reaktivieren.

Mit freundlichen Grüßen
Andreas aka 1080
ID: 1983181 · Report as offensive
Manfred Reiff
Avatar

Send message
Joined: 27 Apr 18
Posts: 2
Credit: 1,725,994
RAC: 0
Germany
Message 1983184 - Posted: 3 Mar 2019, 18:07:42 UTC

I had problems to upload finished tasks and problems downloading new tasks since Friday morning (0700 UTC). I ran out of tasks during Friday afternoon local time so I decided to swith to other projects.
Since Sunday afternoon (UTC) downloading new tasks seems to work normally. In the beginning I had still some problems to upload finished tasks. This problem seems to be solved too. Since ~1630 UTC I experienced no problems.
But there are 330+ finished tasks hanging around at Mr. Pen Dings coffee shop... ;-)
ID: 1983184 · Report as offensive
Manfred Reiff
Avatar

Send message
Joined: 27 Apr 18
Posts: 2
Credit: 1,725,994
RAC: 0
Germany
Message 1983193 - Posted: 3 Mar 2019, 18:41:57 UTC - in response to Message 1983181.  

This will keep happening until the s/w bloat is sorted and the hardware is updated.


Wäre es dann nicht günstig, wenn die User aufgefordert würden, ihre Rechner für ein bis mehrere Tage offline zu setzen.
Das wäre in der Regel produktiver, als die Server zu berennen. Wäre diese Meldung auf Seti@home nicht gekommen, wäre
ich noch verzweifelt und hätte meinem Sohn jedwede weitere Nutzung verboten, weil ich davon aus ging, dass er meinen PC
gegen die Wand gefahren hat.

Man kann sie ja jederzeit per E-Mail reaktivieren.

Mit freundlichen Grüßen
Andreas aka 1080


Hi Andreas!
Weshalb soll(t)en wir unsere Computer für einen oder mehrere Tage offline setzen?
Sicher, dies würde einiges an Energie einsparen. Aber so lange Andere andere nutzlose Dinge tun, sehe ich für mich keinen Grund meine Rechner abzuschalten. So lange Viele in meiner Gegend gegenwärtig Euro um Euro für die größte Drogen-/Alkoholparty genannt "Karneval" oder geschmacklose "Verkleidungen" ausgeben oder ihr Geld auf andere Art verplempern, werde ich meine Rechner weiterhin für sinnvolle(re), wissenschaftliche Berechnungen bereitstellen.
Bitte jetzt keine Diskussion übers Energiesparen - auf dem Ohr bin ich inzwischen taub angesichts der vielen anderweitigen Energieverschwendungen.

Viele von uns arbeiten an mehr als einem Projekt - ich arbeiten an insgesamt vier Projekten mit. Meine Grafikkarte (siehe unten) arbeitet derzeit zu 85-90% der Zeit an Collatz tasks, den Rest der Zeit an Einstein@Home GPU tasks.
Die 20 Prozessoren arbeiten derzeit für SETI@Home CPU tasks sowie Einstein@Home CPU tasks. In Zeiten, in denen das Projekt down ist, switsche ich um auf Milkyway@Home. An diesem Wochenende waren unglücklicherweise beide Projekte down - Milkyway@Home ist es immer noch. Seti@Home arbeitet seit Sonntag Nachmittag wieder. Allerdings befinden sich mehr als 330 beendete Tasks gegenwärtig noch in der Überprüfung.
Ich habe Seti@Home in den vergangenen Monaten zu Gunsten anderer Projekte vernachlässigt. Da aber zumindest bei einem anderen Projekt, Milkyway@Home, schon seit Monaten immer wieder kurzfristig, zum Teil längere, Serverausfälle zu beklagen sind, habe ich Seti@Home reaktiviert. Ich hatte im Vorfeld (am Donnerstag) die Anzahl der gedownloadeten Tasks für alle Projekte deutlich erhöht (eine Eingebung?), doch wurden diese bis Samstag Abend alle abgearbeitet.
Meine Konfiguration: Intel Core i9-7900X@4100 MHz mit Wasserkühlung, 64 GB RAM@3000 MHz, nVidia GTX 1080 Ti mit 11 GB RAM, MS Windows 10 Home.

Auf meinem zweiten Rechner hatte ich die Bearbeitung von Seti@Home zeitweise ausgesetzt und das Projekt angehalten. Doch inzwischen läuft auch dort wieder alles normal.
Dessen Konfiguration: Intel Core i7-8700K@3700 MHz mit Lüftkühlung, 32 GB RAM@3000 MHz, nVidia GTX 1070 Ti mit 8 GB RAM, MS Windows 10 Pro.

Gruß
Manfred
ID: 1983193 · Report as offensive
1 · 2 · Next

Message boards : News : Download server glitch


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.