Panic Mode On (24) Server problems

Abundant work this time. I'm holding roughly like 800 MB of units on my three machines, since data flow started. I put 5 days buffer in preferencies. Overdid it a bit, tho.
____________

Knowledgeable Opinion:

So those of You with Large Caches are causing everyone else to Suffer. You have overwhelmed the Upload Server, the Scheduler and Downloads. Then you wonder when things will die down so that things return to normal...

I am sorry, it is time you start taking the responsibility for those issues. When there are no problems it all works! When there is Problems, it takes days to sort out. The machines that are running in "auto" and "no one" is looking does not care, they probably get theirs first. They are running in "auto." Boinc sorts it out!

Everyone else is Inflating Something! When will some of you learn?

Regards

____________
Please consider a Donation to the Seti Project.

We have very large caches so that we are not affected by these problems

But when you get right down to it, we are using no more bandwidth in the long run by using turned up caches. We are using the exact same amount, but we are getting it all at one time for downloads. For uploads, we usually upload as we complete, just like everyone else.

Perhaps if BOINC put a few minute "gap" between successful downloads, kind of like how 6.6.38 and later doesn't try every upload independently -- if a couple fail, they'll all fail.

If they have different retry times, why fail all of them because the latest attempts failed? Do they all then retry again later, at the same time?

If the coders can eliminate upload server "problems" (quitting transfers after they've started, etc.) I think we'll all be at least a little happier.

Martin

A client SHOULD keep a list of hosts it cannot reach and corresponding connection timeouts, rather than just retrying queued mail items.

Vistro wrote:

We have very large caches so that we are not affected by these problems

But when you get right down to it, we are using no more bandwidth in the long run by using turned up caches. We are using the exact same amount, but we are getting it all at one time for downloads. For uploads, we usually upload as we complete, just like everyone else.

At worst the sched request file was up to 16MB

I need to jump in here.. ;-)

As user of an QUAD_GTX260-216_GPU_cruncher..

All four GPUs make ~ 600 AR 0.44x WU/day [<10 min./GPU].
Shorties [~ 2 min./GPU] 5 times faster so ~ 3,000 WUs/day/whole cruncher.

I test now again BOINC V6.6.38 . [now I'm down to ~ 3 day cache, will test how high is possible]

It's more unstable as BOINC V6.4.7 . [max. ~ 5 day cache]


So higher the WU cache on the GPU cruncher, so longer is the time for the DLs/ULs and the reports to the scheduler.
Yes, it's very strange - the time for UL and DL increase.

Yes.. I think after I posted it nearly everywhere.. so you all know it.. ;-) ..I have only DSL light.. 384/64.. more isn't available in my village and will never be.. :-( ..this is also involved..

____________


>Das Deutsche Cafe. The German Cafe.<

Ruh roh....the replica database just went offline....
Hope that just means they are tweaking things and not another crash.

EDIT....It came back online on the next update.
____________
******

"Ask not, what your kitty can do for you. Ask what you can do for your kitty."

As it is kitten, so shall it be done.

Trying one and skipping the rest drops the load by two orders of magnitude.

Artificially, yes, you've reduced the server workload from that one client, until it retries. But, sometimes it is hard to know how things will play out in a live setting without trying them first.

You do that by reducing the demands on the server.

Dropping transfers mid-stream adds to the load on the server and clients.

Martin

Dropping transfers mid-stream adds to the load on the server and clients.

[...]

We have a model we can copy: the retry logic in SMTP is quite mature.

The basic theory is "if I try to push a message through now, and it does not go, the odds of pushing another message through a minute later are pretty small."

On your second point:

First of all, the client is not the bottleneck, and can be ignored. If you have 20,000 clients trying to upload, and just one server, it seems intuitively obvious to the most casual observer that you really need to only consider the server.

You seem to think that BOINC is directly responsible for aborting transfers: that it lets the transfer start, gets half-way, and then intentionally says "no, stop."

What really happens is: the BOINC client tells the IP stack to open a connection, and when opened, starts dumping in data. The stack transmits it to the server, and the server starts storing the data.

All of the IP protocol is inside the stack. The client and the server are not aware of it.

If there are too many simultaneous connections, the stack, while trying valiantly, will give up, and that is reported on the client side and on the server side.

You fix that by not trying to push 500 megabits through a 100 megabit connection.

... and you do that by making the BOINC client try less.

Cut the connections in half, and you double the available bandwidth for each one. Repeat until you just exactly match the bandwidth available, and you'll be at the maximum throughput.

[edit]You're suggesting that I'm in favor of dropping connections. Quite the opposite. I'm saying that once a connection is made, we need to give it every chance possible of finishing -- by making sure it gets the bandwidth it needs to finish.[/edit]

Dropping transfers mid-stream adds to the load on the server and clients.

Very slightly.

The real problem is that bandwidth is wasted by discarding whatever data had successfully made it through. Worse still, that data then has to be resent.

If the data loss is due to congestion at a link bottleneck, then maintaining a high level of congestion will lead to a disgraceful degradation until you ultimately get no data successfully transferred even though the link appears to be maxed out.

All servers are running, so my question:
Where are my 550,000 Pending?
Look here:

http://setiathome.berkeley.edu/pending.php
____________

All servers are running, so my question:
Where are my 550,000 Pending?
Look here:

http://setiathome.berkeley.edu/pending.php

Ned,

I am only trying to understand the changed upload process. I meant nothing negative about you at all.

As always, I do appreciate your efforts and other folks efforts as well!

Martin
[edited]

All servers are running, so my question:
Where are my 550,000 Pending?
Look here:

http://setiathome.berkeley.edu/pending.php

Better to go in the NC forum [http://setiathome.berkeley.edu/forum_forum.php?id=10] for this kind of questions.. ;-)

If I go this URL [made it clickable], I see only my pending Credits.. your pending Credits aren't available for others.

The pening Credits will granted, if your 'wingmen' will send the results also and if they match..

BTW.
My pending Credits are 244,867.25

-------------------------------------------------------

Es wäre besser für solche Fragen in s NC Forum [http://setiathome.berkeley.edu/forum_forum.php?id=10] zu gehen.. ;-)

Wenn ich der URL folge [machte sie klickbar], sehe ich meine "schwebenden" Credits.. Deine "schwebenden" Credits sind nicht einsehbar für andere.

Die "schwebenden" Credits werden gutgeschrieben, wenn Deine "Flügelmänner" auch ihre Resultate einschicken und sie gleich sind..

Am Rande erwähnt.
Meine "schebenden" Credits sind 244,867.25

[Schwuppdiwupp.. Beiträge wurden verschoben.]



Ach so.. ;-)

Hmm.. komisch.. die Kopie-Datenbank ist eigentlich online.. von der kommen die Daten.
[http://setiathome.berkeley.edu/sah_status.html]

Bei mir dauert es auch machmal bis zu 30 Sekunden bis die Seite fertig geladen hat..

Aber das nichts angezeigt wird ist mir noch nicht passiert.
Wenn dann mit Fehlermeldung.. 'no database access' oder so..


So - don't know where the problem is, why nothing is shown at your pending list.

____________


>Das Deutsche Cafe. The German Cafe.<