Could it be the recent deadline expiry of the unfinished WOW! tasks? More workunits than usual in the database with three, four, ... task replications?

Edit - the validation backlog is currently 69. It's the "waiting for wingmate" backlog which is 4,453,648.

It's the "waiting for wingmate" backlog which is 4,453,648.

But are we actually waiting on the wingman, or waiting for the Validators to do their work?
I noticed last night when looking at another user's system after they made some changes- all of their returned work over a period of an hour was going in to Pending. All of it. Not a one became Valid or Inconclusive (they had a roughly 3/4 of a day's cache).

So i vote for "Waiting on the Validators to do their work" option. As mentioned before, as much as we are concerned about WUs that take forever to Validate the huge majority do so within a few days. This present issue occurred with a sudden jump just on 3 days ago, and has continued to grow since then.
The usual value is around 3.7 million. You get the occasional spike, but this isn't a spike- it's a protracted period well above the usual number of Results waiting to be validated.
The Haveland graphs show this very clearly (along with the corresponding increase in WU awaiting assimilation (you can't assimilate what has been returned, but hasn't yet been validated)).

I would have thought any after effects of the WoW event would be a bit by bit increase in Awaiting validation numbers as people loaded up on work that wasn't returned, with a noticeable drop each day as WUs then time out & are re-released, Validated & cleared. That certainly isn't happening here.



Edit-
Actually the WU awaiting assimilation numbers are probably the best indicator there is an issue with the Validators.
Normally the Awaiting Validation numbers are the "Waiting on wingman" numbers- the matching result hasn't been returned yet, so the WU can't be validated, and the WU waiting assimilation number is 0.

But now we have work that has been returned by all parities involved, but it's hasn't been validated yet- hence the higher than usual Awaiting Validation & WU awaiting assimilation numbers.

---

Grant
Darwin NT

The system does seem "sluggish". It seems like it is always in that "recovery after an outage" state. I think there is an issue. It is like a hard drive going bad, slowing down the system with retries and errors, but hasn't quite crashed yet.

Awaiting Validation backlog has started to clear (and along with it he WU awaiting assimilation)- dropped by almost half overnight.

---

Grant
Darwin NT

I am aware this requires manual intervention from users. When you receive work I generally look through my task list and see if I have any tasks ending with _2 or above if I do I suspend all work apart from the task currently running and push _2 and above tasks to the front returning them faster in the hope that they will validate and drop the validation/administration backlog

---

I am aware this requires manual intervention from users. When you receive work I generally look through my task list and see if I have any tasks ending with _2 or above if I do I suspend all work apart from the task currently running and push _2 and above tasks to the front returning them faster in the hope that they will validate and drop the validation/administration backlog

This has nothing to do with that.
This is work that has been returned, but due to server issues hasn't yet been Validated.


Interestingly, as the MB Awaiting validation backlog continues to clear, the AP Awaiting validation has started to backup.

---

Grant
Darwin NT

I will have to take your word on that one Grant. I have returned 2 results with _2 and as soon as I have returned them they have turned valid

---

no outage today? or did i miss it?

---

Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

Didn't notice one here either. I only notice cause I tend to use the downtime to update, clean and inspect my systems. Ah well, 1 week without won't hurt.

There is alot going on in the Berkeley area right now. Between fires, smoke, power outages, I'm just happy the system is up and working so well. I hope everyone on the seti team and their loved ones are safe.

There is alot going on in the Berkeley area right now. Between fires, smoke, power outages, I'm just happy the system is up and working so well. I hope everyone on the seti team and their loved ones are safe.

. . Certainly that ... but can the system keep working properly without an outage?

. . Where's the KaBoom?

Stephen

???

There is alot going on in the Berkeley area right now. Between fires, smoke, power outages, I'm just happy the system is up and working so well. I hope everyone on the seti team and their loved ones are safe.

. . Certainly that ... but can the system keep working properly without an outage?

. . Where's the KaBoom?

Stephen

???

Its coming.... on Sunday morning, of course!




ROFLing.... (and under the table).

Tom

---

A proud member of the OFA (Old Farts Association).

. . Where's the KaBoom?

https://www.youtube.com/watch?v=t9wmWZbr_wQ

---

. . Certainly that ... but can the system keep working properly without an outage?
. . Where's the KaBoom?
Stephen

Its coming.... on Sunday morning, of course!
ROFLing.... (and under the table).
Tom

. . Nah! Eric fixed that one ...

{arms behind back with fingers crossed on both hands}

Stephen

:)

Nice short outage, good recovery. Just happened 11 hrs or so later than normal ...

---

Nice short outage, good recovery.

Well, the splitters are still having problems getting up to speed again...

---

Grant
Darwin NT

Twice in the last couple of days I have noticed that one of my Seti@home machines had a pretty good back log of "ready to report" tasks although they appear to have already uploaded.

So I manually kicked off an update. And down comes a bunch of tasks.
Since I am setup to poll once every 5 minutes or so, this is worrying....

Is this my problem or the servers problem?

Tom

---

A proud member of the OFA (Old Farts Association).

It was probably SETI's maintenance outage (during which, you can't report tasks), coupled with BOINC's automatic extending backoffs.

You can't set a 5-minute repeat update: the server tells you to delay that long, as a minimum. You did the right thing, with your single mouse click.

It was probably SETI's maintenance outage (during which, you can't report tasks), coupled with BOINC's automatic extending backoffs.

You can't set a 5-minute repeat update: the server tells you to delay that long, as a minimum. You did the right thing, with your single mouse click.

Sorry I misspoke. I have the "store up additional work" preference set to 0.1 which usually results in a 5 minute update cycle.

Tom

---

A proud member of the OFA (Old Farts Association).

Looks like the splitters have finally recovered form the weekly outage.

---

Grant
Darwin NT