How about a new thread?

Last night I noticed the assimilators were failing, and this led to the usual conclusion: we ran out of extents on a table in the master database - this time the result table itself. Each result has a very basic entry in this table - it's basically a bridge between its signals and the parent workunit. Anyway, the solution is simple but painful - we gotta rebuild the whole table again.

Luckily, we can do this in parallel with inserting new stuff. So I made a new result table this morning (hence the assimilator queues backing up in the meantime) and then over the next weeks (months) I can silently shovel the older results into this new table. There is a balancing act between size limits and performance. When building these tables we hope to build them big enough so we don't run into these logical barriers, but not so big they don't perform. Sometimes we aim too low, and then we hit these barriers (like running out of extents).

In the case of the Astropulse database, I'm still making a massive backup copy of the whole database as it lives on disk (about 13TB) to archival storage before I do any of the next steps. The plan is still such as it is last I mentioned it - we will begin the reloads from the beginning using a new method that will only take about 2 months. In the meantime we will set up another functionally temporary db to insert new Astropulse data, which we will then merge again after these 2 months (or however long it takes). So we may see AP back on line again in the near future. Anyway, it's still a mess, but there's slow/steady/cautious progress.

I'm keeping "resend lost results" off for now - this functionality has been clobbering the BOINC/mysql database for a while. I think this is partially due to that database being a bit bloating with undigested workunits (i.e. stuff that's been stuck a while due to the Astropulse issues).

- Matt

---

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Thanks for the info

---

Thanks for the update Matt,

Do you know what's happening at Seti Beta with the 'WUs with very unusual Autocorr parameters'?

and why the scheduler there is still down for maintenance?

Edit: Of cause, as soon as I post that, it's no longer down for maintenance.

Claggy

Matt, these updates from you and/or the staff go light years with us crunchers.
Thank you!

Steve

---

Warning, addicted to SETI crunching!
Crunching as a member of GPU Users Group.
GPUUG Website

Thanx for the update Matt.

---

rOZZ
Music
Pictures

Matt, these updates from you and/or the staff go light years with us crunchers.
Thank you!

Steve

Yep, thanks for the update! I can't wait for some AP :)

Yes Thanks Matt

Minor update:

The result table is being merged nicely at this point. Not sure how long it'll take, but we'll monitor its progress.

The astropulse project is also moving forward - I'm back to rebuilding the signal table on the temporary server (the part that will take 6-8 weeks) and am cleaning up/backing up the main server - we might have Astropulse rolling again tomorrow, but more likely Monday. We'll cross that bridge of merging these two databases 8 weeks from now.

In case you're worried this is eating up all my time, it's not. I'm actually mostly working on playing with SERENDIP VI data, making sure it works, helping Jeff track down networking problems between the data sources and the compute nodes down at Arecibo...

- Matt

---

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Great update Matt, can't wait to get some AP's again.

Thanks for keeping us informed and for the dedication to the project.

---

"Sour Grapes make a bitter Whine." <(0)>

There is something that has bothered me for a very long time -

1) How large are the table extents. Are they at their maximum and are succeeding extents after the first n (i.e. 7), the same or progressively lesser then the first and are each succeeding group of extents being done in the same manner?

2) Is IDS (Informix Dynamic Server data compression and storage optimization data compression) being utilized?

---

I don't buy computers, I build them!!

AP is up and starting to validate the backlog. Yay!!!

---

AP is up and starting to validate the backlog. Yay!!!

Woohoo!!

---

rOZZ
Music
Pictures

Okay.. AP, such as it is, is back and creating/assimilating new work. Good. Of course while revving up all these engines lando decided to have a little NFS freakout. I just had to hard power cycle it but it's behaving now. I'll post a more formal news item on Monday, once we're sure things are stable...

- Matt

---

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Thanks Matt !!

Thank you Matt.

Claggy

Oh now it looks like georgem is crashing too with the same NFS errors. Ha ha. Happy Friday afternoon...

- Matt

---

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Oh now it looks like georgem is crashing too with the same NFS errors. Ha ha. Happy Friday afternoon...

- Matt

Ouch, everything bad happens on Friday near quitting time. :(

---

Oh now it looks like georgem is crashing too with the same NFS errors. Ha ha. Happy Friday afternoon...

- Matt

Ouch, everything bad happens on Friday near quitting time. :(

And it is (was here) the 13th, good to see marvin up and running.

---

rOZZ
Music
Pictures

Oh now it looks like georgem is crashing too with the same NFS errors. Ha ha. Happy Friday afternoon...

- Matt

Ouch, everything bad happens on Friday near quitting time. :(

In my extensive experience as an experimental physicist at various institutes around the world, everything bad happens on a Friday afternoon after the technicians have left, but before the scientists start knocking off for the weekend.

---

...Things always break just after the support staff reach the "Point of Unreachablilty".
Thanks for your efforts Matt et-al, its good to see APs back in production :-)

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?