Happy Tuesday. We had the usual outage rigamarole today and should be recovering from that in due time. Right after the backup was finished we restarted mysql with full query logging turned on. We knew this would choke the server a bit, and would just be on temporarily. After about a half hour we had over a million queries in the log, so we brought everything back down and turned logging off. We'll parse this log file, and perhaps others we generate over the next 24 hours, in order to find pesky unoptimized queries, anything that would die if we remove all multidimensional indexes, or queries running far more often than we expected.

Also during the outage I moved some big directories around - more NAS shell games. Other than that I've been reconfiguring some more web server stuff (internal use pages) and trying to maximize the raw data pipeline plumbing to get as much work online as possible. It doesn't help that a lot of our raw data drives are showing weird signs of corruption. Don't worry - we do checksums at every important transfer to ensure the data are sound, and the splitters cannot operate on garbage (there are keyword strings occurring regularly throughout the files). Nevertheless, we're having to throw away some files, which is sad. My spider sense tells me this has to do with our general SATA enclosure mounting/unmounting woes. For example, we're finding drives that are 500GB thinking they are 750GB when mounted. Was this because a drive previously on that mount point was 750GB and some bookkeeping bits haven't been cleared? I dunno, but I'm sure this isn't good.

In a couple hours I get to call a number where an automated voice will tell me if I have to attend jury duty tomorrow or not. I get dragged in for potential jury duty an astonishing amount (pretty much the legal maximum) considering I never actually get selected for trial, and never will.

- Matt

---

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Thanks for the update. Happy Thanksgiving, Matt!

---

If the kernel doesn't think the entire SATA device driver has been reloaded, it may keep the partition table from the previous drive. This is bad. This behavior depends a lot on the driver and hardware you're using, but a reboot should always fix it (if that's the case).

Have a happy Thanksgiving!

If the kernel doesn't think the entire SATA device driver has been reloaded, it may keep the partition table from the previous drive. This is bad. This behavior depends a lot on the driver and hardware you're using, but a reboot should always fix it (if that's the case).

If that really is the case, then a reboot isn't needed. Simpler/quicker is just to issue a "hdparm -z /dev/sdX" for whatever the drive is. (Make sure it has a "umount /dev/sdX" first!)

Good luck,
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

Good luck on the jury duty Matt. I was selected for mine starting Monday. Out of curiosity, I called up the jury line this week to see what their schedule was like. No jurors were called all week, so I expect to be called in a somewhat busier week.

---

I get dragged in for potential jury duty an astonishing amount (pretty much the legal maximum) considering I never actually get selected for trial, and never will.

- Matt

This quote elicited a chuckle. I guess Matt is not the typical brain dead drone who tends to remain on the juries after the voir dire process. It is sad that juries are constructed from the most 'normal' of people, those who you really don't want to invite to your parties because the conversations would be about grass growing. I guess that is ok from the accused perspective because, if I were one, then I wouldn't want too creative of a thinker judging me, unless of course I were guilty. On the other hand, it is also sad that these folks are viewed as acceptable peers, which says something about the accused, I suppose.

I believe the raw data is put into the high capacity storage ultimately. So why not put it there to begin with directly from Puerto Rico and download it to Berkeley as needed? Shuffling disks around is proving tedious and error prone, it sounds.

And since this is a Berkeley project, maybe the case could be made that an upload/download scheme would be greener than physically shipping the disks back and forth.

I believe the raw data is put into the high capacity storage ultimately. So why not put it there to begin with directly from Puerto Rico and download it to Berkeley as needed? Shuffling disks around is proving tedious and error prone, it sounds.

Not quite. If I know islands like I do, the "main pipe" out of the island is quite small in bandwidth compared to mainland networks. All the internet connections must share this network pipe. It's probably much faster to ship drives, especially as you send more data.

Now, if we changed SETI's telescope to be somewhere in New Mexico, then there are more options for remote data transfer.

Yeah.. we can collect up to 100s of gigabytes a day. We'd need to be transporting data at (very) roughly 20Mbits a second 24/7 to keep up. We don't have that spare bandwidth, and Arecibo surely does not. About 10 years ago the ENTIRE connection between Arecibo and the rest of the world was a single 56.6 modem. It's much better now, but still not quite up to state-of-the-art speed.

- Matt

---

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Good luck on the jury duty Matt. I was selected for mine starting Monday. Out of curiosity, I called up the jury line this week to see what their schedule was like. No jurors were called all week, so I expect to be called in a somewhat busier week.

And I've been called up for a second time this year - Christmas week. I suspect I won't be going in. :-)

---

I believe the raw data is put into the high capacity storage ultimately. So why not put it there to begin with directly from Puerto Rico and download it to Berkeley as needed? Shuffling disks around is proving tedious and error prone, it sounds.

Not quite. If I know islands like I do, the "main pipe" out of the island is quite small in bandwidth compared to mainland networks. All the internet connections must share this network pipe. It's probably much faster to ship drives, especially as you send more data.

Now, if we changed SETI's telescope to be somewhere in New Mexico, then there are more options for remote data transfer.

All that needs to be done is "park" a communications satellite in the range of the Arecibo Telescope... et voilà, SETI@home's own GB/s backbone ;)

---

Robi

I believe the raw data is put into the high capacity storage ultimately. So why not put it there to begin with directly from Puerto Rico and download it to Berkeley as needed? Shuffling disks around is proving tedious and error prone, it sounds.

Not quite. If I know islands like I do, the "main pipe" out of the island is quite small in bandwidth compared to mainland networks. All the internet connections must share this network pipe. It's probably much faster to ship drives, especially as you send more data.

Now, if we changed SETI's telescope to be somewhere in New Mexico, then there are more options for remote data transfer.

All that needs to be done is "park" a communications satellite in the range of the Arecibo Telescope... et voilà, SETI@home's own GB/s backbone ;)

Which would interfere with the radio astronomy.

---

BOINC WIKI