Warner (Mar 11 2009)


log in

Advanced search

Message boards : Technical News : Warner (Mar 11 2009)

Previous · 1 · 2
Author Message
1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 875049 - Posted: 13 Mar 2009, 0:07:59 UTC - in response to Message 875042.

How about an IEEE 1394 raid cluster with it's own ps+case? Then can ship whole enclosure an no need for hot swapping.

Just musing outloud here..

Something like this would likely work because the drive connectors aren't being cycled.
____________

Zydor
Send message
Joined: 4 Oct 03
Posts: 172
Credit: 491,111
RAC: 0
United Kingdom
Message 875054 - Posted: 13 Mar 2009, 0:19:37 UTC - in response to Message 874712.
Last modified: 13 Mar 2009, 0:20:30 UTC

Already said by many - worth repeating nonetheless.

Thank You Matt for taking the time and trouble to Post these Notes. It helps enormously to understand what you guys are doing for all of us, the hassles you go through, and the times when you need the space to do it without excess Posted questions.

During the tough times, keep smiling, there are a huge number of people that you help by what you do. All of you over in the emporium make our lives easier, the silent majority are willing you on, posts or no posts.

There are few things that can be as rewarding in what you all do, nor appreciated so much by so many.

Regards
Zy
____________

Profile Gary Charpentier
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12147
Credit: 6,427,170
RAC: 8,071
United States
Message 875074 - Posted: 13 Mar 2009, 0:58:00 UTC - in response to Message 874712.

Thanks for the update. Wish I had a solution, but with the strong possibility of Arecibo going dark I know spending money isn't in the cards.

____________

Profile Borgholio
Avatar
Send message
Joined: 2 Aug 99
Posts: 651
Credit: 11,877,139
RAC: 4,163
United States
Message 875105 - Posted: 13 Mar 2009, 3:12:46 UTC - in response to Message 875074.

Thanks for the update. Wish I had a solution, but with the strong possibility of Arecibo going dark I know spending money isn't in the cards.


Well if Arecibo does go dark, what would Seti@home likely do? I asked that question in the Seti@home Science forum but never got an answer. :-(

http://setiathome.berkeley.edu/forum_thread.php?id=52239
____________


You will be assimilated...bunghole!

Profile Westsail and *Pyxey*
Volunteer tester
Avatar
Send message
Joined: 26 Jul 99
Posts: 338
Credit: 20,538,216
RAC: 0
United States
Message 875143 - Posted: 13 Mar 2009, 7:31:11 UTC - in response to Message 875105.


Well if Arecibo does go dark, what would Seti@home likely do? I asked that question in the Seti@home Science forum but never got an answer. :-(



*chuckle*
____________
"The most exciting phrase to hear in science, the one that heralds new discoveries, is not Eureka! (I found it!) but rather, 'hmm... that's funny...'" -- Isaac Asimov

Profile Virtual Boss*
Volunteer tester
Avatar
Send message
Joined: 4 May 08
Posts: 417
Credit: 6,170,354
RAC: 797
Australia
Message 875153 - Posted: 13 Mar 2009, 11:13:17 UTC - in response to Message 875105.

Well if Arecibo does go dark, what would Seti@home likely do? I asked that question in the Seti@home Science forum but never got an answer. :-(


Here's a crazy idea?

Maybe the crunchers on each continent could put dishes on their roofs,point them all at the samesame part of the sky, sync them with GPS timing and we could start another boinc project to combine all the outputs into one data stream for Seti.

The effective dish size would be orders of magnitude larger than Aricebo, there would be more than one part of the sky being scanned simultaneously, and localised interference would get swamped by the overall signal combining.

High speed internet connection would be needed though. Apologies to dial-up users.

Jesse Viviano
Send message
Joined: 27 Feb 00
Posts: 95
Credit: 474,230
RAC: 0
United States
Message 875215 - Posted: 13 Mar 2009, 16:27:25 UTC
Last modified: 13 Mar 2009, 16:28:02 UTC

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB? BD-RE (Blu-ray Rewritable) might or might not be practical because nobody has produced a double-layer version, so they currently max out at 25GB.

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8270
Credit: 4,072,496
RAC: 374
United Kingdom
Message 875226 - Posted: 13 Mar 2009, 17:30:07 UTC - in response to Message 875215.

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB?...

How durable are they and what cost for the drives and media?

And will they survive 24/7 operation?

The drives will also need to automount the media or you'll need lots of drives! (There's no students to spare to keep feeding disks!)

Otherwise, how does 2-bay or 4-bay NAS enclosures with Gbit ethernet compare?

Keep searchin',
Martin

____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 1902
Credit: 9,209,841
RAC: 13,053
United States
Message 875233 - Posted: 13 Mar 2009, 18:05:18 UTC - in response to Message 875215.
Last modified: 13 Mar 2009, 18:06:29 UTC

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB? BD-RE (Blu-ray Rewritable) might or might not be practical because nobody has produced a double-layer version, so they currently max out at 25GB.


I'm not sure that any double-layer rewritable is possible... If it were technically feasable, wouldn't double-layer rewritable DVD (- or +R) have been available for some time?
____________
.

Cosmic_Ocean
Avatar
Send message
Joined: 23 Dec 00
Posts: 2238
Credit: 8,456,293
RAC: 4,183
United States
Message 875258 - Posted: 13 Mar 2009, 20:45:18 UTC

WEll if Arecibo goes dark, I guess we could change the focus of the project to distributing the work for NTPCkr?
____________

Linux laptop uptime: 1484d 22h 42m
Ended due to UPS failure, found 14 hours after the fact

Josef W. Segur
Volunteer developer
Volunteer tester
Send message
Joined: 30 Oct 99
Posts: 4206
Credit: 1,030,844
RAC: 270
United States
Message 875260 - Posted: 13 Mar 2009, 20:47:08 UTC - in response to Message 875215.

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB? BD-RE (Blu-ray Rewritable) might or might not be practical because nobody has produced a double-layer version, so they currently max out at 25GB.

The multibeam recorder at Arecibo doesn't record those file sizes, it produces a series of data blocks on a 500 GB or larger disk. Recording 14 full data channels plus the radar blanking gives a rate of over 18 MegaBytes/Second. When the disk is getting full another is started. The full disk is eventually shipped back to Berkeley, where the data is pulled off and divided into those 50.2 GB files.

The initial configuration of the multibeam recorder used 300 GB tapes, but that form of removable media didn't work out. I don't know if there's an alternative removable media which might be reliable.

The recorder was also designed to be able to send the data back immediately, that simply requires a connection from Puerto Rico which could pass about 20 MB/S of data at an affordable price.
Joe

Cosmic_Ocean
Avatar
Send message
Joined: 23 Dec 00
Posts: 2238
Credit: 8,456,293
RAC: 4,183
United States
Message 875268 - Posted: 13 Mar 2009, 21:23:12 UTC - in response to Message 875260.

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB? BD-RE (Blu-ray Rewritable) might or might not be practical because nobody has produced a double-layer version, so they currently max out at 25GB.

The multibeam recorder at Arecibo doesn't record those file sizes, it produces a series of data blocks on a 500 GB or larger disk. Recording 14 full data channels plus the radar blanking gives a rate of over 18 MegaBytes/Second. When the disk is getting full another is started. The full disk is eventually shipped back to Berkeley, where the data is pulled off and divided into those 50.2 GB files.

The initial configuration of the multibeam recorder used 300 GB tapes, but that form of removable media didn't work out. I don't know if there's an alternative removable media which might be reliable.

The recorder was also designed to be able to send the data back immediately, that simply requires a connection from Puerto Rico which could pass about 20 MB/S of data at an affordable price.
Joe

If I remember correctly from a very old tech news post, the 50gb size was picked because that was the file size limit for the offsite storage..either that, or it was convenient because of that size..or something. That memory is a little bit fuzzy. I do remember hearing something about why that size was chosen.
____________

Linux laptop uptime: 1484d 22h 42m
Ended due to UPS failure, found 14 hours after the fact

Profile Gary Charpentier
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12147
Credit: 6,427,170
RAC: 8,071
United States
Message 875325 - Posted: 14 Mar 2009, 1:04:54 UTC - in response to Message 875105.

Thanks for the update. Wish I had a solution, but with the strong possibility of Arecibo going dark I know spending money isn't in the cards.


Well if Arecibo does go dark, what would Seti@home likely do? I asked that question in the Seti@home Science forum but never got an answer. :-(

http://setiathome.berkeley.edu/forum_thread.php?id=52239


I don't think you will get an official answer, but here is what would happen. After the last data unit is crunched and validated, the data (on the science server) will be checked for likely candidates. Of those other telescope time will be requested for a follow up. Then they get busy and write the papers and publish. This will happen even if they get another data stream from another source for us to crunch.


____________

Profile Gary Charpentier
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12147
Credit: 6,427,170
RAC: 8,071
United States
Message 875332 - Posted: 14 Mar 2009, 1:21:50 UTC - in response to Message 875226.

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB?...

How durable are they and what cost for the drives and media?

And will they survive 24/7 operation?

The drives will also need to automount the media or you'll need lots of drives! (There's no students to spare to keep feeding disks!)

Otherwise, how does 2-bay or 4-bay NAS enclosures with Gbit ethernet compare?

Keep searchin',
Martin

More importantly can they record data as fast as the telescope generates it? I think they are actually ruled out because we have seven data streams being recorded at once. While being written they are like a tape and not random access like a hard disk. Then there is the cost issue. Even in bulk BD-R's are not cheap yet. A handful of them and you have the price of a SATA drive.

I'm sure Eric knows all the storage options and chose SATA for an important reason. Unfortunate that the connectors have become an issue.

Don't know if any hot swap enclosure manufacturer out there takes a SATA drive puts in on a carrier that uses an e-SATA to mate to the enclosure, but it sounds like that could be the solution.


____________

Cosmic_Ocean
Avatar
Send message
Joined: 23 Dec 00
Posts: 2238
Credit: 8,456,293
RAC: 4,183
United States
Message 875339 - Posted: 14 Mar 2009, 1:33:45 UTC - in response to Message 875332.

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB?...

How durable are they and what cost for the drives and media?

And will they survive 24/7 operation?

The drives will also need to automount the media or you'll need lots of drives! (There's no students to spare to keep feeding disks!)

Otherwise, how does 2-bay or 4-bay NAS enclosures with Gbit ethernet compare?

Keep searchin',
Martin

More importantly can they record data as fast as the telescope generates it? I think they are actually ruled out because we have seven data streams being recorded at once. While being written they are like a tape and not random access like a hard disk. Then there is the cost issue. Even in bulk BD-R's are not cheap yet. A handful of them and you have the price of a SATA drive.

I'm sure Eric knows all the storage options and chose SATA for an important reason. Unfortunate that the connectors have become an issue.

Don't know if any hot swap enclosure manufacturer out there takes a SATA drive puts in on a carrier that uses an e-SATA to mate to the enclosure, but it sounds like that could be the solution.


I think from this point, there are two options. The 5-bay eSATA enclosure I've got is iStar-USA dAge540TL-ES. It uses bare drives (what a trayless design means). True this involves using the SATA connector repeatedly which is the current problem.

The alternative is what I use at work (we have at least a thousand of these in both SATA and IDE), which is by CRU, and is the Dataport 5 (V) Series. You install the drive in the caddy, and then the contacts on the caddy and the frame are rated for 50,000 mating cycles. CDW has them for sale.
____________

Linux laptop uptime: 1484d 22h 42m
Ended due to UPS failure, found 14 hours after the fact

Norwich Gadfly
Avatar
Send message
Joined: 29 Dec 08
Posts: 100
Credit: 488,414
RAC: 0
United Kingdom
Message 875395 - Posted: 14 Mar 2009, 8:41:19 UTC - in response to Message 875332.


More importantly can they record data as fast as the telescope generates it? I think they are actually ruled out because we have seven data streams being recorded at once. While being written they are like a tape and not random access like a hard disk.

Why is it necessary for the data from the telescope to be saved directly to the device used to transport the data to the lab ? Once a "tape" is full, it can be copied at leisure while the telescope is writing to another "tape".

Richard Haselgrove
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8375
Credit: 46,774,273
RAC: 23,048
United Kingdom
Message 875409 - Posted: 14 Mar 2009, 10:51:48 UTC - in response to Message 875395.


More importantly can they record data as fast as the telescope generates it? I think they are actually ruled out because we have seven data streams being recorded at once. While being written they are like a tape and not random access like a hard disk.

Why is it necessary for the data from the telescope to be saved directly to the device used to transport the data to the lab ? Once a "tape" is full, it can be copied at leisure while the telescope is writing to another "tape".

Because the copying would have to be done at Arecibo, and SETI@home doesn't have any staff based there. AFAIK, SETI have a pretty limited agreement with Arecibo to supply power to the recorder, and manpower to swap disks, and that's about it. Everything else has to be automated, or it doesn't happen.

When the new multibeam recorder was needed, SETI had to design and build it in their labs at Berkeley (you can't buy something like that off the shelf), and then travel to Arecibo themselves to install it. Any change to the recording/transporting procedure would mean new hardware built at Berkeley, another trip to Arecibo to install it, and a new legal agreement with the telescope operators. That sounds like a last resort to me.

Norwich Gadfly
Avatar
Send message
Joined: 29 Dec 08
Posts: 100
Credit: 488,414
RAC: 0
United Kingdom
Message 875414 - Posted: 14 Mar 2009, 11:32:16 UTC - in response to Message 875409.
Last modified: 14 Mar 2009, 11:33:08 UTC

Thanks for the explanation !

clive G1FYE
Volunteer moderator
Send message
Joined: 4 Nov 04
Posts: 1300
Credit: 23,054,144
RAC: 0
United Kingdom
Message 875809 - Posted: 15 Mar 2009, 15:28:29 UTC

I like the new layout and wording on the`tasks for user` pages,
it better explains the state of the workunit, thanks.

Profile Sirius B
Volunteer tester
Avatar
Send message
Joined: 26 Dec 00
Posts: 10257
Credit: 1,530,132
RAC: 264
United Kingdom
Message 876137 - Posted: 16 Mar 2009, 13:23:08 UTC
Last modified: 16 Mar 2009, 13:24:52 UTC

With regards to enclosures, I have one that can be used, either as a usb or rj45 connection. Would these offset the sata problems?

Safecom Network Attached Storage enclosure.
____________

Previous · 1 · 2

Message boards : Technical News : Warner (Mar 11 2009)

Copyright © 2014 University of California