Warner (Mar 11 2009)

Message boards : Technical News : Warner (Mar 11 2009)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 875049 - Posted: 13 Mar 2009, 0:07:59 UTC - in response to Message 875042.  

How about an IEEE 1394 raid cluster with it's own ps+case? Then can ship whole enclosure an no need for hot swapping.

Just musing outloud here..

Something like this would likely work because the drive connectors aren't being cycled.
ID: 875049 · Report as offensive
Zydor

Send message
Joined: 4 Oct 03
Posts: 172
Credit: 491,111
RAC: 0
United Kingdom
Message 875054 - Posted: 13 Mar 2009, 0:19:37 UTC - in response to Message 874712.  
Last modified: 13 Mar 2009, 0:20:30 UTC

Already said by many - worth repeating nonetheless.

Thank You Matt for taking the time and trouble to Post these Notes. It helps enormously to understand what you guys are doing for all of us, the hassles you go through, and the times when you need the space to do it without excess Posted questions.

During the tough times, keep smiling, there are a huge number of people that you help by what you do. All of you over in the emporium make our lives easier, the silent majority are willing you on, posts or no posts.

There are few things that can be as rewarding in what you all do, nor appreciated so much by so many.

Regards
Zy
ID: 875054 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 31014
Credit: 53,134,872
RAC: 32
United States
Message 875074 - Posted: 13 Mar 2009, 0:58:00 UTC - in response to Message 874712.  

Thanks for the update. Wish I had a solution, but with the strong possibility of Arecibo going dark I know spending money isn't in the cards.

ID: 875074 · Report as offensive
Profile Borgholio
Avatar

Send message
Joined: 2 Aug 99
Posts: 654
Credit: 18,623,738
RAC: 45
United States
Message 875105 - Posted: 13 Mar 2009, 3:12:46 UTC - in response to Message 875074.  

Thanks for the update. Wish I had a solution, but with the strong possibility of Arecibo going dark I know spending money isn't in the cards.


Well if Arecibo does go dark, what would Seti@home likely do? I asked that question in the Seti@home Science forum but never got an answer. :-(

http://setiathome.berkeley.edu/forum_thread.php?id=52239
You will be assimilated...bunghole!

ID: 875105 · Report as offensive
Profile Westsail and *Pyxey*
Volunteer tester
Avatar

Send message
Joined: 26 Jul 99
Posts: 338
Credit: 20,544,999
RAC: 0
United States
Message 875143 - Posted: 13 Mar 2009, 7:31:11 UTC - in response to Message 875105.  


Well if Arecibo does go dark, what would Seti@home likely do? I asked that question in the Seti@home Science forum but never got an answer. :-(



*chuckle*
"The most exciting phrase to hear in science, the one that heralds new discoveries, is not Eureka! (I found it!) but rather, 'hmm... that's funny...'" -- Isaac Asimov
ID: 875143 · Report as offensive
Profile Virtual Boss*
Volunteer tester
Avatar

Send message
Joined: 4 May 08
Posts: 417
Credit: 6,440,287
RAC: 0
Australia
Message 875153 - Posted: 13 Mar 2009, 11:13:17 UTC - in response to Message 875105.  

Well if Arecibo does go dark, what would Seti@home likely do? I asked that question in the Seti@home Science forum but never got an answer. :-(


Here's a crazy idea?

Maybe the crunchers on each continent could put dishes on their roofs,point them all at the samesame part of the sky, sync them with GPS timing and we could start another boinc project to combine all the outputs into one data stream for Seti.

The effective dish size would be orders of magnitude larger than Aricebo, there would be more than one part of the sky being scanned simultaneously, and localised interference would get swamped by the overall signal combining.

High speed internet connection would be needed though. Apologies to dial-up users.
ID: 875153 · Report as offensive
Jesse Viviano

Send message
Joined: 27 Feb 00
Posts: 100
Credit: 3,949,583
RAC: 0
United States
Message 875215 - Posted: 13 Mar 2009, 16:27:25 UTC
Last modified: 13 Mar 2009, 16:28:02 UTC

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB? BD-RE (Blu-ray Rewritable) might or might not be practical because nobody has produced a double-layer version, so they currently max out at 25GB.
ID: 875215 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21241
Credit: 7,508,002
RAC: 20
United Kingdom
Message 875226 - Posted: 13 Mar 2009, 17:30:07 UTC - in response to Message 875215.  

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB?...

How durable are they and what cost for the drives and media?

And will they survive 24/7 operation?

The drives will also need to automount the media or you'll need lots of drives! (There's no students to spare to keep feeding disks!)

Otherwise, how does 2-bay or 4-bay NAS enclosures with Gbit ethernet compare?

Keep searchin',
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 875226 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 875233 - Posted: 13 Mar 2009, 18:05:18 UTC - in response to Message 875215.  
Last modified: 13 Mar 2009, 18:06:29 UTC

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB? BD-RE (Blu-ray Rewritable) might or might not be practical because nobody has produced a double-layer version, so they currently max out at 25GB.


I'm not sure that any double-layer rewritable is possible... If it were technically feasable, wouldn't double-layer rewritable DVD (- or +R) have been available for some time?
.

Hello, from Albany, CA!...
ID: 875233 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 875258 - Posted: 13 Mar 2009, 20:45:18 UTC

WEll if Arecibo goes dark, I guess we could change the focus of the project to distributing the work for NTPCkr?
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 875258 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 875260 - Posted: 13 Mar 2009, 20:47:08 UTC - in response to Message 875215.  

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB? BD-RE (Blu-ray Rewritable) might or might not be practical because nobody has produced a double-layer version, so they currently max out at 25GB.

The multibeam recorder at Arecibo doesn't record those file sizes, it produces a series of data blocks on a 500 GB or larger disk. Recording 14 full data channels plus the radar blanking gives a rate of over 18 MegaBytes/Second. When the disk is getting full another is started. The full disk is eventually shipped back to Berkeley, where the data is pulled off and divided into those 50.2 GB files.

The initial configuration of the multibeam recorder used 300 GB tapes, but that form of removable media didn't work out. I don't know if there's an alternative removable media which might be reliable.

The recorder was also designed to be able to send the data back immediately, that simply requires a connection from Puerto Rico which could pass about 20 MB/S of data at an affordable price.
                                                              Joe
ID: 875260 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 875268 - Posted: 13 Mar 2009, 21:23:12 UTC - in response to Message 875260.  

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB? BD-RE (Blu-ray Rewritable) might or might not be practical because nobody has produced a double-layer version, so they currently max out at 25GB.

The multibeam recorder at Arecibo doesn't record those file sizes, it produces a series of data blocks on a 500 GB or larger disk. Recording 14 full data channels plus the radar blanking gives a rate of over 18 MegaBytes/Second. When the disk is getting full another is started. The full disk is eventually shipped back to Berkeley, where the data is pulled off and divided into those 50.2 GB files.

The initial configuration of the multibeam recorder used 300 GB tapes, but that form of removable media didn't work out. I don't know if there's an alternative removable media which might be reliable.

The recorder was also designed to be able to send the data back immediately, that simply requires a connection from Puerto Rico which could pass about 20 MB/S of data at an affordable price.
                                                              Joe

If I remember correctly from a very old tech news post, the 50gb size was picked because that was the file size limit for the offsite storage..either that, or it was convenient because of that size..or something. That memory is a little bit fuzzy. I do remember hearing something about why that size was chosen.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 875268 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 31014
Credit: 53,134,872
RAC: 32
United States
Message 875325 - Posted: 14 Mar 2009, 1:04:54 UTC - in response to Message 875105.  

Thanks for the update. Wish I had a solution, but with the strong possibility of Arecibo going dark I know spending money isn't in the cards.


Well if Arecibo does go dark, what would Seti@home likely do? I asked that question in the Seti@home Science forum but never got an answer. :-(

http://setiathome.berkeley.edu/forum_thread.php?id=52239


I don't think you will get an official answer, but here is what would happen. After the last data unit is crunched and validated, the data (on the science server) will be checked for likely candidates. Of those other telescope time will be requested for a follow up. Then they get busy and write the papers and publish. This will happen even if they get another data stream from another source for us to crunch.


ID: 875325 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 31014
Credit: 53,134,872
RAC: 32
United States
Message 875332 - Posted: 14 Mar 2009, 1:21:50 UTC - in response to Message 875226.  

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB?...

How durable are they and what cost for the drives and media?

And will they survive 24/7 operation?

The drives will also need to automount the media or you'll need lots of drives! (There's no students to spare to keep feeding disks!)

Otherwise, how does 2-bay or 4-bay NAS enclosures with Gbit ethernet compare?

Keep searchin',
Martin

More importantly can they record data as fast as the telescope generates it? I think they are actually ruled out because we have seven data streams being recorded at once. While being written they are like a tape and not random access like a hard disk. Then there is the cost issue. Even in bulk BD-R's are not cheap yet. A handful of them and you have the price of a SATA drive.

I'm sure Eric knows all the storage options and chose SATA for an important reason. Unfortunate that the connectors have become an issue.

Don't know if any hot swap enclosure manufacturer out there takes a SATA drive puts in on a carrier that uses an e-SATA to mate to the enclosure, but it sounds like that could be the solution.


ID: 875332 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 875339 - Posted: 14 Mar 2009, 1:33:45 UTC - in response to Message 875332.  

I noticed that the maximum data file size on the server is 50.20GB. If you could tell them to record slightly shorter file sizes, could you instead go to double-layered BD-R (Blu-Ray Recordable), which max out at 50GB?...

How durable are they and what cost for the drives and media?

And will they survive 24/7 operation?

The drives will also need to automount the media or you'll need lots of drives! (There's no students to spare to keep feeding disks!)

Otherwise, how does 2-bay or 4-bay NAS enclosures with Gbit ethernet compare?

Keep searchin',
Martin

More importantly can they record data as fast as the telescope generates it? I think they are actually ruled out because we have seven data streams being recorded at once. While being written they are like a tape and not random access like a hard disk. Then there is the cost issue. Even in bulk BD-R's are not cheap yet. A handful of them and you have the price of a SATA drive.

I'm sure Eric knows all the storage options and chose SATA for an important reason. Unfortunate that the connectors have become an issue.

Don't know if any hot swap enclosure manufacturer out there takes a SATA drive puts in on a carrier that uses an e-SATA to mate to the enclosure, but it sounds like that could be the solution.


I think from this point, there are two options. The 5-bay eSATA enclosure I've got is iStar-USA dAge540TL-ES. It uses bare drives (what a trayless design means). True this involves using the SATA connector repeatedly which is the current problem.

The alternative is what I use at work (we have at least a thousand of these in both SATA and IDE), which is by CRU, and is the Dataport 5 (V) Series. You install the drive in the caddy, and then the contacts on the caddy and the frame are rated for 50,000 mating cycles. CDW has them for sale.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 875339 · Report as offensive
Norwich Gadfly
Avatar

Send message
Joined: 29 Dec 08
Posts: 100
Credit: 488,414
RAC: 0
United Kingdom
Message 875395 - Posted: 14 Mar 2009, 8:41:19 UTC - in response to Message 875332.  


More importantly can they record data as fast as the telescope generates it? I think they are actually ruled out because we have seven data streams being recorded at once. While being written they are like a tape and not random access like a hard disk.

Why is it necessary for the data from the telescope to be saved directly to the device used to transport the data to the lab ? Once a "tape" is full, it can be copied at leisure while the telescope is writing to another "tape".
ID: 875395 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14679
Credit: 200,643,578
RAC: 874
United Kingdom
Message 875409 - Posted: 14 Mar 2009, 10:51:48 UTC - in response to Message 875395.  


More importantly can they record data as fast as the telescope generates it? I think they are actually ruled out because we have seven data streams being recorded at once. While being written they are like a tape and not random access like a hard disk.

Why is it necessary for the data from the telescope to be saved directly to the device used to transport the data to the lab ? Once a "tape" is full, it can be copied at leisure while the telescope is writing to another "tape".

Because the copying would have to be done at Arecibo, and SETI@home doesn't have any staff based there. AFAIK, SETI have a pretty limited agreement with Arecibo to supply power to the recorder, and manpower to swap disks, and that's about it. Everything else has to be automated, or it doesn't happen.

When the new multibeam recorder was needed, SETI had to design and build it in their labs at Berkeley (you can't buy something like that off the shelf), and then travel to Arecibo themselves to install it. Any change to the recording/transporting procedure would mean new hardware built at Berkeley, another trip to Arecibo to install it, and a new legal agreement with the telescope operators. That sounds like a last resort to me.
ID: 875409 · Report as offensive
Norwich Gadfly
Avatar

Send message
Joined: 29 Dec 08
Posts: 100
Credit: 488,414
RAC: 0
United Kingdom
Message 875414 - Posted: 14 Mar 2009, 11:32:16 UTC - in response to Message 875409.  
Last modified: 14 Mar 2009, 11:33:08 UTC

Thanks for the explanation !
ID: 875414 · Report as offensive
.clair.

Send message
Joined: 4 Nov 04
Posts: 1300
Credit: 55,390,408
RAC: 69
United Kingdom
Message 875809 - Posted: 15 Mar 2009, 15:28:29 UTC

I like the new layout and wording on the`tasks for user` pages,
it better explains the state of the workunit, thanks.
ID: 875809 · Report as offensive
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 24913
Credit: 3,081,182
RAC: 7
Ireland
Message 876137 - Posted: 16 Mar 2009, 13:23:08 UTC
Last modified: 16 Mar 2009, 13:24:52 UTC

With regards to enclosures, I have one that can be used, either as a usb or rj45 connection. Would these offset the sata problems?

Safecom Network Attached Storage enclosure.
ID: 876137 · Report as offensive
Previous · 1 · 2

Message boards : Technical News : Warner (Mar 11 2009)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.