upload issues and other news (Oct 5, 2007)

Message boards : Technical News : upload issues and other news (Oct 5, 2007)
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Jeff Cobb Project Donor
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Mar 99
Posts: 122
Credit: 40,367
RAC: 0
United States
Message 654763 - Posted: 5 Oct 2007, 21:54:54 UTC

Matt is still away on his well deserved vacation so I will summarize the week.

Last weekend we had 3 servers go down, as Eric described in the previous tech note. Two of these were attached to a UPS that malfunctioned. Not good, but at least we understand what happened. The third machine, bruno, crashes every week or two and hangs on reboot for reasons we have yet to understand. Our best guess at this time is that the fiber connection to the disk array that holds the upload directory is sometimes throwing garbage onto the bus that the machine cannot gracefully handle. This is an old fiber array that we would like to phase out anyway, so we thought about different storage devices that we currently have that could hold the uploads. We came up with the underutilized disk space on the master science database machine, thumper. This could have the added benefit of hosting the assimilators on the same machine that hosts the back end science database. Eric ran a script that gradually migrated the uploads over to thumper.

This worked fine until the migration reach a critical point, at which time the loads on the two download machines shot up to the 80-100 range (they are usually at 5 or less). The high loads were because each instance of the file_upload_handler was taking a long time to write the uploaded results over to thumper. To make a long story short, it turns out that the volume on thumper that held the new upload directory was getting slammed by the uploads. It was running at nearly 100% utilization (local disk, not network, utilization). This was, and still is, a bit surprising. The volume on bruno is software RAID50 and on thumper the volume is software RAID5, the latter having 2 more spindles than each of the RAID50 mirrors on bruno. At any rate, we are migrating back to the fiber array on bruno and have already seen download performance normalize. We'll have to figure this one out...

The other systems news of the week involves database replication on both of our production databases. The seti_boinc database (users, hosts, teams, recent results) replica was lost to a machine crash. We restored from the master and the replica is once again running normally. We are getting very close to having a replica of the back end science database. The initial data load is nearly complete. We will turn on replication either over the weekend or early next week.

Over in science development we are getting the splitter ready to handle the radar blanking signal that will be embedded in all new data once Arecibo comes back on line later this month.

-- Jeff
ID: 654763 · Report as offensive
Profile Blurf
Volunteer tester

Send message
Joined: 2 Sep 06
Posts: 8962
Credit: 12,678,685
RAC: 0
United States
Message 654782 - Posted: 5 Oct 2007, 22:35:07 UTC

Thank you for the update Jeff.


ID: 654782 · Report as offensive
Profile petrusbroder

Send message
Joined: 2 Dec 01
Posts: 9
Credit: 55,701,905
RAC: 1
Sweden
Message 654791 - Posted: 5 Oct 2007, 22:59:50 UTC - in response to Message 654763.  

Thanks for the info, I'll post a link @ TeAm AnandTech - there have been a few ;) questions ...
ID: 654791 · Report as offensive
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 654827 - Posted: 6 Oct 2007, 0:29:06 UTC


Thanks Immensely for the Update Jeff . . . Nice Work Berkeley . . .

ID: 654827 · Report as offensive
DJStarfox

Send message
Joined: 23 May 01
Posts: 1066
Credit: 1,226,053
RAC: 2
United States
Message 654882 - Posted: 6 Oct 2007, 2:25:06 UTC - in response to Message 654763.  

I think that damn military radar has been the source of the -9 errors and noise in a lot of the WU in the last 6 months or so.
ID: 654882 · Report as offensive
Profile Jim-R.
Volunteer tester
Avatar

Send message
Joined: 7 Feb 06
Posts: 1494
Credit: 194,148
RAC: 0
United States
Message 654941 - Posted: 6 Oct 2007, 4:12:26 UTC - in response to Message 654882.  

I think that damn military radar has been the source of the -9 errors and noise in a lot of the WU in the last 6 months or so.

It is in *all* of the multibeam data. As I recall, the linefeed array was lower down on the structure so the radar signal was blocked by a nearby hill. The multibeam array is positioned higher so it receives the signal. The splitters have been configured to remove most of it, but some still gets through. Right now it's been a tradeoff between the amount of radar signal in the work units and the amount of good data that is not split due to the possibility of radar contamination. When they get the new setup working they can remove the radar signal and *only* the radar signal.
Jim

Some people plan their life out and look back at the wealth they've had.
Others live life day by day and look back at the wealth of experiences and enjoyment they've had.
ID: 654941 · Report as offensive
JLDun
Volunteer tester
Avatar

Send message
Joined: 21 Apr 06
Posts: 573
Credit: 196,101
RAC: 0
United States
Message 655042 - Posted: 6 Oct 2007, 10:04:19 UTC - in response to Message 654941.  

I think that damn military radar has been the source of the -9 errors and noise in a lot of the WU in the last 6 months or so.

It is in *all* of the multibeam data. As I recall, the linefeed array was lower down on the structure so the radar signal was blocked by a nearby hill. The multibeam array is positioned higher so it receives the signal. The splitters have been configured to remove most of it, but some still gets through. Right now it's been a tradeoff between the amount of radar signal in the work units and the amount of good data that is not split due to the possibility of radar contamination. When they get the new setup working they can remove the radar signal and *only* the radar signal.

Thanks for this explanation.


When they get the new setup working they can remove the radar signal and *only* the radar signal.

I know I've seen an explanation before, but how can SETI 'be sure' they've removed the "d*** military radar" from the WU's? Without going through every second of WU data before it's sent?
ID: 655042 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14644
Credit: 200,643,578
RAC: 874
United Kingdom
Message 655067 - Posted: 6 Oct 2007, 10:27:03 UTC - in response to Message 655042.  

I know I've seen an explanation before, but how can SETI 'be sure' they've removed the "d*** military radar" from the WU's? Without going through every second of WU data before it's sent?

There's a Scientific Newsletter article about it.
ID: 655067 · Report as offensive
Scrat

Send message
Joined: 12 Jul 00
Posts: 2
Credit: 254,582
RAC: 1
Austria
Message 655268 - Posted: 6 Oct 2007, 17:45:42 UTC

If you have pending problems with garbage on the fibre cables you could check the mass points on the system.

One problem could be that the shielding of the fibre cables is directly connected on both ends of the cable. Its strongly recommended to connect ONLY one point directly with the mass point and the other over a spezial surge protector.

Another problem could be that the mass points of the server array is to long. If you connect too many servers over a larger distance with the same mass point the equalizing current of the mass wire could also cause serious problems on the sending and receiving modules. (You know these problems if you ever connected your hifi with the television....)

These two problems can occour and cause such problems. Mostly this problems are not taken care of at the construction of the servers, cause of the increased costs of building.

I already had such problems..... maybe i could help.....


Excuse me for the english.... its a long time since last using it ;)
ID: 655268 · Report as offensive
Profile Clyde C. Phillips, III

Send message
Joined: 2 Aug 00
Posts: 1851
Credit: 5,955,047
RAC: 0
United States
Message 655284 - Posted: 6 Oct 2007, 18:29:14 UTC

I'm suggesting making an occulting screen probably something like that of a drive-in movie theatre but that just may be too expensive and on someone elses property. Also the probability of hurricanes and tornadoes. Maybe a mesh, rather than a light-opaque structure would be just as effective.
ID: 655284 · Report as offensive
Profile Odan

Send message
Joined: 8 May 03
Posts: 91
Credit: 15,331,177
RAC: 0
United Kingdom
Message 655312 - Posted: 6 Oct 2007, 19:13:41 UTC - in response to Message 655268.  

If you have pending problems with garbage on the fibre cables you could check the mass points on the system .....


Scrat, the fibres are _optical_ fibres so electrical problems such as grounding (mass?) do not have any effect.

Excuse me for the english.... its a long time since last using it ;)

Your English is fine :~}
ID: 655312 · Report as offensive
Profile Jim-R.
Volunteer tester
Avatar

Send message
Joined: 7 Feb 06
Posts: 1494
Credit: 194,148
RAC: 0
United States
Message 655315 - Posted: 6 Oct 2007, 19:15:32 UTC - in response to Message 655284.  

I'm suggesting making an occulting screen probably something like that of a drive-in movie theatre but that just may be too expensive and on someone elses property. Also the probability of hurricanes and tornadoes. Maybe a mesh, rather than a light-opaque structure would be just as effective.

Hmm, good suggestion. It wouldn't have to be very big, just big enough to block the radar signal from the receiving antennas. Unless there's reflections from some other structure. It would have to be made of something opaque to the freq of the radar. Maybe the sort of radar absorbing material they use on the stealth and other planes. Absorbing the signal would be preferable to reflecting it for the radar op's sake! hehe Unless it were reflected at an angle so as not to bounce straight back at them!
Jim

Some people plan their life out and look back at the wealth they've had.
Others live life day by day and look back at the wealth of experiences and enjoyment they've had.
ID: 655315 · Report as offensive
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 655352 - Posted: 6 Oct 2007, 20:15:17 UTC - in response to Message 655315.  

I'm suggesting making an occulting screen probably something like that of a drive-in movie theatre but that just may be too expensive and on someone elses property. Also the probability of hurricanes and tornadoes. Maybe a mesh, rather than a light-opaque structure would be just as effective.

Hmm, good suggestion. It wouldn't have to be very big, just big enough to block the radar signal from the receiving antennas. Unless there's reflections from some other structure. It would have to be made of something opaque to the freq of the radar. Maybe the sort of radar absorbing material they use on the stealth and other planes. Absorbing the signal would be preferable to reflecting it for the radar op's sake! hehe Unless it were reflected at an angle so as not to bounce straight back at them!



The First Invisibility Shield . . . Technology mighten be 'incorporated into' said 'meshin' . . . hmmm?
ID: 655352 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 655371 - Posted: 6 Oct 2007, 20:43:36 UTC - in response to Message 655268.  

If you have pending problems with garbage on the fibre cables you could check the mass points on the system.

One problem could be that the shielding of the fibre cables is directly connected on both ends of the cable. Its strongly recommended to connect ONLY one point directly with the mass point and the other over a spezial surge protector.

Another problem could be that the mass points of the server array is to long. If you connect too many servers over a larger distance with the same mass point the equalizing current of the mass wire could also cause serious problems on the sending and receiving modules. (You know these problems if you ever connected your hifi with the television....)

These two problems can occour and cause such problems. Mostly this problems are not taken care of at the construction of the servers, cause of the increased costs of building.

I already had such problems..... maybe i could help.....


Excuse me for the english.... its a long time since last using it ;)


Sorry, but you are confusing electrical signal cables with fiber optic cables. The grounding advice you gave is well applied to electrical cables, but fiber optics are immune to RFI and other electical interference, and do not require shielding which is earthed (grounded).
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 655371 · Report as offensive
Profile Andy Lee Robinson
Avatar

Send message
Joined: 8 Dec 05
Posts: 630
Credit: 59,973,836
RAC: 0
Hungary
Message 655391 - Posted: 6 Oct 2007, 21:18:14 UTC - in response to Message 655315.  
Last modified: 6 Oct 2007, 21:19:53 UTC

Absorbing the signal would be preferable to reflecting it for the radar op's sake! hehe Unless it were reflected at an angle so as not to bounce straight back at them!


Well I'd send it back to them amplified a few billion times... but that's me!

Problem with any amount of terrestrial rfi, it massively outweighs those we are looking for. Blocking rfi is difficult because of diffraction and reradiation, and the pylons will also resonate and reradiate all kinds of frequencies and pick up much more than just the radar. A huge multiwalled Faraday cage might help.

Damned inconvenient to have one within line-of-sight-and-a-bit-more near Arecibo. The power of these things can be 10s of kW so even when not facing the antenna, leakage could still be considerable.

Those responsible for the radar should also be responsible for it not interfering with their neighbours, and a radar absorber/attenuator/blocker or switcher offer would be more effective closer to the source.

Saving grace is that the signal can be neutralized somewhat, or skipped over but there can still be annoying traces.

Ideally would need to build a really big mountain around the dish and turn it
into a large tube, or move to the far side of the Moon and recruit a few convenient craters.
ID: 655391 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 655396 - Posted: 6 Oct 2007, 21:24:36 UTC - in response to Message 655391.  

Absorbing the signal would be preferable to reflecting it for the radar op's sake! hehe Unless it were reflected at an angle so as not to bounce straight back at them!


Well I'd send it back to them amplified a few billion times... but that's me!

Problem with any amount of terrestrial rfi, it massively outweighs those we are looking for. Blocking rfi is difficult because of diffraction and reradiation, and the pylons will also resonate and reradiate all kinds of frequencies and pick up much more than just the radar. A huge multiwalled Faraday cage might help.

Damned inconvenient to have one within line-of-sight-and-a-bit-more near Arecibo. The power of these things can be 10s of kW so even when not facing the antenna, leakage could still be considerable.

Those responsible for the radar should also be responsible for it not interfering with their neighbours, and a radar absorber/attenuator/blocker or switcher offer would be more effective closer to the source.

Saving grace is that the signal can be neutralized somewhat, or skipped over but there can still be annoying traces.

Ideally would need to build a really big mountain around the dish and turn it
into a large tube, or move to the far side of the Moon and recruit a few convenient craters.

If it were a civilian installation, they might be held responsible for the problem, but being a military installation, no such restrictions apply.

"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 655396 · Report as offensive
Scrat

Send message
Joined: 12 Jul 00
Posts: 2
Credit: 254,582
RAC: 1
Austria
Message 655481 - Posted: 7 Oct 2007, 0:03:39 UTC - in response to Message 655371.  



Sorry, but you are confusing electrical signal cables with fiber optic cables. The grounding advice you gave is well applied to electrical cables, but fiber optics are immune to RFI and other electical interference, and do not require shielding which is earthed (grounded).


:) I know that fibre cables normally have no shielding, but sometimes the rodent protection of the cables is connected to the sending and the receiving modul, or to the server rack. This can cause such problems, not because of electrical interference in the cable itself, but in the sending or receiving modul.


I just wanted to mention it, cause i had such problems twice (with fibre optic) and it took me 2 months to locate the problem.
ID: 655481 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30593
Credit: 53,134,872
RAC: 32
United States
Message 655484 - Posted: 7 Oct 2007, 0:07:23 UTC - in response to Message 655315.  
Last modified: 7 Oct 2007, 0:16:03 UTC

Don't worry about sending it back to them. Their software will just remove it as ground clutter.

Gary

I'm suggesting making an occulting screen probably something like that of a drive-in movie theatre but that just may be too expensive and on someone elses property. Also the probability of hurricanes and tornadoes. Maybe a mesh, rather than a light-opaque structure would be just as effective.

Hmm, good suggestion. It wouldn't have to be very big, just big enough to block the radar signal from the receiving antennas. Unless there's reflections from some other structure. It would have to be made of something opaque to the freq of the radar. Maybe the sort of radar absorbing material they use on the stealth and other planes. Absorbing the signal would be preferable to reflecting it for the radar op's sake! hehe Unless it were reflected at an angle so as not to bounce straight back at them!


ID: 655484 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 655489 - Posted: 7 Oct 2007, 0:12:43 UTC - in response to Message 655481.  



Sorry, but you are confusing electrical signal cables with fiber optic cables. The grounding advice you gave is well applied to electrical cables, but fiber optics are immune to RFI and other electical interference, and do not require shielding which is earthed (grounded).


:) I know that fibre cables normally have no shielding, but sometimes the rodent protection of the cables is connected to the sending and the receiving modul, or to the server rack. This can cause such problems, not because of electrical interference in the cable itself, but in the sending or receiving modul.


I just wanted to mention it, cause i had such problems twice (with fibre optic) and it took me 2 months to locate the problem.

I suppose if the 2 interconnected pieces of equipment were not on the same ground bus there could be some differential between them thus inducing some current flow from one to the other. Fair enough. When you don't have an obvious solution to a problem, you must look for other obscure causes.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 655489 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13715
Credit: 208,696,464
RAC: 304
Australia
Message 655492 - Posted: 7 Oct 2007, 0:15:19 UTC - in response to Message 655489.  

I suppose if the 2 interconnected pieces of equipment were not on the same ground bus there could be some differential between them thus inducing some current flow from one to the other.

All too common.
Earth loops can be an nightmare to track down & resolve.

Grant
Darwin NT
ID: 655492 · Report as offensive
1 · 2 · Next

Message boards : Technical News : upload issues and other news (Oct 5, 2007)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.