Minor issues over the weekend. One night penguin (the download server) got in a snit with the network and needed to be rebooted. No big deal there, except that traffic was vastly reduced for several hours there. Of greater concern was the swelling ready-to-assimilate queue. Normally this wouldn't be that big a deal and could wait until Monday to diagnose, but this backlog left extra workunits on disk (since they have to be assimilated before they can be deleted). Add this to our lower quorums and rising results-to-send queue, and the workunit file system almost filled up! I had to halt splitting for a while to keep this from happening. I also tried adding extra assimilator processes but this didn't help.

Jeff found the problem this morning: some new assimilator code to update the "hot pix" table in the science database was doing sequential scans for row updates. A simple "update stats" on the informix table cleaned that right up quick. The "hot pix" table will be used for the near time persistency checker (yep - we're actually working on that stuff slowly but surely). The queue, and therefore the workunit storage usage, should be draining now.

Today I've been working on getting new disk volumes on line (a continuation from my last post). Not sure why I didn't know this already, but it turns out the ext3 filesystem has an 8 Terabyte limit. So we had to adjust certain plans for volume configuration until they come out with ext4. I have no time or interest in trying any other filesystems at this point.

Last night woken up around 3:00am by a nearby 2.3 earthquake and again at 3:10am by a 2.4 at the same exact location. Actually this has been an active hot spot for the past year - right at the base of the Claremont Hotel (about a mile or two away from campus). Tonight I'll be up again around the same time to catch the full lunar eclipse, or at least I'll try to be. I'm kinda wrecked.

- Matt

---

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

There are 48bit patches on sourceforge to make 1024 PByte possible - but I think still marked experimental...

However, from what I read - and experienced! - the e2fsck time will be incredibly large - about 2 hours for 3TB in my case.

---

mic.

Thanks Immensely for the Update Matt - It is Appreciated Sir!

> reminders of 25 Years Experience w/ California Quakes - oh Lord - i won't go there . . .

< nice work by Each of You @ Berkeley - Keep it up . . .

Matt
The clogged servers could probably explain a problem I've been having lately....v-e-e-r-r-y slow uploads and downloads. I have a cable modem connection and have been noticing download and upload speeds of less than 5000 kbps. I thought maybe my connection was having the issue, but all my other downloads(software, my other BOINC project, etc) have been up to speed.

Any feedback would be helpful. Thanks.

---

Matt
The clogged servers could probably explain a problem I've been having lately....v-e-e-r-r-y slow uploads and downloads. I have a cable modem connection and have been noticing download and upload speeds of less than 5000 kbps. I thought maybe my connection was having the issue, but all my other downloads(software, my other BOINC project, etc) have been up to speed.

Any feedback would be helpful. Thanks.

5000 kbps or 5mbps or 5,000,000 bps? This does not seem to bad to me at all. If you meant 5000 bps or 5kbps, then there is a problem someplace.

---

BOINC WIKI

Thanks for the update.

You must be a light sleeper. 2.3 wakes you up? Loma Prieta in 1989 allowed me to calibrate what it takes to wake me (we lived in south San Jose at the time, 6 miles from the epicenter of the main quake) and anything less than 5.0 and I don't even stir. :-)

Keep up the good work.

---

Matt
The clogged servers could probably explain a problem I've been having lately....v-e-e-r-r-y slow uploads and downloads. I have a cable modem connection and have been noticing download and upload speeds of less than 5000 kbps. I thought maybe my connection was having the issue, but all my other downloads(software, my other BOINC project, etc) have been up to speed.

Any feedback would be helpful. Thanks.

5000 kbps or 5mbps or 5,000,000 bps? This does not seem to bad to me at all. If you meant 5000 bps or 5kbps, then there is a problem someplace.

My last three SETI downloads were at 5573, 4496, and 3810 bytes per sec according to the BOINC Manager log. Call that ~50,000 bits per sec average. That's a lot slower than the 2,336,000 bits per second my ADSL router is currently capable of pulling off the internet.

I think the difference is that there currently seems to be a long period of 'waiting time' (of the order of a minute) after BOINC has made the connection, but before the actual data starts flowing. 65 seconds of waiting, followed by 10 seconds transfer at 36.7 KBps, makes the figures average to the 5 KBps shown in the logs. Edit: it's a bit like air travel - the planes themselves are fast, but if you add in all the time spent hanging around in the terminal, and collecting your bags at the other end, then the average doesn't look so hot anymore.

There's some more discussion of this and related questions in Number Crunching at the moment, in the 21 second timeout question.... thread, if anyone wants to continue the conversation.

My last three SETI downloads were at 5573, 4496, and 3810 bytes per sec according to the BOINC Manager log. Call that ~50,000 bits per sec average. That's a lot slower than the 2,336,000 bits per second my ADSL router is currently capable of pulling off the internet.

Yes, 50Kbps is slow. 5000Kbps isn't.

---

BOINC WIKI

I've been getting several days of the following:

8/28/2007 5:33:13 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 5:33:35 PM||Project communication failed: attempting access to reference site
8/28/2007 5:33:35 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.11719.7843.8.5.40: system connect
8/28/2007 5:33:35 PM|SETI@home|Backing off 36 min 24 sec on download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 5:33:41 PM||Access to reference site succeeded - project servers may be temporarily down.
8/28/2007 5:44:24 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 5:44:46 PM||Project communication failed: attempting access to reference site
8/28/2007 5:44:46 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.12318.4980.9.5.209: system connect
8/28/2007 5:44:46 PM|SETI@home|Backing off 57 min 57 sec on download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 5:44:47 PM||Access to reference site succeeded - project servers may be temporarily down.
8/28/2007 6:10:00 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 6:10:22 PM||Project communication failed: attempting access to reference site
8/28/2007 6:10:22 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.11719.7843.8.5.40: system connect
8/28/2007 6:10:22 PM|SETI@home|Backing off 1 hr 2 min 16 sec on download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 6:10:26 PM||Access to reference site succeeded - project servers may be temporarily down.
8/28/2007 6:42:45 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 6:43:08 PM||Project communication failed: attempting access to reference site
8/28/2007 6:43:08 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.12318.4980.9.5.209: system connect
8/28/2007 6:43:08 PM|SETI@home|Backing off 3 hr 21 min 17 sec on download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 6:43:10 PM||Access to reference site succeeded - project servers may be temporarily down.
8/28/2007 6:51:38 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 6:51:41 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 6:51:59 PM||Project communication failed: attempting access to reference site
8/28/2007 6:51:59 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.12318.4980.9.5.209: system connect
8/28/2007 6:51:59 PM|SETI@home|Backing off 3 hr 24 min 32 sec on download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 6:52:00 PM||Access to reference site succeeded - project servers may be temporarily down.
8/28/2007 6:52:02 PM||Project communication failed: attempting access to reference site
8/28/2007 6:52:02 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.11719.7843.8.5.40: system connect
8/28/2007 6:52:02 PM|SETI@home|Backing off 3 hr 46 min 18 sec on download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 6:52:04 PM||Access to reference site succeeded - project servers may be temporarily down.

I have a DSL connection and 2 GB of RAM. Any thoughts?

Thanks.

---

My last three SETI downloads were at 5573, 4496, and 3810 bytes per sec according to the BOINC Manager log. Call that ~50,000 bits per sec average. That's a lot slower than the 2,336,000 bits per second my ADSL router is currently capable of pulling off the internet.

Yes, 50Kbps is slow. 5000Kbps isn't.

You're right-I just checked again, and it should have been 5000 bytes, not kbytes.

---

I've been getting several days of the following:

8/28/2007 5:33:13 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 5:33:35 PM||Project communication failed: attempting access to reference site
8/28/2007 5:33:35 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.11719.7843.8.5.40: system connect
8/28/2007 5:33:35 PM|SETI@home|Backing off 36 min 24 sec on download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 5:33:41 PM||Access to reference site succeeded - project servers may be temporarily down.
8/28/2007 5:44:24 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 5:44:46 PM||Project communication failed: attempting access to reference site
8/28/2007 5:44:46 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.12318.4980.9.5.209: system connect
8/28/2007 5:44:46 PM|SETI@home|Backing off 57 min 57 sec on download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 5:44:47 PM||Access to reference site succeeded - project servers may be temporarily down.
8/28/2007 6:10:00 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 6:10:22 PM||Project communication failed: attempting access to reference site
8/28/2007 6:10:22 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.11719.7843.8.5.40: system connect
8/28/2007 6:10:22 PM|SETI@home|Backing off 1 hr 2 min 16 sec on download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 6:10:26 PM||Access to reference site succeeded - project servers may be temporarily down.
8/28/2007 6:42:45 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 6:43:08 PM||Project communication failed: attempting access to reference site
8/28/2007 6:43:08 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.12318.4980.9.5.209: system connect
8/28/2007 6:43:08 PM|SETI@home|Backing off 3 hr 21 min 17 sec on download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 6:43:10 PM||Access to reference site succeeded - project servers may be temporarily down.
8/28/2007 6:51:38 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 6:51:41 PM|SETI@home|[file_xfer] Started download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 6:51:59 PM||Project communication failed: attempting access to reference site
8/28/2007 6:51:59 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.12318.4980.9.5.209: system connect
8/28/2007 6:51:59 PM|SETI@home|Backing off 3 hr 24 min 32 sec on download of file 11ja07aa.12318.4980.9.5.209
8/28/2007 6:52:00 PM||Access to reference site succeeded - project servers may be temporarily down.
8/28/2007 6:52:02 PM||Project communication failed: attempting access to reference site
8/28/2007 6:52:02 PM|SETI@home|[file_xfer] Temporarily failed download of 11ja07aa.11719.7843.8.5.40: system connect
8/28/2007 6:52:02 PM|SETI@home|Backing off 3 hr 46 min 18 sec on download of file 11ja07aa.11719.7843.8.5.40
8/28/2007 6:52:04 PM||Access to reference site succeeded - project servers may be temporarily down.

I have a DSL connection and 2 GB of RAM. Any thoughts?



Thanks.

No answers, but the same messages....I don't have as much RAM, but am using a cable modem connection.

---

have a try @ this:

With Windows:


Shut down BOINC-Manager.
On the Commandline (run) type: ipconfig /flushdns
<enter> then
On the Commandline (run) type: ipconfig /registerdns
<enter> then . . .
Restart BOINC-Manager


This solves HTTP-Problems . . .

have a try @ this:

With Windows:


Shut down BOINC-Manager.
On the Commandline (run) type: ipconfig /flushdns
<enter> then
On the Commandline (run) type: ipconfig /registerdns
<enter> then . . .
Restart BOINC-Manager


This solves HTTP-Problems . . .

The ipconfig /flushdns is definitely worth doing on Windows XP. The registerdns bit is not necessary as discussed in a previous thread some months ago.

This will not do anything on Windows 98, I tried it yesterday when my Win 98 box had a stuck download. Not sure for other flavors of Windows.

I also noticed that a lot of Results/WUs are being sent out within 2 - 5 seconds of creation. I think this indicates that one of the processes (the splitters ? ) is or was under pressure to keep up.

---

Sir Arthur C Clarke 1917-2008

Matt
The clogged servers could probably explain a problem I've been having lately....v-e-e-r-r-y slow uploads and downloads. I have a cable modem connection and have been noticing download and upload speeds of less than 5000 kbps. I thought maybe my connection was having the issue, but all my other downloads(software, my other BOINC project, etc) have been up to speed.

Any feedback would be helpful. Thanks.

5000 kbps or 5mbps or 5,000,000 bps? This does not seem to bad to me at all. If you meant 5000 bps or 5kbps, then there is a problem someplace.

My last three SETI downloads were at 5573, 4496, and 3810 bytes per sec according to the BOINC Manager log. Call that ~50,000 bits per sec average. That's a lot slower than the 2,336,000 bits per second my ADSL router is currently capable of pulling off the internet.

I think the difference is that there currently seems to be a long period of 'waiting time' (of the order of a minute) after BOINC has made the connection, but before the actual data starts flowing. 65 seconds of waiting, followed by 10 seconds transfer at 36.7 KBps, makes the figures average to the 5 KBps shown in the logs. Edit: it's a bit like air travel - the planes themselves are fast, but if you add in all the time spent hanging around in the terminal, and collecting your bags at the other end, then the average doesn't look so hot anymore.

There's some more discussion of this and related questions in Number Crunching at the moment, in the 21 second timeout question.... thread, if anyone wants to continue the conversation.

I think this supposition is confirmed. I've just watched this download happening:

29/08/2007 11:39:53|SETI@home|[file_xfer] Started download of file 11ja07ab.927.4980.11.5.122
29/08/2007 11:41:30|SETI@home|[file_xfer] Finished download of file 11ja07ab.927.4980.11.5.122
29/08/2007 11:41:30|SETI@home|[file_xfer] Throughput 3924 bytes/sec

As you can see, the download apparently lasted for 1 minute 37 seconds, but by my reckoning nothing happened in the first 1 minute 25 seconds: all the data arrived in the final 12 seconds, at a perfectly respectable speed.

Unfortunately, the discussion thread I linked to yesterday has been hidden, for an unfathomable and unexplained "administrative purpose". I suggest future discussion should take place in Number Crunching, not here, but someone will need to start a new thread if they want to carry on.

Matt
The clogged servers could probably explain a problem I've been having lately....v-e-e-r-r-y slow uploads and downloads. I have a cable modem connection and have been noticing download and upload speeds of less than 5000 kbps. I thought maybe my connection was having the issue, but all my other downloads(software, my other BOINC project, etc) have been up to speed.

Any feedback would be helpful. Thanks.

5000 kbps or 5mbps or 5,000,000 bps? This does not seem to bad to me at all. If you meant 5000 bps or 5kbps, then there is a problem someplace.

My last three SETI downloads were at 5573, 4496, and 3810 bytes per sec according to the BOINC Manager log. Call that ~50,000 bits per sec average. That's a lot slower than the 2,336,000 bits per second my ADSL router is currently capable of pulling off the internet.

I think the difference is that there currently seems to be a long period of 'waiting time' (of the order of a minute) after BOINC has made the connection, but before the actual data starts flowing. 65 seconds of waiting, followed by 10 seconds transfer at 36.7 KBps, makes the figures average to the 5 KBps shown in the logs. Edit: it's a bit like air travel - the planes themselves are fast, but if you add in all the time spent hanging around in the terminal, and collecting your bags at the other end, then the average doesn't look so hot anymore.

There's some more discussion of this and related questions in Number Crunching at the moment, in the 21 second timeout question.... thread, if anyone wants to continue the conversation.

I think this supposition is confirmed. I've just watched this download happening:

29/08/2007 11:39:53|SETI@home|[file_xfer] Started download of file 11ja07ab.927.4980.11.5.122
29/08/2007 11:41:30|SETI@home|[file_xfer] Finished download of file 11ja07ab.927.4980.11.5.122
29/08/2007 11:41:30|SETI@home|[file_xfer] Throughput 3924 bytes/sec

As you can see, the download apparently lasted for 1 minute 37 seconds, but by my reckoning nothing happened in the first 1 minute 25 seconds: all the data arrived in the final 12 seconds, at a perfectly respectable speed.

Unfortunately, the discussion thread I linked to yesterday has been hidden, for an unfathomable and unexplained "administrative purpose". I suggest future discussion should take place in Number Crunching, not here, but someone will need to start a new thread if they want to carry on.

I had a recent download come down at 3200 bps(+-) However, the uploads are going at 50000+ bps. The way my provider works is usually the downloads are faster (?)

---

have a try @ this:

With Windows:


Shut down BOINC-Manager.
On the Commandline (run) type: ipconfig /flushdns
<enter> then
On the Commandline (run) type: ipconfig /registerdns
<enter> then . . .
Restart BOINC-Manager


This solves HTTP-Problems . . .

The ipconfig /flushdns is definitely worth doing on Windows XP. The registerdns bit is not necessary as discussed in a previous thread some months ago.

This will not do anything on Windows 98, I tried it yesterday when my Win 98 box had a stuck download. Not sure for other flavors of Windows.

I also noticed that a lot of Results/WUs are being sent out within 2 - 5 seconds of creation. I think this indicates that one of the processes (the splitters ? ) is or was under pressure to keep up.

This should work on Win98:

Shutdown BOINC
WIN-R ---> cmd
ipconfig /renew_all

---

This should work on Win98:

Shutdown BOINC
WIN-R ---> cmd
ipconfig /renew_all

This would renew all the DHCP leases, but I don't think it flushes any local DNS cache.

Windows 98 used WINS to resolve NETBIOS names, 2K and later use a Microsoft-kluged DNS, which is why the later versions of IPCONFIG can flush DNS (and have the registerdns command to update the DNS server).

---

As for the ext3 8TiB limit, ext3 will only go to 8TiB limit if you set it up for 2KiB block size. 4KiB block size will take you to 16TiB, and only some systems support 8KiB (32TiB) blocks.

Personally I prefer ReiserFS. Dynamic i-node allocation, on-the-fly defragging, tail-packing (gets rid of the wasted space at the end of blocks when files aren't in even block-sized increments), and online expansion of volumes through the use of LVM.

Info: http://en.wikipedia.org/wiki/ReiserFS
Specs: http://namesys.com/faq.html#reiserfsspecs

Nope - 8TB is all the ext3 version we have (and all others I know about) can handle at maximum block size. Not a big deal - 8TB is fine. And I'm not sure I can bring myself to use ReiserFS given the author's current criminal status (guilty or otherwise).

- Matt

As for the ext3 8TiB limit, ext3 will only go to 8TiB limit if you set it up for 2KiB block size. 4KiB block size will take you to 16TiB, and only some systems support 8KiB (32TiB) blocks.

Personally I prefer ReiserFS. Dynamic i-node allocation, on-the-fly defragging, tail-packing (gets rid of the wasted space at the end of blocks when files aren't in even block-sized increments), and online expansion of volumes through the use of LVM.

Info: http://en.wikipedia.org/wiki/ReiserFS
Specs: http://namesys.com/faq.html#reiserfsspecs

---

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Nope - 8TB is all the ext3 version we have (and all others I know about) can handle at maximum block size. Not a big deal - 8TB is fine. And I'm not sure I can bring myself to use ReiserFS given the author's current criminal status (guilty or otherwise).

I don't know if it's good to judge the software by it's author...

However, what is more important is that ReiserFS - with the given 8TB - will finish a fsck in 10 min and ext3 will take 4-5 hrs.
Long outage if it goes down hard...

---

mic.