Increasing Numbers (Jul 01 2009)


log in

Advanced search

Message boards : Technical News : Increasing Numbers (Jul 01 2009)

Previous · 1 · 2 · 3
Author Message
DJStarfox
Send message
Joined: 23 May 01
Posts: 1040
Credit: 540,292
RAC: 561
United States
Message 918457 - Posted: 16 Jul 2009, 14:41:28 UTC - in response to Message 918304.

Every five seconds (by default; SETI may be changing that delay), the feeder checks if there is any free space in the queue and fills it with data from the database. And while the feeder is busy doing that, querying the database, the schedulers can still get items from the queue.

But when things are really busy, the 100 tasks can disappear in a fraction of a second.

That's definitely true. But the original analogy is inaccurate either way :)


Seems to me, from a computer scientist's point of view, that this system does not scale up well. It would make more sense to have two buffers (queues) in the feeder. One thread reads from buffer #1 and gives tasks to clients requesting work. The other thread is busy filling buffer #2. When buffer #1 is empty, both threads swap buffers. Hopefully, someone smarter than me knows how to program this most efficiently, but this approach would avoid the "fill the queue" delay that the feeder currently has. Recall that Matt mentioned that the "fill the queue" delay is quite significant, given that the database is always hammered.

Perhaps I should bring this up with the BOINC development team, if there is an agreement from someone on the SETI staff that things this idea is worth pursuing.

Josef W. Segur
Volunteer developer
Volunteer tester
Send message
Joined: 30 Oct 99
Posts: 4203
Credit: 1,030,619
RAC: 266
United States
Message 918509 - Posted: 16 Jul 2009, 19:03:52 UTC - in response to Message 918291.

Nicolas wrote:
Every five seconds (by default; SETI may be changing that delay), the feeder checks if there is any free space in the queue and fills it with data from the database. And while the feeder is busy doing that, querying the database, the schedulers can still get items from the queue.

Interesting point, in Matt's post last December on Feeder issues, he said "two seconds".

The default 5 seconds would be somewhat too long when only S@H Enhanced work is available, steady state would be near 60 MBits/sec of download. But when there are also 3 Astropulse tasks steady state would be about 99 MBits/sec and that's obviously too much.

As a command line argument is needed to deviate from the default, that feature isn't really helpful for handling short term variations.
Joe

Previous · 1 · 2 · 3

Message boards : Technical News : Increasing Numbers (Jul 01 2009)

Copyright © 2014 University of California