Man, I wish there was a BOINC-Queue already

Message boards : Number crunching : Man, I wish there was a BOINC-Queue already
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile FalconFly
Avatar

Send message
Joined: 5 Oct 99
Posts: 394
Credit: 18,053,892
RAC: 0
Germany
Message 23471 - Posted: 8 Sep 2004, 0:06:22 UTC
Last modified: 8 Sep 2004, 0:10:24 UTC

Ever since I partly switched one machine from SETI Classic to BOINC, I really like it :)

From running BOINC only at night and SETI Classic at day, I've now moved to running BOINC almost all day on the machine.

But with one machine having "not quite" the power to effectively push along 3 interesting Projects in parallel, I sure would like to put some of my other equipment to BOINC use.
(we're talking 27 other CPUs currently running SETI Classic along, Proxy is not an Option in those numbers with DSL on Dial-Up policy due to Network integrity/security)

Any word (even on low Priority, long-range planning), or even at least an acknowledgement on a feature similar addition like SETIqueue to BOINC ?

Toss in a Client for 64bit DEC Alpha and SGI MIPS platforms, and I'll be a thoroughly happy camper ;)
ID: 23471 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 23520 - Posted: 8 Sep 2004, 2:31:25 UTC

What feature in particular were you missing in SETI queue?

If it was the redistribution of work to several computers anonymously, that is strictly out. The servers currently remember the machine that the work was sent to, and if the same machine does not return the result, it will be marked as an attempted cheat and dropped. Any BOINC queue will have to collect work for a specific machine. You can thank the cheaters in S@H classic for this.

If it is the ability to have one machine be the only connection to the internet, that is indeed missing. However, you may be able to use a proxy instead.
ID: 23520 · Report as offensive
Profile FalconFly
Avatar

Send message
Joined: 5 Oct 99
Posts: 394
Credit: 18,053,892
RAC: 0
Germany
Message 23553 - Posted: 8 Sep 2004, 3:50:45 UTC - in response to Message 23520.  

> If it is the ability to have one machine be the only connection to the
> internet, that is indeed missing. However, you may be able to use a proxy
> instead.

"...Proxy is not an Option..." :/

ID: 23553 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 23554 - Posted: 8 Sep 2004, 3:57:57 UTC - in response to Message 23471.  

If you log into setiweb.ssl.berkeley.edu and go to "your account" you can set your preferences for "minimum" and "maximum" work. I've got my fast machine set to 1 day minimum, 4 days maximum.

> Any word (even on low Priority, long-range planning), or even at least an
> acknowledgement on a feature similar addition like SETIqueue to BOINC ?

ID: 23554 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 23733 - Posted: 8 Sep 2004, 15:35:11 UTC - in response to Message 23553.  

>
> "...Proxy is not an Option..." :/
>
>

Since proxy isn't an option, I'm having problems understanding why you want something similar as SetiQueue since "SetiQueue is a proxy server"... ;)
ID: 23733 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20283
Credit: 7,508,002
RAC: 20
United Kingdom
Message 23868 - Posted: 8 Sep 2004, 21:36:45 UTC - in response to Message 23471.  

[...]
> Toss in a Client for 64bit DEC Alpha and SGI MIPS platforms, and I'll be a
> thoroughly happy camper ;)

The source code is there. Time to dust off your respective compilers!

Good luck,
Martin

ID: 23868 · Report as offensive
Profile Sir Ulli
Volunteer tester
Avatar

Send message
Joined: 21 Oct 99
Posts: 2246
Credit: 6,136,250
RAC: 0
Germany
Message 23870 - Posted: 8 Sep 2004, 21:39:46 UTC - in response to Message 23868.  

> [...]
> > Toss in a Client for 64bit DEC Alpha and SGI MIPS platforms, and I'll be
> a
> > thoroughly happy camper ;)
>
> The source code is there. Time to dust off your respective compilers!
>
> Good luck,
> Martin
>
>
>

if this were so easy, i had an AMD64 client running, since Days / Weeks ..

S@h Berkeley's Staff Friends Club ©

Greetings from Germany NRW
Ulli [/url]<a>
ID: 23870 · Report as offensive
Profile FalconFly
Avatar

Send message
Joined: 5 Oct 99
Posts: 394
Credit: 18,053,892
RAC: 0
Germany
Message 24008 - Posted: 9 Sep 2004, 6:26:22 UTC - in response to Message 23870.  
Last modified: 9 Sep 2004, 6:27:58 UTC

> See this thread:
> http://setiweb.ssl.berkeley.edu/forum_thread.php?id=3774

Unfortunately, Cacheing wont work if the machine has no own Internet connection privileges. I initially thought it might work, but since even CPDN is supposed to Tickle intermediate Results, wouldn't be of any use or help.
ID: 24008 · Report as offensive
Profile FalconFly
Avatar

Send message
Joined: 5 Oct 99
Posts: 394
Credit: 18,053,892
RAC: 0
Germany
Message 24009 - Posted: 9 Sep 2004, 6:29:17 UTC - in response to Message 24008.  
Last modified: 9 Sep 2004, 6:48:54 UTC

> Since proxy isn't an option, I'm having problems understanding why you want
> something similar as SetiQueue since "SetiQueue is a proxy server"... ;)

Common error!

SETIqueue is NOT a Proxy, unless specifically configured to perform Pass-Through operations. (which is disabled by default and for a good reason).

Try having ANY Program connect to the Internet via SETIqueue. NoGO, it naturally can't. Will only allow SETI Clients to connect, only connect to one address, only for one purpose.
Try doing it with a Proxy : works great, Network can physically be compromised, game over worst case.

Now think your Server, Proxy and Network behind it is connected very frequently and !autonomously! , without any chance of User interaction nor intervention, over years. All within an ever-changing Threat scenario, and with a number of confirmed people on the net that you know would do !anything! to cause you any damage they can.
Don't know about you, but I certainly don't have a warm fuzzy feeling putting an entire Network at risk all day, that took me 5 years of building.
I'd hate to lose it to some fresh Exploit, Script Kiddie or alike. (and I know people in the past experienced exactly that)

Since it's such a common error :
Try having a Client connect to SETIqueue, while the System is offline (no Internet connection).
Works great, clients gets assigned its work without any Internet connection, thus being 100% secure, all communication limited to the secured Intranet.

Try it to a Proxy... It will naturally fail, as it simply works by forwarding Data from the Client directly to the Internet.

One small difference, resulting in entirely different dimensions of Network Security issues. And all happening while you're asleep, or not even present at all, silently on 5, 10, 25 or even 1000 machines.
No sane, self-respecting Network Admin would allow that unless forced to do so, thus normally resulting in Clients not having Internet Connection privileges at all.
And that's why BOINC currently is held in place mainly by small, single to two System Users only. Most of the much more powerful Networks cannot (and will not ever be able to) participate due to IT Security issues involved, unless a BOINC-Queue is implemented to remove those issues.
ID: 24009 · Report as offensive
Profile JigPu
Avatar

Send message
Joined: 16 Feb 00
Posts: 99
Credit: 2,513,738
RAC: 0
Message 24028 - Posted: 9 Sep 2004, 9:44:58 UTC - in response to Message 24009.  

Now, I will admit that networks most certianly AREN'T my thing, but here goes anyway =)

It is my understanding that proxy server is a server which talks "on behalf of" another server further upsteam. It does this by storing (some of) the upsteam server's data locally, so that future requests need not leave the intranet. It can have the ability to forward those requests upstream to the server (probably for the latest data), but by no means is it obligated to.

Such a proxy would limit internet access, as only items cached could be delivered. A proxy set up without pass-through would be difficult, if not nearly impossible, to work with since any data that is going to be accessed would have to be specifically cached beforehand. It is for that reason that a great majority of all HTTP proxies include the ability to pass-though.

SetiQueue does not need pass-though though. It knows exactly what kind of data to proxy (WUs), and is designed soley for that use. It will request WUs from Berkeley beforehand and pass them out on behalf of Berkeley when it recieves a request. The only reason it does not need pass-though is because SETI@Home does not care what WU it gets, it just requests one and will take anything it recieves. Thus, SetiQueue can cache a ton of WUs, and act as a proxy when a client needs more.



Of course, none of that helps solve your "no proxy" problem. If you can't/won't use of some kind of HTTP proxy (or find a way to get those other boxes net access), you're not going to be able to get them crunching with BOINC. BOINC sends each client very specific data, so they cannot be proxied beforehand with a client that does not have pass-though.
_______________________________
Proud Member of the Overclockers.com SETI Team!

<img>
ID: 24028 · Report as offensive
Profile wilsen
Volunteer developer

Send message
Joined: 15 Apr 00
Posts: 68
Credit: 73,527
RAC: 0
Germany
Message 24033 - Posted: 9 Sep 2004, 10:28:59 UTC - in response to Message 23471.  

> Any word (even on low Priority, long-range planning), or even at least an
> acknowledgement on a feature similar addition like SETIqueue to BOINC ?
>
Just look at http://boinc.berkeley.edu/proxy_server.php
Find someone who can implement that or something similar and be happy ;)
---

KEINE PANIK
ID: 24033 · Report as offensive
Profile mikey
Volunteer tester
Avatar

Send message
Joined: 17 Dec 99
Posts: 4215
Credit: 3,474,603
RAC: 0
United States
Message 24035 - Posted: 9 Sep 2004, 10:32:56 UTC - in response to Message 24028.  

> Of course, none of that helps solve your "no proxy" problem. If you
> can't/won't use of some kind of HTTP proxy (or find a way to get those other
> boxes net access), you're not going to be able to get them crunching with
> BOINC. BOINC sends each client very specific data, so they cannot be proxied
> beforehand with a client that does not have pass-though.
> _______________________________
BUT if some programmer could figure out EXACTLY what the Berkeley Boinc server was looking for when it accessed out individual machines, it could then capture that same data and with a little ingenuity pass that onto Berkeley thru a program. That program can be called Boinc Queue, for now, and what it would have to do is to capture and retain that data, put the units that go with THAT data into a specific directory, and then let ONLY that machine access that directory.
I have a small farm, if Boinc Queue knew what Berkeley was looking for when each of my machines connected to both get and return units, why couldn't it then cache units based on that data? Of course each machine would have to have access to its OWN directory and no other machine could use it!

ID: 24035 · Report as offensive
Profile FalconFly
Avatar

Send message
Joined: 5 Oct 99
Posts: 394
Credit: 18,053,892
RAC: 0
Germany
Message 24042 - Posted: 9 Sep 2004, 10:57:08 UTC - in response to Message 24028.  

> BOINC sends each client very specific data, so they cannot be proxied
> beforehand with a client that does not have pass-though.

I'm sure BOINC would be just as happy got get all this Data collected by a Local BOINC Queue as well.

Everything the BOINC Client now collects and reports to directly to the Server could easily be stored and communicated by a Queue, it would make not much difference, it's just one step more in storing/forwarding the Data.
ID: 24042 · Report as offensive
Profile FalconFly
Avatar

Send message
Joined: 5 Oct 99
Posts: 394
Credit: 18,053,892
RAC: 0
Germany
Message 24044 - Posted: 9 Sep 2004, 10:59:59 UTC - in response to Message 24042.  
Last modified: 9 Sep 2004, 11:26:07 UTC

> http://boinc.berkeley.edu/proxy_server.php

Yes, I've seen that before.
Unfortunately, it just shows they did not realize any of the issues/impossibilities it imposes on bigger Networks.

I'd rather see it as their "Quick & Dirty" way of getting tiny home Networks with no Security concerns involved into the loop. They just didn't (or still don't) understand what SETIqueue actually meant for SETI, and why it is critically needed if they're looking for the serious Performance they still have there due to it.
ID: 24044 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 24056 - Posted: 9 Sep 2004, 11:34:00 UTC - in response to Message 24009.  

> > Since proxy isn't an option, I'm having problems understanding why you
> want
> > something similar as SetiQueue since "SetiQueue is a proxy server"... ;)
>
> Common error!
>
> SETIqueue is NOT a Proxy, unless specifically configured to perform
> Pass-Through operations. (which is disabled by default and for a good
> reason).
>

Looks like you don't know that "proxy" means. ;)
Proxy is "to act as a substitute for another", and since setiqueue very much is a substitute to upload/download results/wu on behalf of the seti@home-clients, it's a proxy-server.

Granted it's a limited proxy-server designed to only accept connections from seti@home-clients, but this does not stop it being a proxy. :)
ID: 24056 · Report as offensive
Profile Chip Long
Avatar

Send message
Joined: 2 Aug 00
Posts: 445
Credit: 503,693
RAC: 0
United States
Message 24063 - Posted: 9 Sep 2004, 12:06:59 UTC - in response to Message 24056.  
Last modified: 10 Sep 2004, 23:30:42 UTC

Internet --- Bastion Firewall --- Proxy Server ----
--------------------------------------------------|
|
--Client
--Client
--Client
--Client

Bastion Firewall Ruleset ONLY allows traffic between Berkeley and Proxy...

http://bastionfirewall.sourceforge.net/

[EDIT] You really don't even need the PROXY as the clients will only talk to the 'Projects' and the Firewall Ruleset will only allow traffic from the 'Projects' onto the local subnet, if done correctly...

Shouldn't be too big a problem for you SysAdmins to get a handle on...

Regards

chip
w3range.net

ID: 24063 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 24082 - Posted: 9 Sep 2004, 13:52:23 UTC

Since the source code for both S@H and BOINC is publicly available, it should be possible to write a BOINC proxy that would would be able to transfer information between BOINC and projects and do nothing else. Since the individual machines already have the queues, it might be easier to have the BOINC proxy just do the limited forwarding of requests and replies. This would be a substantially thinner proxy than BOINC queue.
ID: 24082 · Report as offensive
Profile FalconFly
Avatar

Send message
Joined: 5 Oct 99
Posts: 394
Credit: 18,053,892
RAC: 0
Germany
Message 24130 - Posted: 9 Sep 2004, 15:57:38 UTC - in response to Message 24063.  
Last modified: 9 Sep 2004, 16:00:58 UTC

> http://bastionfirewall.sourceforge.net/

That's an interesting approach, but still, the machine would be offline for most of the day, and completely offline on days where I'm not present.

After all, any Software can fail (due to a System Problem, for example), thus the machine will (regardless of Software installed) never go online autonomously.
------------
Yep, I think the same, especially since the entire Communication of BOINC and its Project is availble to public and described in detail.
SETIqueue didn't have any of that, and it still worked awesome.

Maybe given some time, someone with the right abilities and the foresight to see what BOINC really needs, can finally realize it. It's basically the absolute breakthrough in computing performance and scalabiliby by connecting entire Sub-Grids, that (so far) only SETI 1 had, making it the attractive success it is.
ID: 24130 · Report as offensive
Profile Sir Ulli
Volunteer tester
Avatar

Send message
Joined: 21 Oct 99
Posts: 2246
Credit: 6,136,250
RAC: 0
Germany
Message 24214 - Posted: 9 Sep 2004, 20:57:20 UTC

yes Setiqueue was a fine think, but... you have to remember, that Setiqueue was sending WUs, and sending and sending...

at SAH classic there was no mechanismen to controll what WU came from what host...
but at BOINC every WU is bound to his Host, so i think it is not so easy to build a queue for this.

S@h Berkeley's Staff Friends Club ©

Greetings from Germany NRW
Ulli [/url]


ID: 24214 · Report as offensive
Profile Sir Ulli
Volunteer tester
Avatar

Send message
Joined: 21 Oct 99
Posts: 2246
Credit: 6,136,250
RAC: 0
Germany
Message 25249 - Posted: 11 Sep 2004, 20:19:58 UTC

Why won't SETIQueue work with BOINC?

Unlike SETI@home Classic, with its 'one size fits all' work units, BOINC allows work units that have extreme requirements (memory, disk, CPU) and makes sure they're sent only to hosts that can handle them. In BOINC, a client communicates directly with the server, telling the server about its hardware (memory size, CPU speed etc.) and the server chooses work for it accordingly. Furthermore, BOINC has separate scheduling and data servers (in SETI@home Classic, a single server played both roles).

so look here

http://boinc.ssl.berkeley.edu/proxy_server.php

Greetings from Germany NRW
Ulli [/url]

S@h Berkeley's Staff Friends Club m7 ©
ID: 25249 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : Man, I wish there was a BOINC-Queue already


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.