"BOINC portable" for Windows hosts

Message boards : Number crunching : "BOINC portable" for Windows hosts
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1891758 - Posted: 25 Sep 2017, 12:54:49 UTC
Last modified: 25 Sep 2017, 12:55:02 UTC

Looks like value in script for gui_rpc_auth.cfg password does not match. The 2nd line set pass=password needs to be the same.
My folder structure is like this:
C:\BOINC\
	boinc.exe
	boinccmd.exe
	boincupdate.bat
C:\BOINC\Data1\
	cc_config.xml
	client_state.xml
	gui_rpc_auth.cfg
C:\BOINC\Data2\
	cc_config.xml
	client_state.xml
	gui_rpc_auth.cfg
C:\BOINC\Data3\
	cc_config.xml
	client_state.xml
	gui_rpc_auth.cfg


Looks like I didn't include my start up script. It launches BOINC for each data directory. Along the lines of
boinc ----gui_rpc_port 31416 --dir Data1  --detach
boinc ----gui_rpc_port 31417 --dir Data2  --detach
boinc ----gui_rpc_port 31418 --dir Data3  --detach

I can copy/paste it once I return home from work in a few hours.

Each data folder has a cc_config.xml with at least these values set.
<allow_multiple_clients>1</allow_multiple_clients>
<allow_remote_gui_rpc>1</allow_remote_gui_rpc>
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1891758 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1891832 - Posted: 25 Sep 2017, 22:57:50 UTC
Last modified: 25 Sep 2017, 22:58:58 UTC

BM Start.bat
@ECHO OFF & cls & pushd %~dp0
set desc1=Starting BOINC Client
set doffset1=0
set doffset2=10
set prt=31416
set /a prt=%prt%+%doffset1%
:loopy
set /a nn=%nn%+1
@ECHO %desc1% %nn% %prt%
boinc.exe --gui_rpc_port %prt% --dir %~dp0Data%nn%\ --detach
set /a prt=%prt%+1
if %nn% LSS %doffset2% goto loopy


BM Stop.bat
@ECHO OFF & cls & pushd %~dp0
set desc1=Stopping BOINC Client
set pass=password
set doffset1=0
set doffset2=10
set prt=31416
set /a prt=%prt%+%doffset1%
:loopy
set /a nn=%nn%+1
@ECHO %desc1% %nn% %prt%
boinccmd --host localhost:%prt% --passwd %pass% --quit
timeout 1
set /a prt=%prt%+1
if %nn% LSS %doffset2% goto loopy
timeout 20
taskkill /im boinc.exe /t /f
taskkill /im boincmgr.exe /t /f
del job_log*.* /s /q
del *.old /s /q

SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1891832 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1891884 - Posted: 26 Sep 2017, 5:51:07 UTC - in response to Message 1891832.  

Thanks!
As I see the single command for BOINC is to update project. It seems you don't use any of "exit after/before" switches.
Are they ineffective? There is nothing to prevent overcommitting of internet host CPU: while it handles upload/download of new tasks it will do CPU/GPU computations for all instances. Maybe worth to include suspend computations command somewhere?
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1891884 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1891886 - Posted: 26 Sep 2017, 6:13:09 UTC - in response to Message 1891832.  

And another question.
What will be if some of data directories are missing?
As I understand data directorycan be either on internet host or on one of processing hosts. If data gathering from processing host includesnot all hosts eachtime some ofdata dirson internet host can be omitted.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1891886 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1891915 - Posted: 26 Sep 2017, 23:29:24 UTC

Over commit of internet host resources can be an issue. I don't recall exactly what I used in the past.
If I was setting up offline hosts today I would use a shutdown and start up script. Which would suspend processing before shutting down and resume after starting.

startup.bat
boinc.exe --skip_cpu_benchmarks --detach
timeout 5
boinccmd --set_run_mode auto

shutdown.bat
boinccmd --set_run_mode never
timeout 5
boinccmd --quit

If you wan to handle the suspending resume of BOINC on the internet host then it could be done easily.

BM Start.bat
@ECHO OFF & cls & pushd %~dp0
set desc1=Starting BOINC Client
set pass=password
set doffset1=0
set doffset2=10
set prt=31416
set /a prt=%prt%+%doffset1%
:loopy
set /a nn=%nn%+1
@ECHO %desc1% %nn% %prt%
boinc.exe --gui_rpc_port %prt% --dir %~dp0Data%nn%\ --detach
timeout 10
boinccmd --host localhost:%prt% --passwd %pass% --set_run_mode never
set /a prt=%prt%+1
if %nn% LSS %doffset2% goto loopy

BM Stop.bat
@ECHO OFF & cls & pushd %~dp0
set desc1=Stopping BOINC Client
set pass=password
set doffset1=0
set doffset2=10
set prt=31416
set /a prt=%prt%+%doffset1%
:loopy
set /a nn=%nn%+1
@ECHO %desc1% %nn% %prt%
boinccmd --host localhost:%prt% --passwd %pass% --set_run_mode auto
timeout 5

boinccmd --host localhost:%prt% --passwd %pass% --quit
timeout 1
set /a prt=%prt%+1
if %nn% LSS %doffset2% goto loopy
timeout 20
taskkill /im boinc.exe /t /f
taskkill /im boincmgr.exe /t /f
del job_log*.* /s /q
del *.old /s /q
The timeout value needed will depend on host. I normally start with a high value like 30 seconds and make smaller until I find the lowest that works.

Another option could be to use --start_delay N on starting. A value longer than the time needed to finish upload, reporting, fetch new tasks, and close BOINC could be used.
boinc.exe --gui_rpc_port %prt% --dir %~dp0Data%nn%\ --detach --start_delay 7200
I don't know how large the delay can be. I have only used to 600 seconds for a 10 min delay.

To avoid starting a BOINC client when there is no data directory I would probably use if exist when starting and look for an important file like client_state.xml.
if exist %~dp0Data%nn%\client_state.xml boinc.exe --gui_rpc_port %prt% --dir %~dp0Data%nn%\ --detach
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1891915 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1892028 - Posted: 27 Sep 2017, 19:20:58 UTC - in response to Message 1891915.  

Thanks a lot!
Startup/shutdown scripts are exactly that direction I moving today.
But do I understand right that if I add --no_gui_rpc I will break boinccmd operation too?
So "GUI" is deceiving term here?

Also I have big issue - on each host boinc.exe takes full CPU :/ Perhaps I cuttoo much of boinc installation set and it misses some required part...
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1892028 · Report as offensive
Profile Darrell
Volunteer tester
Avatar

Send message
Joined: 14 Mar 03
Posts: 267
Credit: 1,418,681
RAC: 0
United States
Message 1892038 - Posted: 27 Sep 2017, 20:38:46 UTC - in response to Message 1892028.  



Also, I have a big issue - on each host boinc.exe takes full CPU :/ Perhaps I cut too much of boinc installation set and it misses some required part...


If your boinc.exe is one built by Berkeley and not built locally, it is quite possible that it is looking for some response to a query of the registry that it is not getting an answer to. I could see this easily getting it stuck in a loop someplace. So not having a locally built client would make it very difficult to attach a debugger to a running process to see where it is getting stuck. Just a thought :)
... and still I fear, and still I dare not laugh at the Mad Man!

Queen - The Prophet's Song
ID: 1892038 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1892079 - Posted: 27 Sep 2017, 23:14:47 UTC - in response to Message 1892028.  

I just tested using --no_gui_rpc on one of my normal hosts.
boinc.exe --skip_cpu_benchmarks --no_gui_rpc --detach
start boincmgr.exe /s

As expected BOINC Manager didn't connect to the BOINC client. I would guess they used "gui" in the switch name because of the BOINC Manager GUI?
Using boinccmd was also not successful.
	boinccmd --get_tasks
	can't connect to local host

Additionally boinc.exe was using a full CPU. Which it does not normally do.
I saw nothing in the stdoutdae.txt that looked odd. Other than no mention of the no_gui_rpc switch being used. Other logs like stderrgui.txt, stderrdae.txt, & stdoutgui.txt were empty.
Perhaps some kind of bug? I also tried boinc.exe --no_gui_rpc with the same results.

Similar to you I cut out some of the BOINC files for my hosts. I am sure there are a few more files I could get rid of but here are the ones I keep.
Z:\BOINC versions\Boinc 7.6.x\7.6.33\x64\
boinc.exe
boinccmd.exe
boincsvcctrl.exe
ca-bundle.crt
libcurl.dll
libeay32.dll
LiberationSans-Regular.ttf
msvcp100.dll
msvcr100.dll
sqlite3.dll
ssleay32.dll
zlib1.dll
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1892079 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1892127 - Posted: 28 Sep 2017, 8:24:14 UTC - in response to Message 1892079.  

Oh, thanks for such detailed confirmation of my observations! Very good to know it's repeatable issue. So I will report it to BOINC devs then.

Just one clarification - did you see 100% CPU usage on host with reduced files set or on absolutely normal BOINC installation?
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1892127 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1892132 - Posted: 28 Sep 2017, 9:29:55 UTC

Looking at the code (did anyone do that?), the action of --no_gui_rpc is to disable the creation of the network socket - boinc will not listen for any commands from the user environment while this switch is in operation. It will be literally out of control.
ID: 1892132 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1892157 - Posted: 28 Sep 2017, 13:23:54 UTC - in response to Message 1892132.  

Looking at the code (did anyone do that?), the action of --no_gui_rpc is to disable the creation of the network socket - boinc will not listen for any commands from the user environment while this switch is in operation. It will be literally out of control.

It'sOK, but why being out of control means to consume full CPU ?... Still looks like bug.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1892157 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1892159 - Posted: 28 Sep 2017, 13:30:17 UTC - in response to Message 1892157.  

Looking at the code (did anyone do that?), the action of --no_gui_rpc is to disable the creation of the network socket - boinc will not listen for any commands from the user environment while this switch is in operation. It will be literally out of control.
It'sOK, but why being out of control means to consume full CPU ?... Still looks like bug.
I would agree, but just checking that you would still want to run it with no control possible: to me, that feels like a test mode that would only be used for debugging, and hence it wouldn't matter if it went into a spin loop or something - no real science would be active.
ID: 1892159 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1892169 - Posted: 28 Sep 2017, 14:06:50 UTC - in response to Message 1892159.  

Well, actually I'm not sure if I want to use that option ot not.
The reason why I started to add more options is that on one of host Windows Firewall poped up with question if I want allow communications for BOINC or not. As result 2 rules in firewall rules list were added. I would like to avoid any "pollution" of offline hosts as much as possible so definitely don't like such interaction with firewall.
So far I saw this only on one PC from ~10 (they almost identical ), but I added few more options regarding net suppressing:
--no_info_fetch
--suppress_net_info

Maybe those switches helped, maybe that first host has little different firewall setup... But after deletion of those rules _and_ removal of --no_gui_rpc switch no new interference from Firewall and no 100% CPU usage. So, I'm happy for now (the only think to do is to reduce amount of manual work and test interaction with boinccmd command).
Would be good if --no_gui_rpc option would be fixed too just for completeness.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1892169 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1892170 - Posted: 28 Sep 2017, 14:11:44 UTC - in response to Message 1892127.  

Oh, thanks for such detailed confirmation of my observations! Very good to know it's repeatable issue. So I will report it to BOINC devs then.

Just one clarification - did you see 100% CPU usage on host with reduced files set or on absolutely normal BOINC installation?

Reduced file set. I have not used a standard installation of BOINC since about 2010.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1892170 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1892173 - Posted: 28 Sep 2017, 14:17:26 UTC - in response to Message 1892170.  

so, very small possibility exists that bug will not happen on usual install - i'll check that soon.

But on reduced set --no_gui_rpc removal cured boinc indeed.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1892173 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1892175 - Posted: 28 Sep 2017, 14:22:22 UTC
Last modified: 28 Sep 2017, 14:25:15 UTC

And now my "portable boinc" experiment has new bottleneck - time to put data from internet host to flash drive.
Estimated as ~16 minutes... Will be inconvenient on regular usage.
But cause I move whole data directory all executables/binary caches are moven back and forth too - quite considerable share of whole data transfer actually.
Here some optimisation should be possible....
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1892175 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1892180 - Posted: 28 Sep 2017, 14:42:26 UTC - in response to Message 1892175.  

And now my "portable boinc" experiment has new bottleneck - time to put data from internet host to flash drive.
Estimated as ~16 minutes... Will be inconvenient on regular usage.
But cause I move whole data directory all executables/binary caches are moven back and forth too - quite considerable share of whole data transfer actually.
Here some optimisation should be possible....

I had used something like export.bat and import.bat on mine. Which only copied over project data and BOINC .xml files.
I don't recall exactly what I used but perhaps xcopy /exclude *.exe or something like that
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1892180 · Report as offensive
Juha
Volunteer tester

Send message
Joined: 7 Mar 04
Posts: 388
Credit: 1,857,738
RAC: 0
Finland
Message 1892195 - Posted: 28 Sep 2017, 15:47:29 UTC

Technical nitpickery. Even with --no_gui_rpc the client still has some limited control options.

The client responds to Ctrl-C/Ctrl-Break and WM_QUERYENDSESSION/WM_ENDSESSION to the hidden window by quitting.
ID: 1892195 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1894162 - Posted: 8 Oct 2017, 20:51:25 UTC
Last modified: 8 Oct 2017, 20:53:48 UTC

I hoped to avoid boinccmd usage (this would be much simplier - no need to specify/use particular ports) . Issue with overcommitting of internet host was solved by --start_delay 60000 switch. Butafter few iterations I found that some instances not reporting results even being online few hours. Strange. I launch ~10 instances. After few hours online some reported all results and downloaded new tasks but some still have many result files in project directory.

Seems forced project update is required and it in turn requires boinccmd usage to order boinc do communication with project:/

P.S. NowI do upload/download directly from flash that eliminates one copy. Updating offline hosts still time consuming though.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1894162 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1894165 - Posted: 8 Oct 2017, 20:56:30 UTC - in response to Message 1894162.  

err - cc_config has an option for <report_results_immediately>. Why would that not do?
ID: 1894165 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · Next

Message boards : Number crunching : "BOINC portable" for Windows hosts


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.