GUPPI Rescheduler for Linux and Windows - Move GUPPI work to CPU and non-GUPPI to GPU

Message boards : Number crunching : GUPPI Rescheduler for Linux and Windows - Move GUPPI work to CPU and non-GUPPI to GPU
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 29 · 30 · 31 · 32 · 33 · 34 · 35 . . . 37 · Next

AuthorMessage
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1859495 - Posted: 5 Apr 2017, 2:08:01 UTC - in response to Message 1859490.  



. . If you feel that this thread is unsuitable for this discussion can I pm you about this?

Stephen

??

If you do end up in another thread on this, I'd love to follow along...
ID: 1859495 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1859512 - Posted: 5 Apr 2017, 4:12:41 UTC - in response to Message 1859495.  



. . If you feel that this thread is unsuitable for this discussion can I pm you about this?

Stephen

??

If you do end up in another thread on this, I'd love to follow along...


. . If I make any progress I will bear that in mind :)

. . At the moment I am working with Stubbles' script because it is shorter and less complicated. It is mostly echo commands and even that has taken me ages to get to run through without errors. Everything else bombs every time. The linux workaround for the pause command ... bombs, even the sleep command that works from the command line in terminal won't work in the script. I have lots of reading to do it seems. The alternative command to 'TYPE' which is 'cat' works AOK from the command line in terminal too, but fails to find the target file when run from the script. Once I have worked out enough of the basic translations to get this script working (am I aiming too high?) I will see if I still have the motivation to tackle your script. I think that one will be a LOT of work.

Stephen

<shrug>
ID: 1859512 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1853
Credit: 268,616,081
RAC: 1,349
United States
Message 1859533 - Posted: 5 Apr 2017, 6:06:27 UTC - in response to Message 1859512.  
Last modified: 5 Apr 2017, 6:11:49 UTC



. . If you feel that this thread is unsuitable for this discussion can I pm you about this?

Stephen

??

If you do end up in another thread on this, I'd love to follow along...


. . If I make any progress I will bear that in mind :)

. . At the moment I am working with Stubbles' script because it is shorter and less complicated. It is mostly echo commands and even that has taken me ages to get to run through without errors. Everything else bombs every time. The linux workaround for the pause command ... bombs, even the sleep command that works from the command line in terminal won't work in the script. I have lots of reading to do it seems. The alternative command to 'TYPE' which is 'cat' works AOK from the command line in terminal too, but fails to find the target file when run from the script. Once I have worked out enough of the basic translations to get this script working (am I aiming too high?) I will see if I still have the motivation to tackle your script. I think that one will be a LOT of work.

Stephen

<shrug>

Fair enough. Just remember there was a bunch of stuff in that early version that either didn't work or worked improperly, hence all the differences.
ID: 1859533 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1859766 - Posted: 6 Apr 2017, 7:30:47 UTC - in response to Message 1859533.  


If you do end up in another thread on this, I'd love to follow along...

. . If I make any progress I will bear that in mind :)
<shrug>

Fair enough. Just remember there was a bunch of stuff in that early version that either didn't work or worked improperly, hence all the differences.


. . I don't suppose you would know why commands that work AOK from the command line in a terminal window ALL fail when launched from a shell script? They all either return command not found or reject the parameter switches or "cannot find file" when it is right there in the same folder. Even a simple built in command like exit bombs in the script. Linux totally befuddles me. And if I search on a question about this I get a lot of "stuff" that has nothing to do with my problem. I even tried to take advantage of an offer of a free handbook for dummies, err I mean beginners like me, to help write shell scripts and use the BASH interpreter, but you had to give them your email and I still have not seen the email with the link to the handbook.

. . Talk about bashing my head on a brick wall :(

Stephen

:(
ID: 1859766 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1859823 - Posted: 6 Apr 2017, 15:24:17 UTC - in response to Message 1859766.  

You either don't have permission to run it, or you are not defining the path if it can't find something.

Try running a single command from within a script to find out what you are missing.
ID: 1859823 · Report as offensive
Sleepy
Volunteer tester
Avatar

Send message
Joined: 21 May 99
Posts: 219
Credit: 98,947,784
RAC: 28,360
Italy
Message 1859857 - Posted: 6 Apr 2017, 18:44:39 UTC

Is there a way to graciously quit the Boinc Manager and the clients from a command line (Windows)?

I have been trying with boinccmd --quit, but it blanks the Manager without quitting it and leaves the clients running. Invoking boincmgr again from the command line opens a new instance of Boinc.

taskkill also gives no good effect, at least the ways I have tried.

Thank you in advance!
ID: 1859857 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1859879 - Posted: 6 Apr 2017, 20:40:27 UTC - in response to Message 1859823.  

You either don't have permission to run it, or you are not defining the path if it can't find something.

Try running a single command from within a script to find out what you are missing.


. . Nope, I have tried every form of the path from the root directory down as in /home/user/BOINC/level3/filename down to ./filename and all result in file not found. I have "remarked" out every command except the echo commands which are working then tried re-instating them one at a time and they still fail with the same errors. Yet when I run those same commands from the command line in a terminal window they work AOK. As I said, even exit at the end of the script returns an error. I have tried each command on it's own in a window and they all work fine, except for GOTO for which I cannot find a Linux equivalent. :(

. . Sleep 5 in a window sleeps for 5 secs as it should, but in the script it returns an error that the parameter is wrong showing 5\r. So what does 'x'\r mean? I cannot find a reference to it.

. . Such a strange system.

Stephen

??
ID: 1859879 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1859892 - Posted: 6 Apr 2017, 23:04:48 UTC - in response to Message 1859879.  

I just tried a very simple script and sleep seems to work fine, I just added the date command to see the 5 second delay.

brent@Brent-P9X79:~$ cat /home/brent/Desktop/TestSleep.sh
date
sleep 5
date
brent@Brent-P9X79:~$ /home/brent/Desktop/TestSleep.sh
Thu Apr  6 16:58:48 CST 2017
Thu Apr  6 16:58:53 CST 2017
brent@Brent-P9X79:~$
ID: 1859892 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1860047 - Posted: 7 Apr 2017, 13:11:51 UTC - in response to Message 1859892.  
Last modified: 7 Apr 2017, 13:16:58 UTC

I just tried a very simple script and sleep seems to work fine, I just added the date command to see the 5 second delay.

brent@Brent-P9X79:~$ cat /home/brent/Desktop/TestSleep.sh
date
sleep 5
date
brent@Brent-P9X79:~$ /home/brent/Desktop/TestSleep.sh
Thu Apr  6 16:58:48 CST 2017
Thu Apr  6 16:58:53 CST 2017
brent@Brent-P9X79:~$

. . Hey, while that did not directly solve my problem it did put me on the right track. When I created a whole new script file with just the date and sleep commands in it, it worked as it should. This caused me to realise that in copying the Windows .cmd file (by loading it into Gedit and then saving it) something from the windows file that does not show as text must have tagged along and THAT something was screwing with the Bash command interpreter. I opened a fresh empty text file with Sublime Text and copied the text contents from the orginal over to it then saved before making the basic changes that I know so far should work and saving it again. This new version works much better. Sleep {wait} and the pretend pause command now work AOK. I have even found out how to set screen colours (with limited success) so I am getting closer to getting it working. Linux is a tad cantankerous when commands in Dos like '| more' translate to '| less', I never saw that one coming :). But many thanks for the help.

. . But it still has a functional problem, while it does page the output as the original script did, it locks it into a mode of it's own and will not exit, so the script cannot continue. It seems there is always another stumbling block with Linux. Not to mention finding a workaround for LABELs and GOTO statements, 'IF exists" function and other commands.

. . Maybe I will have it working before Christmas? LOL

brent@Brent-P9X79:~$ cat /home/brent/Desktop/TestSleep.sh

. . BTW, what is the purpose of the first line, that is to say, the part that is underlined?

Stephen

:)
ID: 1860047 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1860061 - Posted: 7 Apr 2017, 14:04:56 UTC - in response to Message 1860047.  
Last modified: 7 Apr 2017, 14:41:59 UTC

. . BTW, what is the purpose of the first line, that is to say, the part that is underlined?
Just to show the contents of the script that I was testing.

EDIT: Oh, Same as The DOS prompt with more detail.
brent@Brent-P9X79:~$
brent -- Username
Brent-P9X79 -- Computer Name
~ -- Working directory (home)

BTW, 'pwd' will show your directory too.
ID: 1860061 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1860129 - Posted: 7 Apr 2017, 16:54:50 UTC

As far as I can remember from my time with Linux, you can't directly copy DOS or Windows text files into Linux without stripping the Line Feed/Carriage Return at every line. Linux only uses a Line Feed to indicate EOL. That is what screwed up your direct import. There are Linux text utilities that can strip the DOS EOL and reformat the text into compatible Linux text as a function of the import command.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1860129 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1860177 - Posted: 7 Apr 2017, 20:33:30 UTC - in response to Message 1860061.  

. . BTW, what is the purpose of the first line, that is to say, the part that is underlined?

Just to show the contents of the script that I was testing.

EDIT: Oh, Same as The DOS prompt with more detail.
brent@Brent-P9X79:~$
brent -- Username
Brent-P9X79 -- Computer Name
~ -- Working directory (home)

BTW, 'pwd' will show your directory too.


. . That is useful to know. It could help get this script working better. Does the tilde {~} have to be used as part of the full prompt or can it be employed alone? That is will ~$ return the current directory as a string? Or are ~$ and pwd $ equivalent functions?

Stephen

?
ID: 1860177 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1860180 - Posted: 7 Apr 2017, 20:46:16 UTC - in response to Message 1860129.  

As far as I can remember from my time with Linux, you can't directly copy DOS or Windows text files into Linux without stripping the Line Feed/Carriage Return at every line. Linux only uses a Line Feed to indicate EOL. That is what screwed up your direct import. There are Linux text utilities that can strip the DOS EOL and reformat the text into compatible Linux text as a function of the import command.


. . OK that does explain the problem. But thankfully copying the text itself from the original file to a blank text file and resaving seems to have achieved the same result, i.e. stripped the unwanted carriage return characters. The commands I have "discovered" are now working instead of returning errors. But I now still have to find a way to get the '| less' paging function [I still want to say '| more'] to exit on completion and return to the script to carry on. And if anyone knows how to achieve the same result in a bash shell file as GOTO and LABEL in Dos I could have the problem halfway beat! :)

Stephen

:)
ID: 1860180 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1860884 - Posted: 11 Apr 2017, 5:08:01 UTC

. . Hell's Bells,

. . I am getting close enough to a workable shell file that I have had a look at the client_state.XML file. It is very very different to the windows equivalent. I don't think I can find way to change the entries to move a task from one queue to the other. Too many things are different in the way BOINC treats the file. :( And with MrKevvy's effort not working for me I guess I am screwed :( ... oh well!

Stephen

:(
ID: 1860884 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1861019 - Posted: 12 Apr 2017, 11:50:09 UTC

.
. . @ Kevin,

. . Any chance you could give me a little help. The rescheduler keeps saying there are no non-guppi tasks in my CPU queue to move. Is there something that I might be doing wrong that I need to change to get it to work. There have been between 17 and 45 non-guppi tasks present when it gives that result.

Stephen

??
ID: 1861019 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1867570 - Posted: 16 May 2017, 4:05:54 UTC

I wasn't quite sure where to post this, so I will put it here.

I have been fairly aggressively using cpu2gpu (a variant of GUPPI reschedule) to increase my LINUX GPU cache size for outages, and it has been working well getting a little over 1000 tasks for about 13 hours worth.

But now I have run into an issue since I have pulled my CPU estimated times down to as little as 1.5m/task. So now I get (on 2 machines AVX and ssse3):
197 (0x000000C5) EXIT_TIME_LIMIT_EXCEEDED . . . but no more details in Stderr output.
errors on my CPU tasks at around 1h45m. From what I've found that is from BOINC killing the task because it exceeded 20x the estimated runtime. OK, I understand that.

A work around I found is to search/replace in client_state
<rsc_fpops_bound> . . . with
<rsc_fpops_bound>1
to increase the limit on tasks,

That seems to be working so far for current tasks, but anything new will again error out I'm certain. And I can't complete CPU tasks now without doing that :(((

A note I found about adding <flops> to app_info might be a solution, but not really sure since it was a discussion regarding GPUs and this error. And I don't see it in BIONC notes regarding CPU apps.

So I have:
Benchmark results:
Number of CPUs: 12
3481 floating point MIPS (Whetstone) per CPU
12936 integer MIPS (Dhrystone) per CPU

So does this look right??
   </app>
      <file_info>
         <name>MBv8_8.05r3345_avx_linux64</name>
         <executable/>
      </file_info>
     <app_version>
     <app_name>setiathome_v8</app_name>
     <platform>x86_64-pc-linux-gnu</platform>
     <version_num>800</version_num>
       ...... <flops>3481</flops> ......  
      <file_ref>
        <file_name>MBv8_8.05r3345_avx_linux64</file_name>
        <main_program/>
      </file_ref>
   </app_version>
Or is that times 1,000,000
Or will it even work?

Anyone know? Anyone else run into this and found a way out?
ID: 1867570 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1867910 - Posted: 18 May 2017, 10:33:15 UTC - in response to Message 1867570.  
Last modified: 18 May 2017, 10:36:46 UTC

Just to add a note for anyone else that may have this issue. Adding <flops> to the CPU app_info does work to adjust CPU task runtimes.

For me:
Benchmark results:
Number of CPUs: 12
3481 floating point MIPS (Whetstone) per CPU
12936 integer MIPS (Dhrystone) per CPU

Using 3481 x 1,000,000 / 50 ... or
<p_fpops> (from client_state) / 50
Works to 'normalize' runtimes for new CPU task downloads.
Don't ask why 50 seems to work, but it does, at least for my 2 computers.
At least it is close, it can be tweaked from there.
ID: 1867910 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1869712 - Posted: 27 May 2017, 3:45:12 UTC - in response to Message 1867570.  

. . Hi Brent

. . I am going to have to look into thaty CPU2GPU app. I need to do something to top up my GPU cache for the outages. Especially when I convert Bertie over to Linux.

. . BTW, was it you who mentioned something about a configurable setting in Ubuntu that allows you to move the max/min and close buttons from the default top left corner to the top right corner? I really would like to be able to do that now.

Stephen

??
ID: 1869712 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1869726 - Posted: 27 May 2017, 5:35:12 UTC - in response to Message 1869712.  

No I didn't mention moving the buttons. But Crimson Mint does have a very Windoze look to it, including Max/Min/Close.
The answer is in the other thread about changing the GUI to one you like. I know you read that :)
I may try that too at some point, but I can live with it.
ID: 1869726 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1871480 - Posted: 7 Jun 2017, 0:29:50 UTC - in response to Message 1803959.  

Ok, I'm confused. I got this message because I am running SetiBeta.

Reading configuration files...
Warning: This machine appears to be attached to SETI@Home Beta.
If there are active Beta work units, this program may reassign them improperly,
as they look like SETI@Home work. OK to proceed (Y to continue)?Y


So what am I supposed to do, I don't want the SetiBeta files to get scrambled into the production queue?

Thanks,
Tom
A proud member of the OFA (Old Farts Association).
ID: 1871480 · Report as offensive
Previous · 1 . . . 29 · 30 · 31 · 32 · 33 · 34 · 35 . . . 37 · Next

Message boards : Number crunching : GUPPI Rescheduler for Linux and Windows - Move GUPPI work to CPU and non-GUPPI to GPU


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.