More full report on how task reacts on different stop conditions:
1) Suspend: just paused VM, leaved in memory
2) BOINC exit: VM snapshot taken so resumed exactly from same place on BOINC restart.
3) BOINC daemon kill: VM process exited soon after. VM state seems to be lost. ~minutes passed to restore task running state inside VM. And again, just as with case of VM powering down via VirtualBox (w/o snapshot), it seems inner app lost ability to communicate with web-based interface.
http://localhost:49307/logs/cmsRun-stdout.log file doesn't get updates after restart (though csmRun uses CPU well more than 3mins already). Maybe this is something that worth to fix in next versions.

[Look for times of file update - app's log is frozen:
[TXT] FrameworkJobReport.xml 17-Aug-2015 16:33 0
[ ] MasterLog 17-Aug-2015 18:42 20K
[ ] ProcLog 17-Aug-2015 18:43 233K
[ ] StartdLog 17-Aug-2015 18:41 313K
[ ] StarterLog 17-Aug-2015 18:40 25K
[TXT] _condor_stderr 17-Aug-2015 16:32 0
[ ] _condor_stdout 17-Aug-2015 18:33 26K
[ ] cmsRun-stderr.log 17-Aug-2015 16:33 243
[TXT] cmsRun-stdout.log 17-Aug-2015 18:19 139K
[TXT] cron-stderr 17-Aug-2015 18:43 0
[TXT] cron-stdout 17-Aug-2015 18:43 0
[TXT] getProxystderr 17-Aug-2015 16:32 647
[ ] getProxystdout 17-Aug-2015 16:32 474
[ ] runGlideinerr 17-Aug-2015 16:32 28K
[ ] runGlideinout 17-Aug-2015 16:32 4.2K
[ ] scramOutput.log 17-Aug-2015 16:33 658
[ ] stderr 17-Aug-2015 16:32 747
[TXT] stdout 17-Aug-2015 16:32 0
]

In general, this VM approach look similar to one used with Parallella hardware. They download some OS image (as I understand not VM but just own Linux distro image), flash it and then device becomes node in their distributed network. BOINC's VM approach though has bigger overhead of VM much more flexible in part that device remains usable for smth else.

After some time (perhaps when data chunk finished and csmRun app restarted with new data) link to log files restored:

[TXT] FrameworkJobReport.xml 17-Aug-2015 21:08 0
[ ] MasterLog 17-Aug-2015 21:07 4.8K
[ ] ProcLog 17-Aug-2015 21:12 22K
[ ] StartdLog 17-Aug-2015 21:08 45K
[ ] StarterLog 17-Aug-2015 21:08 8.1K
[TXT] _condor_stderr 17-Aug-2015 21:07 0
[ ] _condor_stdout 17-Aug-2015 21:08 27K
[ ] cmsRun-stderr.log 17-Aug-2015 21:08 244
[TXT] cmsRun-stdout.log 17-Aug-2015 21:12 1.0K
[TXT] cron-stderr 17-Aug-2015 21:12 0
[TXT] cron-stdout 17-Aug-2015 21:12 0
[TXT] getProxystderr 17-Aug-2015 21:07 647
[ ] getProxystdout 17-Aug-2015 21:07 553
[ ] runGlideinerr 17-Aug-2015 21:07 28K
[ ] runGlideinout 17-Aug-2015 21:07 4.2K
[ ] scramOutput.log 17-Aug-2015 21:08 653
[ ] stderr 17-Aug-2015 21:07 747
[TXT] stdout 17-Aug-2015 21:07 0

EDIT: indeed, top shows run time for csmRun ~4mins only - fresh start was recently. So, regular fresh start updates links to log while restart after unexpected shutdown - doesn't.

Next to try is manual VM reconfigure to increase available memory (this host can afford that).

With VM's RAM increased to 4GB app still uses "low mem mode" (if I interprets that msg right):

* LHAPDF Version 5.9.1 *
* Configured for the following: *
* All PDFs *
* LOW MEMORY option *
* Maximum 1 concurrent set(s) *

But VM itself made use of additional RAM. 2,7G showed as used in top. Mostly for cache. Hope this would reduce disc activity from VM.

Interesting that this memory accounted in Windows task manager performance tab (used ~5GB of RAM total) but not attributed to any process on processes tab. So it's semi-invisible for Windows.

With VM's RAM increased to 4GB app still uses "low mem mode" (if I interprets that msg right):

* LHAPDF Version 5.9.1 *
* Configured for the following: *
* All PDFs *
* LOW MEMORY option *
* Maximum 1 concurrent set(s) *

But VM itself made use of additional RAM. 2,7G showed as used in top. Mostly for cache. Hope this would reduce disc activity from VM.

Interesting that this memory accounted in Windows task manager performance tab (used ~5GB of RAM total) but not attributed to any process on processes tab. So it's semi-invisible for Windows.

I believe that message is about a configuration option in the LHAPDF library, to limit the amount of memory used to store PDFs in memory (http://lhapdf.hepforge.org/lhapdf5/configure). [PDF = Parton Distribution Function, see https://en.wikipedia.org/wiki/Parton_(particle_physics) for example.]

---

First BOINC task finished, reported and got credits.
And another one started.
One more observation of inefficiency: with end of prev task and start new one VM was restarted. That is, few mins of overhead/idle CPU while OS inside VM boots and app inside VM preparing for run.
Wouldn't better not to restart VM from scratch with new BOINC task but just pause it with snapshot as done on BOINC exit? That way when new BOINC task will be launched computations inside VM would be resumed w/o initialization overhead.

EDIT: Actually, it's even more compex that appears on the first sight. It's not the same VM started on secondary task. First one I customized to use 4GB of RAM. But this one uses only 2GB again. So, each BOINC task is new VM? It would increase overhead even more, not?

Well, judging from slot/project content on new task VM image copied from project directory to slot directory and then BOINC launches VM image from slot. That is, 1,3GB VM's image copied each new task. And no inheritance between tasks possible in such architecture decision.

But why not to copy link to VM instead? That way each new task could use same VM image so VM state could be preserved between tasks and lot of startup overhead would be removed. Also, no HDD training with 1,3GB moving around ;)

LOL Raistmer if you keep this up you may get a job offer from Cern.

---

I personally would like to help them but I can't babysit a machine.

I like the fact he's pointing out all of these things. If they can streamline it enough, maybe they won't have to babysit it, then it might someday be like the rest of the projects where you can join and forget.

That would be nice....

The whole concept of projects using VM is not a good idea.

It makes it less efficient(extra overhead on users machines) and shifts the work (labor) from the project to the users.
Instead of the project writing apps for each platform (as most projects do) the users have to install and fight with the VMs.(and install a VM to run a linux app on a linux machine????)

Just my opinion.

I will still support it, but i think, it is not the best approach.

Scientists of CERN don't have the time to compile executables for dozen of different hw/sw architectures. The CERN programs are all written in Scientific Linux, which is Red Hat plus some libraries developed at CERN. Installing Virtual Box is not difficult, I have two SuSE Linux boxes and one Windowas 10 PC with Virtual Box installed and running.
Tullio

---

I tend to agree with Rasputin42. I know it is easy to install a virtual box vm as I've done it many times in the past, but I do not want to have to setup a virtual machine that must be powered on all the time to crunch, and I don't want to be forced into installing Linux on vm (and to state what should be obvious, I do not have anything against Linux, I simply don't want it).

I understand the fact that the programs are written in Scientific Linux, but they are doing themselves a big disservice by not investing the time to compile binaries for a wider audience.

Sucks for me too because I really like all the fascinating stuff I read that's going on at CERN and I'd really love the chance to help out.

I tend to agree with Rasputin42. I know it is easy to install a virtual box vm as I've done it many times in the past, but I do not want to have to setup a virtual machine that must be powered on all the time to crunch, and I don't want to be forced into installing Linux on vm (and to state what should be obvious, I do not have anything against Linux, I simply don't want it).

I understand the fact that the programs are written in Scientific Linux, but they are doing themselves a big disservice by not investing the time to compile binaries for a wider audience.

Sucks for me too because I really like all the fascinating stuff I read that's going on at CERN and I'd really love the chance to help out.

I know how you feel but on the other foot. I always seem to be waiting for SETI apps, particularly the enhanced or improved ones, to find their way to binaries that will run on Linux :).

You are not forced to install Linux on the WM. CERN has developed a vboxwrapper which takes care of all. Tasks go on for a standard 24 hours, after which you get credits. You don't even need to install BOINC, there is a CERN Summer Challenge that only needs Virtual Box. The programs running were developed mostly in FORTRAN by many universities and research centers. CERN has not written all of them.
Tullio

---

I know how you feel but on the other foot. I always seem to be waiting for SETI apps, particularly the enhanced or improved ones, to find their way to binaries that will run on Linux :).

If you wanna say they are not you pretty humilidate our Linux porting team! Currently even GPU apps are ported.

I tend to agree with Rasputin42. I know it is easy to install a virtual box vm as I've done it many times in the past, but I do not want to have to setup a virtual machine that must be powered on all the time to crunch, and I don't want to be forced into installing Linux on vm (and to state what should be obvious, I do not have anything against Linux, I simply don't want it).

I understand the fact that the programs are written in Scientific Linux, but they are doing themselves a big disservice by not investing the time to compile binaries for a wider audience.

Sucks for me too because I really like all the fascinating stuff I read that's going on at CERN and I'd really love the chance to help out.

Well, few words in VM approach defense.
First of all, computational overhead of such approach
is obvious.
But consider it from another point of view.
What if dihotomy is to be able to run in VM or
not run at all?
I'm quite sure if original author of AstroPulse would
be forced to write optimized initial app there
would be no SETI AstroPulse at all. He would just
ran up his time and moved out of lab w/o finishing app.
VM allows greatly simplify porting.So it allows
to reach otherwise unreachable for particular
sientific team hardware.
BOINC devised to be scientific tool. And the less
such tool requires to be computer scientist the better.
The more scientists from different areas could use it.
I will leave aside many other situations where
VM is needed. But just from performance
point of view to be able to run only on some
flavour of Linux or to be able to run almost on
anything Windows including makes big
difference project performance-wide.
The real crucial measure of "performance" is
the perceived need/interesting in project from particular
user. If project worhwhile it can be run even under VM :)
It doesnt mean performance should not be improved.
It means it's not show-stopper.

And now regarding "user labor". This part is simply wrong. There is no additional user labour at all (!).
What I had to do to make project running (besides additional registration that part of testing):

1) To install BOINC with VM instead of plain one.
In my case this could be excessive step cause VirtualBox existed on that host before.
2) To make adjustment from default 10GB of disk space to allow work fetch.
Again, hardly this can be considered as labour. It can be fixed in next version and it has nothing to do with VM conception per se.

And that's all (!). All other is testing not required to run project.

Yep, VM has overhead and VM consumes more PC resourses than similar native app provided such native app exists. So much in that single word in bold...

P.S. One of those "another situations" of VM requirement will be required app precision. If app has increased precision requirements it can be impossible to run native build on different environments. Then exactly the same environment in VM will help. Again - it's not question of performance. It's the question of existence for particular project.

VirtualMachines have, as far as I can see, a big advantage for the developer in that they only have to develop, test and distribute one application, that being the one for their operating system of choice. No messing around developing against several versions of Windows, several versions of Linux, and several versions of Mac, and... They choose their target VM and it custom internal operating system, then develop, test deploy one application, job done, sit back and wait for the results to come in....

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

At CPDN, which does not use Virtual Box, they divide programs by OS. Certain tasks go to Windows, others to Linux and others to MAC OS. This to avoid problems with numerical precision in the results.
Tullio

---

At CPDN, which does not use Virtual Box, they divide programs by OS. Certain tasks go to Windows, others to Linux and others to MAC OS. This to avoid problems with numerical precision in the results.
Tullio

And also to reduce the programming and testing overhead of deploying every application for every operating system.

You are not forced to install Linux on the WM. CERN has developed a vboxwrapper which takes care of all. Tasks go on for a standard 24 hours, after which you get credits. You don't even need to install BOINC, there is a CERN Summer Challenge that only needs Virtual Box. The programs running were developed mostly in FORTRAN by many universities and research centers. CERN has not written all of them.
Tullio

You're missing the part that I don't want Linux on my machine though.