Message boards :
Number crunching :
finish file present too long
Message board moderation
Previous · 1 · 2 · 3 · 4
Author | Message |
---|---|
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14654 Credit: 200,643,578 RAC: 874 |
I don't know what'that means... Nor do I - ask Ivan how it works. |
jason_gee Send message Joined: 24 Nov 06 Posts: 7489 Credit: 91,093,184 RAC: 0 |
I don't know what'that means... Dr Ivan's tapped into an alternate source with bosons and stuff. Too busy to send me toys and ideas to play with at the moment I expect :D "Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14654 Credit: 200,643,578 RAC: 874 |
VBox + photobucket between them slowed that machine to a crawl, but I just got there in time to edit. Conclusion - David's tinkerings so far have done nothing whatsoever to address issue #1392 as presented - i.e. that the IO and memory demands of VM tasks under BOINC are too disruptive to normal foreground use. If anything, they're much worse than when I last tried this - but I did deliberately test the machine showing most stress already. |
jason_gee Send message Joined: 24 Nov 06 Posts: 7489 Credit: 91,093,184 RAC: 0 |
well my only contentions remain that in a state of overcommit the only thing you can do to improve things is reduce workload. Tinkering things beyond the point of contention will only ever be a triage situation. "Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions. |
jason_gee Send message Joined: 24 Nov 06 Posts: 7489 Credit: 91,093,184 RAC: 0 |
I would even go out on a limb, to say that if Boinc overnight doubled the default CPU requirement for each task, then the overall throughput might increase by freeing cores on individual hosts. Hard to justify without an actual experiment of some sort of course. "Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14654 Credit: 200,643,578 RAC: 874 |
well my only contentions remain that in a state of overcommit the only thing you can do to improve things is reduce workload. Tinkering things beyond the point of contention will only ever be a triage situation. Yes, just had a very vivid demonstration of that. My screenshot in the edited post shows that the VM machine had been running for 33 minutes. BOINC Manager on the local machine showed that the task had been running for 20 minutes - and it continued to show 20 minutes, for at least the next half hour. I eventually posted from a different machine, and shut down all foreground tasks on that one. Meanwhile, my central BoincView monitoring machine upstairs had failed to contact that client for even longer. When I completely closed the BOINC Manager which had been trying to update every second, the remote machine (which is on a 30-second update cycle) finally caught up. Which I think is a problem Jord put his finger on - RPC priority. With the client in background mode and the machine under stress, an RPC every second is too much - the client couldn't answer the previous request before the next one arrived. I was testing on my 8-core dual Xeon with GTX 470 - so there's lots to report with each RPC. With everything closed, BoincView's (configurable) 30-second refresh is happily keeping pace - but BOINC Manager's RPC rate isn't configurable. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14654 Credit: 200,643,578 RAC: 874 |
I would even go out on a limb, to say that if Boinc overnight doubled the default CPU requirement for each task, then the overall throughput might increase by freeing cores on individual hosts. Hard to justify without an actual experiment of some sort of course. Unconvinced. I think that might need testing on a wide variety of BOINC projects. It might be true here or at Einstein, with tightly optimised CPU applications - but there are a lot of other projects with simpler and more relaxed apps. |
jason_gee Send message Joined: 24 Nov 06 Posts: 7489 Credit: 91,093,184 RAC: 0 |
Yes that's a tenuous limb, but I'll hold onto that straw knowing that each layer of processing is about 10x slower than the previous, so smaller is still better "Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions. |
jason_gee Send message Joined: 24 Nov 06 Posts: 7489 Credit: 91,093,184 RAC: 0 |
Yeah RPC's IMO should be the highest possible priority, on the agreement they are short and sweet. Prompt service is good service. "Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions. |
William Send message Joined: 14 Feb 13 Posts: 2037 Credit: 17,689,662 RAC: 0 |
Can we agree that VM integration is suboptimal? :D A person who won't read has no advantage over one who can't read. (Mark Twain) |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14654 Credit: 200,643,578 RAC: 874 |
Can we agree that VM integration is suboptimal? :D Yes. |
Brent Norman Send message Joined: 1 Dec 99 Posts: 2786 Credit: 685,657,289 RAC: 835 |
You guys give me a headache :( |
William Send message Joined: 14 Feb 13 Posts: 2037 Credit: 17,689,662 RAC: 0 |
well my only contentions remain that in a state of overcommit the only thing you can do to improve things is reduce workload. Tinkering things beyond the point of contention will only ever be a triage situation. as i said. you need to cut those VM tasks when the load goes up. A person who won't read has no advantage over one who can't read. (Mark Twain) |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
I checked around a bit, but can only find a reference to being able to set the call priority on RPCs on a Windows 2000 RPC server, and not much after that. Unless MSMQ works that way on later Windows as well. It probably doesn't. Which then means that the OS dictates the RPC priority, not the client/application. Now, aside from that, these slow start-ups/refreshes are of course also an effect of where the data loaded into the client is coming from. In my case, when I want to start BOINC at Windows start-up I must take into account that all data is read from (relatively) slow HDDs (even though they spin at 7,200 rpms). Is the data read from the outside of the platters (slow), or the middle (fast)? How fragmented is the data (all over the platter)? And of course, at Windows start-up it's not just BOINC that starts up, but also Windows and all of its services and stuff, plus loads of other programs. I prefer to let my computer hibernate, write all data in memory to a file on disk and then close down. This speeds up the starting of the computer, but only if: 1. I don't allow BOINC to run at that time. 2. I close down my Bittorrent client (you don't want to know how much speed you lose on opening 150 ports). 3. I have no AV running. All this would probably be going faster when read from a SSD. But alas, I haven't seen a 2TB SSD yet that's cheap enough to get and test with. ;-) Now, just imagine that your VM has to load 1.4GB of data at Windows start-up on a client that is running at low priority and has update RPCs running at the will of the operating system (ref. Richard's test). No, you do the imagining, my headache is nasty enough already. And I forget, what priority do the VMs run in? And is that just the actual VMs, or also the tasks they run? And what about the hypervisor? |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14654 Credit: 200,643,578 RAC: 874 |
Now, just imagine that your VM has to load 1.4GB of data at Windows start-up on a client that is running at low priority and has update RPCs running at the will of the operating system (ref. Richard's test). No, you do the imagining, my headache is nasty enough already. The details of that particular delay are: 1) BOINC downloads a 500 MB compressed file to the project directory. 2) BOINC unpacks the 500 MB to a 1.4 GB virtual machine image, also in the project directory. I'd been through both those stages months ago, and still had the files, so no delay there. 3) BOINC copies the 1.4 GB vmi from the project directory to the slot directory, at the start of each new task. That's not an RPC, that's BOINC's own asynchronous file copy routine. That's the one which will have occupied most of the four minutes. |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
I never said it was an RPC, I was alluding to the slowness of having to read all that data from a disk drive in the middle of the mess of Windows boot-up. :) |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14654 Credit: 200,643,578 RAC: 874 |
And is that just the actual VMs, or also the tasks they run? And what about the hypervisor? If by hypervisor, you mean vboxheadless.exe, that was running at Normal priority, and with I/O priority normal, Memory priority 5. That's at the thread level - there are lots of them, but I checked all the busiest ones, and they were all the same. VBox Wrapper did run at Low priority (the same as normal CPU tasks under BOINC), but also with I/O priority normal, Memory priority 5. I don't think I can even see, let alone control, the niceness of the threads inside the Linux VM. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.