Message boards :
Number crunching :
Running Out of Data
Message board moderation
Author | Message |
---|---|
centuryx476 Send message Joined: 20 Apr 11 Posts: 27 Credit: 578,121 RAC: 0 |
Hello, I'm about to bring 32 Servers online all dedicated for one purpose and that is to crunch numbers all day every day. I read on another part of the forum that there is not much data left to crunch. Is this true? |
Rasputin42 Send message Joined: 25 Jul 08 Posts: 412 Credit: 5,834,661 RAC: 0 |
No. There is only one type of task AP (Astropulse) that is not going to be available for a while. But there is plenty of work for the other type. So go ahead, bring them online. |
centuryx476 Send message Joined: 20 Apr 11 Posts: 27 Credit: 578,121 RAC: 0 |
Thank you very much. Now the task begins to bring them online. |
Rasputin42 Send message Joined: 25 Jul 08 Posts: 412 Credit: 5,834,661 RAC: 0 |
Just be aware, that there is going to be the weekly server maintenance starting in about 2 hours. This means, that you cannot get new work for about 4h. |
centuryx476 Send message Joined: 20 Apr 11 Posts: 27 Credit: 578,121 RAC: 0 |
That is fine. It will take me some time to get them online and configured properly. Need to create a whole new VLAN, expand my perimeter network, allocate the proper RAM and Hard drive space, make powershell scripts, static IP and subnet configuration Ohh so much to do in so little time. |
centuryx476 Send message Joined: 20 Apr 11 Posts: 27 Credit: 578,121 RAC: 0 |
If I have 32 Linux servers crunching numbers 24/7. They will only be using CPU and no other tasks on the server . GenuineIntel Intel(R) Core(TM) i3-4160 CPU @ 3.60GHz [Family 6 Model 60 Stepping 3] (2 processors) This is the type of processor. How much bandwidth will I be using? Unfortunately comcast just hit my area with a 300 GB a month limit and I do not want to pay more for unlimited. Will number crunching take up any where near that? Thank You |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
Depends on how big a cache you set BOINC up for. Which applications do you allow work from at Seti? Multibeam is ~365KB per task download with a 32K upload, Astropulse is 8MB per task with a 16-32K upload. Run only Seti, or also other projects? |
Dr Grey Send message Joined: 27 May 99 Posts: 154 Credit: 104,147,344 RAC: 21 |
Say each core crunches around 10 workunits a day. That's around 20,000 a month. How big is a workunit? |
Dr Grey Send message Joined: 27 May 99 Posts: 154 Credit: 104,147,344 RAC: 21 |
Depends on how big a cache you set BOINC up for. So, should be below 10 GB per month |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14649 Credit: 200,643,578 RAC: 874 |
Remember that the scheduler requests/replies also contribute to the total bandwidth, not just the uploads/downloads. With a large cache and many tasks on board, they could be larger than the upload total. But I still doubt it would reach 300 GB - unless you watch films as well. |
centuryx476 Send message Joined: 20 Apr 11 Posts: 27 Credit: 578,121 RAC: 0 |
Seti@home is the only project I will run on the servers. My worry is that 32 servers is alot and they can probably crunch through alot in a week so I don't want to get hit with a bandwidth warning. I dont usually stream - not massively though |
HAL9000 Send message Joined: 11 Sep 99 Posts: 6534 Credit: 196,805,888 RAC: 57 |
In your BOINC Computing preferences. There is a setting to limit bandwidth usage. Limit usage to --- MB every --- days This setting is per host. You machines will probably run ~1 hr for MB & ~4 hr for AP tasks. So that would be 48 MB tasks or 12 AP tasks per day. If you use a higher number like 500K to estimate the total traffic for MB then that is about 24MB/day 720MB/mo for each host. For 32 hosts that looks more like 768MB/day 23GB/mo. Again using a high number for AP of 9MB per task. The numbers for a single host would be 108MB/day 3.24GB/mo or for all 32 hosts 3.46GB/day 103.68GB/mo. SETI@home classic workunits: 93,865 CPU time: 863,447 hours Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[ |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
You machines will probably run ~1 hr for MB His 4 current i3 servers have all only been running MBs and have run times of between 6,600 seconds minimum and 24,900 seconds maximum. So we have to wonder what else his servers are doing. Running BOINC/Seti is one thing, but are they also serving something? |
centuryx476 Send message Joined: 20 Apr 11 Posts: 27 Credit: 578,121 RAC: 0 |
The only other things they have running is a network connection (naturally). Those 4 linux servers are test beds and will be replaced once I get all the kinks out and settings correct. They are small servers - Headless with no GUI and running off of 192 MB of RAM and running in Hyper-V Where do you see these run times? I'm still learning the smaller details here on the website. |
HAL9000 Send message Joined: 11 Sep 99 Posts: 6534 Credit: 196,805,888 RAC: 57 |
The only other things they have running is a network connection (naturally). You can see your tasks from Your Account or by going to your host list. From the host list select tasks for the machine you want to view & then either Valid or Validation pending to see tasks you have completed. Valid is preferable so that you know you are viewing tasks that completed correctly. Once you are looking at your tasks for a host you can then select the task ID, far left column, to see the details. MB tasks will contain WU true angle range is :. Which lets us know the angle range of the task. A normal task will have a value near 0.42 & is what you want to look for as a base line. angle range much high we refer to as VHAR(Very High Angle Range), or shorties, & tasks with a low log angle range VLAR(Very Low Angle Range). VLAR tasks will also have vlar in the task name. 03mr11ab.1151.2112.438086664206.12.242.vlar_2 vs 03ap11ac.12434.22153.438086664203.12.97_1 If theses 32 servers are running in Hyper-V how many physical servers are there? I was estimating the run times based on my i5-4670K CPU @ 3.40GHz when running 4 CPU tasks at a time. So running 2 at a time @ 3.6GHz I would imagine that would make up for the slight loss from the VM layer & run about the same time or a bit faster. SETI@home classic workunits: 93,865 CPU time: 863,447 hours Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[ |
Filipe Send message Joined: 12 Aug 00 Posts: 218 Credit: 21,281,677 RAC: 20 |
With aproximately 1500 tapes left to re-process with Sah V7 from 2011 and 2010. Any one guess when we will run out of data? How many months a the current rate of processing? Obviously, i'm not considering future 2015 data from Arecibo (Einstein already as receive more) and GBT data. |
kittyman Send message Joined: 9 Jul 00 Posts: 51468 Credit: 1,018,363,574 RAC: 1,004 |
With aproximately 1500 tapes left to re-process with Sah V7 from 2011 and 2010. Any one guess when we will run out of data? Not any time soon. Eric and Matt and Jeff have more cats they have not pulled from the bag yet. And are working on the GBT thingy as fast as they are able to. We are gonna have more work to process well into the next century. I suspect that it shall be long after I pass away. "Freedom is just Chaos, with better lighting." Alan Dean Foster |
rob smith Send message Joined: 7 Mar 03 Posts: 22160 Credit: 416,307,556 RAC: 380 |
SETI@Home goes through between 5 and 20 tapes a day, depending on the data quality on the tape. So there is between 50 and 200 days of data available (probably nearer the latter) As for the supply from Arecibo, Einstein does not use the same tapes as SETI@Home, so it is quite possible for them to have so new data and S@H not (or visa-versa). There are a few new tapes "on the way", but it is not obvious to anyone "out here" as to when and how many. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
kittyman Send message Joined: 9 Jul 00 Posts: 51468 Credit: 1,018,363,574 RAC: 1,004 |
SETI@Home goes through between 5 and 20 tapes a day, depending on the data quality on the tape. So there is between 50 and 200 days of data available (probably nearer the latter) I have not received a reply as to when that new batch of data is due. "Freedom is just Chaos, with better lighting." Alan Dean Foster |
Filipe Send message Joined: 12 Aug 00 Posts: 218 Credit: 21,281,677 RAC: 20 |
@ Richard How many 2011 tapes have been processed this month? |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.