Porting s@h V7 to Linux

Message boards : Number crunching : Porting s@h V7 to Linux
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4

AuthorMessage
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1377673 - Posted: 6 Jun 2013, 21:49:33 UTC - in response to Message 1377669.  

Here's a partial listing of the benchmark results. Unfortunately we had a network glitch this afternoon in my building, which led to my desktop getting a new address from DHCP and "orphaning" my ssh session to the server. The results look close enough, if only I could come up with a method to find that pesky segfault! (Using the unstripped version didn't result in a traceback or any other information more in the stderr file.) Jason, which version of BOINC are you using on your Frankenputer?


Looks like 6.10.58.


Yep, prepackaged 6.10.58 Berkeley one installed in my home folder.... as opposed to the locally tailored 6.10.58 on my Windows machines.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1377673 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1377675 - Posted: 6 Jun 2013, 21:55:26 UTC - in response to Message 1377673.  

Here's a partial listing of the benchmark results. Unfortunately we had a network glitch this afternoon in my building, which led to my desktop getting a new address from DHCP and "orphaning" my ssh session to the server. The results look close enough, if only I could come up with a method to find that pesky segfault! (Using the unstripped version didn't result in a traceback or any other information more in the stderr file.) Jason, which version of BOINC are you using on your Frankenputer?


Looks like 6.10.58.


Yep, prepackaged 6.10.58 Berkeley one installed in my home folder.... as opposed to the locally tailored 6.10.58 on my Windows machines.


I have 7.1.7 installed on my Linux-Mint machine.

ID: 1377675 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1377678 - Posted: 6 Jun 2013, 22:00:25 UTC - in response to Message 1377675.  

I have 7.1.7 installed on my Linux-Mint machine.


Is it x64 with Cuda 5 drivers ? (if so might pay to get you running sooner rather than later, as it seems my 260 isn't very healthy)
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1377678 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1377679 - Posted: 6 Jun 2013, 22:05:28 UTC - in response to Message 1377678.  
Last modified: 6 Jun 2013, 22:05:58 UTC

I have 7.1.7 installed on my Linux-Mint machine.


Is it x64 with Cuda 5 drivers ? (if so might pay to get you running sooner rather than later, as it seems my 260 isn't very healthy)


Yes, it is currently running 319.xx on a 64-bit install.
http://setiathome.berkeley.edu/show_host_detail.php?hostid=6961644

I have been using it to test Urs apps recently.

ID: 1377679 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1377682 - Posted: 6 Jun 2013, 22:10:39 UTC - in response to Message 1377679.  

I have 7.1.7 installed on my Linux-Mint machine.


Is it x64 with Cuda 5 drivers ? (if so might pay to get you running sooner rather than later, as it seems my 260 isn't very healthy)


Yes, it is currently running 319.xx on a 64-bit install.
[url]
http://setiathome.berkeley.edu/show_host_detail.php?hostid=6961644[/url]

I have been using it to test Urs apps recently.


Great. I'll have to do a custom test build after work for Kepler class (which will be easy) then see you at CA.

My GTX 260, and the test build it's running, are good for seeing what needs to be done (like error handling similar to the Windows builds), but too flaky to tell if it'll run on a stable GPU ok.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1377682 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1377684 - Posted: 6 Jun 2013, 22:14:37 UTC - in response to Message 1377682.  

I have 7.1.7 installed on my Linux-Mint machine.


Is it x64 with Cuda 5 drivers ? (if so might pay to get you running sooner rather than later, as it seems my 260 isn't very healthy)


Yes, it is currently running 319.xx on a 64-bit install.
[url]
http://setiathome.berkeley.edu/show_host_detail.php?hostid=6961644[/url]

I have been using it to test Urs apps recently.


Great. I'll have to do a custom test build after work for Kepler class (which will be easy) then see you at CA.

My GTX 260, and the test build it's running, are good for seeing what needs to be done (like error handling similar to the Windows builds), but too flaky to tell if it'll run on a stable GPU ok.


And of course I will probably end up testing it after I get off work tomorrow morning as I will most likely be going to bed in about 4 hours or so.

ID: 1377684 · Report as offensive
Juha
Volunteer tester

Send message
Joined: 7 Mar 04
Posts: 388
Credit: 1,857,738
RAC: 0
Finland
Message 1377688 - Posted: 6 Jun 2013, 22:31:46 UTC - in response to Message 1377660.  
Last modified: 6 Jun 2013, 22:41:52 UTC

if only I could come up with a method to find that pesky segfault! (Using the unstripped version didn't result in a traceback or any other information more in the stderr file.)

Add a pause at the beginning and once BOINC has started it attach gdb to running process?

[And hope the pause doesn't make the error disappear :)

edit: Come to think of it, you could configure you system to dump core and give that to gdb.
ID: 1377688 · Report as offensive
Profile ivan
Volunteer tester
Avatar

Send message
Joined: 5 Mar 01
Posts: 783
Credit: 348,560,338
RAC: 223
United Kingdom
Message 1377692 - Posted: 6 Jun 2013, 22:35:22 UTC - in response to Message 1377673.  
Last modified: 6 Jun 2013, 22:36:19 UTC

Here's a partial listing of the benchmark results. Unfortunately we had a network glitch this afternoon in my building, which led to my desktop getting a new address from DHCP and "orphaning" my ssh session to the server. The results look close enough, if only I could come up with a method to find that pesky segfault! (Using the unstripped version didn't result in a traceback or any other information more in the stderr file.) Jason, which version of BOINC are you using on your Frankenputer?


Looks like 6.10.58.


Yep, prepackaged 6.10.58 Berkeley one installed in my home folder.... as opposed to the locally tailored 6.10.58 on my Windows machines.


Did you actually build that yourself? Otherwise what are you linking the CUDA executables against?

I'm not very familiar with git -- as far as I understand the git clone command downloaded all commits to the repository, so do I just do a checkout command for the 6.10.58 version (presumably after a make clean...) and then do the ./_autosetup and "./configure --disable-server --enable-manager" commands (plus make) to rebuild it?
ID: 1377692 · Report as offensive
Profile ivan
Volunteer tester
Avatar

Send message
Joined: 5 Mar 01
Posts: 783
Credit: 348,560,338
RAC: 223
United Kingdom
Message 1377694 - Posted: 6 Jun 2013, 22:48:04 UTC - in response to Message 1377688.  

if only I could come up with a method to find that pesky segfault! (Using the unstripped version didn't result in a traceback or any other information more in the stderr file.)

Add a pause at the beginning and once BOINC has started it attach gdb to running process?

[And hope the pause doesn't make the error disappear :)

edit: Come to think of it, you could configure you system to dump core and give that to gdb.

Thanks, I'll give that some consideration, probably at the weekend. I need to concentrate on work work tomorrow, have to come up with a working system of something I've been delegated lately -- almost there!
ID: 1377694 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1377700 - Posted: 6 Jun 2013, 23:07:03 UTC - in response to Message 1377692.  

Did you actually build that yourself? Otherwise what are you linking the CUDA executables against?


I configured/compiled/linked against Boincapi checked out from the old (?) svn location, instead of git, which is calling itself 7.1.0
( http://boinc.berkeley.edu/svn/trunk/boinc ), how up to date that is against Git (or if the same) I don't know.

As for the suspend/exit critical section stuff, to guard GPU code, I'll probably put my semantically different (but working on Windows) implementation in, pending resolution of Boinc devs discussions on fixing problems with theirs with the Einstein crew.



"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1377700 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1377706 - Posted: 6 Jun 2013, 23:40:10 UTC

I am keeping my BOINC at the newest rev available by using these.

sudo add-apt-repository ppa:costamagnagianfranco/boinc
sudo apt-get update

That should work on Ubuntu 12.04, but probably not your system Ivan.

ID: 1377706 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 1377792 - Posted: 7 Jun 2013, 5:03:22 UTC - in response to Message 1377700.  

I configured/compiled/linked against Boincapi checked out from the old (?) svn location, instead of git, which is calling itself 7.1.0
( http://boinc.berkeley.edu/svn/trunk/boinc ), how up to date that is against Git (or if the same) I don't know.
...

That gets code updated to October 11 2012, when the transition to git was made.

One way to get fully up to date code without actually installing git is to download the latest snapshot from http://boinc.berkeley.edu/gitweb/?p=boinc-v2.git;a=summary. The tar.bz2 version was under 40 MiB the last time I did that. Be warned, however, that gives you files which all have the time and date of the snapshot...
                                                                   Joe
ID: 1377792 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1377851 - Posted: 7 Jun 2013, 7:28:02 UTC - in response to Message 1377792.  

I configured/compiled/linked against Boincapi checked out from the old (?) svn location, instead of git, which is calling itself 7.1.0
( http://boinc.berkeley.edu/svn/trunk/boinc ), how up to date that is against Git (or if the same) I don't know.
...

That gets code updated to October 11 2012, when the transition to git was made.

One way to get fully up to date code without actually installing git is to download the latest snapshot from http://boinc.berkeley.edu/gitweb/?p=boinc-v2.git;a=summary. The tar.bz2 version was under 40 MiB the last time I did that. Be warned, however, that gives you files which all have the time and date of the snapshot...
                                                                   Joe


well as typical for when you get a new dev environment up and running, I found the last perfect settings to stuff all the thread safety & Cuda cmpute capability versions into the build & the tree 'broke'... lol. so as I put it back together 'll take a look at the snapshot option...even though I guess I'llhave to figure out git eventually anyway. Back to step 115
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1377851 · Report as offensive
Profile William
Volunteer tester
Avatar

Send message
Joined: 14 Feb 13
Posts: 2037
Credit: 17,689,662
RAC: 0
Message 1377883 - Posted: 7 Jun 2013, 8:19:16 UTC - in response to Message 1377851.  

I configured/compiled/linked against Boincapi checked out from the old (?) svn location, instead of git, which is calling itself 7.1.0
( http://boinc.berkeley.edu/svn/trunk/boinc ), how up to date that is against Git (or if the same) I don't know.
...

That gets code updated to October 11 2012, when the transition to git was made.

One way to get fully up to date code without actually installing git is to download the latest snapshot from http://boinc.berkeley.edu/gitweb/?p=boinc-v2.git;a=summary. The tar.bz2 version was under 40 MiB the last time I did that. Be warned, however, that gives you files which all have the time and date of the snapshot...
                                                                   Joe


well as typical for when you get a new dev environment up and running, I found the last perfect settings to stuff all the thread safety & Cuda cmpute capability versions into the build & the tree 'broke'... lol. so as I put it back together 'll take a look at the snapshot option...even though I guess I'llhave to figure out git eventually anyway. Back to step 115

It's not that difficult (i.e. I managed to get a working copy ;) )
I haven't updated the checkout on the VPN host for some time, I'll try to get to that.
A person who won't read has no advantage over one who can't read. (Mark Twain)
ID: 1377883 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1378347 - Posted: 7 Jun 2013, 23:44:31 UTC
Last modified: 7 Jun 2013, 23:58:00 UTC

Okey doke, got a little further with a few things

@Ivan, there's a few gotchas in the makefile which used nVidia's chirp before & now uses mine. I had to disable the special make target for cudaAcc_CalcChirpData[_sm13].o, since I now use a single chirp file that builds both sm_13 and later code differently, internally with directives, instead of a separate cu file. Also, for supporting all Cuda GPUs with forward compatible PTX, the gpu-architecture parameter in NVCCFLAGS needs to be replaced with:
-gencode arch=compute_10,code=sm_10 \
-gencode arch=compute_11,code=sm_11 \
-gencode arch=compute_12,code=sm_12 \
-gencode arch=compute_13,code=sm_13 \
-gencode arch=compute_20,code=sm_20 \
-gencode arch=compute_20,code=sm_21 \
-gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_30,code=sm_35 \
-gencode arch=compute_30,code=compute_30



@Urs , besides going to 32 bit instead of 64, what needs to change with the linkage here for widest possible compatibility ? I did manage to figure out rpath, so that the Cuda libraries would be found alongside the executable.

jason@SinbadUbi:~/seti/Xbranch/client/test$ readelf -d setiathome_x41zc_x86_64-pc-linux-gnu_cuda50 | grep RPATH
0x000000000000000f (RPATH) Library rpath: [$ORIGIN]
jason@SinbadUbi:~/seti/Xbranch/client/test$ ldd setiathome_x41zc_x86_64-pc-linux-gnu_cuda50
linux-vdso.so.1 => (0x00007fff20df2000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fadf08d6000)
libcudart.so.5.0 => /home/jason/seti/Xbranch/client/test/./libcudart.so.5.0 (0x00007fadf067c000)
libcufft.so.5.0 => /home/jason/seti/Xbranch/client/test/./libcufft.so.5.0 (0x00007fadee6c5000)

libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fadee3c5000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fadee0c9000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fadedeb2000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fadedaf3000)
/lib64/ld-linux-x86-64.so.2 (0x00007fadf0b13000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007faded8ef000)

"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1378347 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1378374 - Posted: 8 Jun 2013, 0:55:50 UTC - in response to Message 1377883.  
Last modified: 8 Jun 2013, 1:01:52 UTC

It's not that difficult (i.e. I managed to get a working copy ;) )
I haven't updated the checkout on the VPN host for some time, I'll try to get to that.
Thanks, Haven't tried updating yet from the remote host mostly due to network issues here after ISP upgrades in the street. Looks like I didn't break the boincapi part though, so I'll stick with what's working until the build system is much improved.

At the moment for the app it's a collection of direct Makefile hacks. I'll need to figure out how the modern _autosetup and configure scripts are created/maintained. Those tools were either not there or not useful enough for what I needed back in 1995, When I last did Linux development on a large scale. Not a huge amount seems to have changed, but what has can be genuinely challenging.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1378374 · Report as offensive
Urs Echternacht
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 692
Credit: 135,197,781
RAC: 211
Germany
Message 1378384 - Posted: 8 Jun 2013, 1:32:11 UTC - in response to Message 1378347.  

You need to check for the apps min. required library versions.
Could you check app with "objdump -x setiathome_x41zc_x86_64-pc-linux-gnu_cuda50" again ? That will dump all eventually needed lib headers and the minimum versions. "readelf -V" should give similar info.

example:
> objdump -x MBv7_7.05r1844_sse2_clATI_HD5_linux64 

MBv7_7.05r1844_sse2_clATI_HD5_linux64:     file format elf64-x86-64
MBv7_7.05r1844_sse2_clATI_HD5_linux64
architecture: i386:x86-64, flags 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
start address 0x0000000000408ea0

Program Header:
...
something
...

Dynamic Section:
  NEEDED               libdl.so.2
  NEEDED               libOpenCL.so.1
  NEEDED               libm.so.6
  NEEDED               libpthread.so.0
  NEEDED               libc.so.6
  NEEDED               ld-linux-x86-64.so.2
...

Version References:
  required from ld-linux-x86-64.so.2:
    0x0d696913 0x00 06 GLIBC_2.3
  required from libpthread.so.0:
    0x09691a75 0x00 04 GLIBC_2.2.5
  required from libOpenCL.so.1:
    0x028568b0 0x00 03 OPENCL_1.0
  required from libc.so.6:
    0x0d696913 0x00 05 GLIBC_2.3
    0x09691a75 0x00 02 GLIBC_2.2.5
...
libcudart.so.5.0 and libcufft.so.5.0 will have to be present when running your CUDA app. Are these delivered with the app or do you want to assume that the libs are installed already ?
...what needs to change with the linkage here for widest possible compatibility ?
Add "-static-libstdc++ -static-libgcc" to LDFLAGS. All other libs seem to be needed.
_\|/_
U r s
ID: 1378384 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1378390 - Posted: 8 Jun 2013, 1:46:00 UTC
Last modified: 8 Jun 2013, 2:26:24 UTC

Cheers! much appreciated.

Yeah those cuda files are redistributables, and plonked in alongside the apps in app_info so the user doesn't have to install the whole Cuda toolkit.

those results... [Edit: examples for this build, min GLIBC 2.15 & CXX 3.4.11 right ? ]

jason@SinbadUbi:~/seti/Xbranch/client/test$ objdump -x setiathome_x41zc_x86_64-pc-linux-gnu_cuda50

setiathome_x41zc_x86_64-pc-linux-gnu_cuda50: file format elf64-x86-64
setiathome_x41zc_x86_64-pc-linux-gnu_cuda50
architecture: i386:x86-64, flags 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
start address 0x000000000040ec50
...
Dynamic Section:
NEEDED libpthread.so.0
NEEDED libcudart.so.5.0
NEEDED libcufft.so.5.0
NEEDED libstdc++.so.6
NEEDED libm.so.6
NEEDED libgcc_s.so.1
NEEDED libc.so.6
RPATH $ORIGIN
...
Version References:
required from libgcc_s.so.1:
0x0b792650 0x00 14 GCC_3.0
required from libpthread.so.0:
0x09691a75 0x00 08 GLIBC_2.2.5
required from libm.so.6:
0x06969195 0x00 05 GLIBC_2.15
0x09691a75 0x00 04 GLIBC_2.2.5
required from libstdc++.so.6:
0x0bafd171 0x00 13 CXXABI_1.3.1
0x0297f861 0x00 11 GLIBCXX_3.4.11
0x056bafd3 0x00 10 CXXABI_1.3
0x02297f89 0x00 06 GLIBCXX_3.4.9
0x08922974 0x00 03 GLIBCXX_3.4
required from libc.so.6:
0x06969194 0x00 15 GLIBC_2.14
0x0d696914 0x00 12 GLIBC_2.4
0x06969191 0x00 09 GLIBC_2.11
0x09691974 0x00 07 GLIBC_2.3.4
0x09691a75 0x00 02 GLIBC_2.2.5
...


Add "-static-libstdc++ -static-libgcc" to LDFLAGS. All other libs seem to be needed.
gotit. Will try that out with enable the proper exit & error handling in the Linux codepath (dying GTX 260, is a good test for that :P)
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1378390 · Report as offensive
Urs Echternacht
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 692
Credit: 135,197,781
RAC: 211
Germany
Message 1378566 - Posted: 8 Jun 2013, 12:11:56 UTC - in response to Message 1378390.  
Last modified: 8 Jun 2013, 12:12:48 UTC

Cheers! much appreciated.

Yeah those cuda files are redistributables, and plonked in alongside the apps in app_info so the user doesn't have to install the whole Cuda toolkit.

those results... [Edit: examples for this build, min GLIBC 2.15 & CXX 3.4.11 right ? ]
...
Yes, these seem to be the highest needed library versions. Try to build it so, that the CUDA-libs are the only relevant needed libs.

Maybe NV has somewhere their lib requirements documented (in a readme, or a changelog or elsehow) for each version of the CUDA-SDK (2.2, 2.3, ...)
_\|/_
U r s
ID: 1378566 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1378610 - Posted: 8 Jun 2013, 16:14:30 UTC - in response to Message 1378566.  
Last modified: 8 Jun 2013, 16:16:24 UTC

Maybe NV has somewhere their lib requirements documented (in a readme, or a changelog or elsehow) for each version of the CUDA-SDK (2.2, 2.3, ...)



Likely, and I'll most likely have to install those toolkits in order. It'll take me a while to work out autotools to regenerate a proper source distro instead of hacking Makefiles. So another thing on the todo list.

More urgent with the test build is to get the error/temp-exit paths operational (finally a use for the broken card), then it'll be open season on making things as static & buildable as possible.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1378610 · Report as offensive
Previous · 1 · 2 · 3 · 4

Message boards : Number crunching : Porting s@h V7 to Linux


 
©2026 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.