VINE -- A numerical code for simulating astrophysical systems using particles II: Implementation and performance characteristics

We continue our presentation of VINE. We begin with a description of relevant architectural properties of the serial and shared memory parallel computers on which VINE is intended to run, and describe their influences on the design of the code itself. We continue with a detailed description of a number of optimizations made to the layout of the particle data in memory and to our implementation of a binary tree used to access that data for use in gravitational force calculations and searches for SPH neighbor particles. We describe modifications to the code necessary to obtain forces efficiently from special purpose `GRAPE hardware. We conclude with an extensive series of performance tests, which demonstrate that the code can be run efficiently and without modification in serial on small workstations or in parallel using OpenMP compiler directives on large scale, shared memory parallel machines. We analyze the effects of the code optimizations and estimate that they improve its overall performance by more than an order of magnitude over that obtained by many other tree codes. Scaled parallel performance of the gravity and SPH calculations, together the most costly components of most simulations, is nearly linear up to maximum machine sizes available to us (120 processors on an Origin~3000). At similar accuracy, performance of VINE, used in GRAPE-tree mode, is approximately a factor two slower than that of VINE, used in host-only mode. Optimizations of the GRAPE/host communications could improve the speed by as much as a factor of three, but have not yet been implemented in VINE.

VINE -- A numerical code for simulating astrophysical systems using particles I: Description of the physics and the numerical methods

We present a Fortran 95 code for simulating the evolution of astrophysical systems using particles to represent the underlying fluid flow. The code is designed to be versatile, flexible and extensible, with modular options that can be selected either at compile time or at run time. We include a number of general purpose modules describing a variety of physical processes commonly required in the astrophysical community. The code can be used as an N-body code to evolve a set of particles in two or three dimensions using either a Leapfrog or Runge-Kutta-Fehlberg integrator, with or without individual timesteps for each particle. Particles may interact gravitationally as $N$-body particles, and all or any subset may also interact hydrodynamically, using the Smoothed Particle Hydrodynamic (SPH) method. Massive point particles (`stars) which may accrete nearby SPH or $N$-body particles may also be included. The default free boundary conditions can be replaced by a module to include periodic boundaries. Cosmological expansion may also be included. An interface with special purpose `GRAPE hardware may also be selected. If available, forces obtained from the GRAPE coprocessors may be transparently substituted for those obtained from the default tree based calculation. The code may be run without modification on single processors or in parallel using OpenMP compiler directives on large scale, shared memory parallel machines. In comparison to the Gadget-2 code of Springel 2005, the gravitational force calculation is $approx 3.5 - 4.8$ times faster with VINE when run on 8 Itanium~2 processors in an SGI Altix, while producing nearly identical outcomes in our test problems. We present simulations of several test problems, including a merger simulation of two elliptical galaxies with 800000 particles.