Commercial cloud platforms are a powerful technology for astronomical research. Despite the benefits of cloud computing -- such as on-demand scalability and reduction of systems management overhead -- confusion over how to manage costs remains, for many, one of the biggest barriers to entry. This confusion is exacerbated by the rapid growth in services offered by commercial providers, by the growth in the number of these providers, and by storage, compute, and I/O metered at separate rates -- all of which can change without notice. As a rule, processing is very cheap, storage is more expensive, and downloading is very expensive. Thus, an application that produces large image data sets for download will be far more expensive than an application that performs extensive processing on a small data set. This Birds of a Feather (BoF) session aimed to quantify the above statement by presenting case studies of the costing of astronomy applications on commercial clouds that covered a range of processing scenarios; these presentations were the basis for discussion by the attendees.
Maximizing the public impact of astronomy projects in the next decade requires NSF-funded centers to support the development of online, mobile-friendly outreach and education activities. EPO teams with astronomy, education, and web development expertise should be in place to build accessible programs at scale and support astronomers doing outreach.
Many astronomy data centres still work on filesystems. Industry has moved on; current practice in computing infrastructure is to achieve Big Data scalability using object stores rather than POSIX file systems. This presents us with opportunities for portability and reuse of software underlying processing and archive systems but it also causes problems for legacy implementations in current data centers.
Commodity cloud computing, as provided by commercial vendors such as Amazon, Google, and Microsoft, has revolutionized computing in many sectors. With the advent of a new class of big data, public access astronomical facility such as LSST, DKIST, and WFIRST, there exists a real opportunity to combine these missions with cloud computing platforms and fundamentally change the way astronomical data is collected, processed, archived, and curated. Making these changes in a cross-mission, coordinated way can provide unprecedented economies of scale in personnel, data collection and management, archiving, algorithm and software development and, most importantly, science.
In recent years Java has matured to a stable easy-to-use language with the flexibility of an interpreter (for reflection etc.) but the performance and type checking of a compiled language. When we started using Java for astronomical applications around 1999 they were the first of their kind in astronomy. Now a great deal of astronomy software is written in Java as are many business applications. We discuss the current environment and trends concerning the language and present an actual example of scientific use of Java for high-performance distributed computing: ESAs mission Gaia. The Gaia scanning satellite will perform a galactic census of about 1000 million objects in our galaxy. The Gaia community has chosen to write its processing software in Java. We explore the manifold reasons for choosing Java for this large science collaboration. Gaia processing is numerically complex but highly distributable, some parts being embarrassingly parallel. We describe the Gaia processing architecture and its realisation in Java. We delve into the astrometric solution which is the most advanced and most complex part of the processing. The Gaia simulator is also written in Java and is the most mature code in the system. This has been successfully running since about 2005 on the supercomputer Marenostrum in Barcelona. We relate experiences of using Java on a large shared machine. Finally we discuss Java, including some of its problems, for scientific computing.
Brief outline of Science Operations Centre activities for Gaia.
The ESAC Gaia team engages in a form of eXtreme programming while the DPAC will follow a series of six month development cycles modeled on this approach. As a project within the European Space Agency the European Committee for Space Standardization (ECSS) standards are required. We present the bringing together of these realms.
