Stoney Supercomputer
Hardware
Stoney (stoney.ichec.ie) is a Bull Novascale R422-E2 cluster with 64 compute nodes. Each compute node has two 2.8GHz Intel (Nehalem EP) Xeon X5560 quad-core processors and 48GB of RAM. This results in a total of 512 cores and 3072GB of RAM available for jobs.
The nodes are interconnected via a half-blocking fat-tree network of ConnectX Infiniband (DDR) providing high bandwidth and low latency for both computional communications and storage access. To maximise interconnect performance, the resource manager is configured to provide fully non-blocking node allocations to jobs. Storage is provided by an EMC CLARiiON CX4-120 SAN with 21TB (formatted) of capacity to the compute nodes via two Lustre filesystems. Each compute node also provides 870GB of local scratch capacity on a directly attached hard disk.
A number of the compute nodes are attached to NVIDIA Tesla S1070 Computing System nodes. These Tesla nodes can potentially deliver 4 TFlops (peak) of single precision floating point performance. Their intended use is research to help address the difficult challenge of harnessing the potential power of GPUs for conventional computing.
In addtion to the compute nodes a set of service, administrative and storage nodes provide user login, batch scheduling, management, the Lustre filesystems, etc.
Stoney is cooled by high-efficiency cool cabinet doors provided by Bull. These doors are mounted directly to the rear of the racks and maximise energy efficiency by providing chilled water heat exchange close to the source.
Stoney was funded under the PRTLI Cycle 4 funded project e-INIS. This grant was awarded to NUI Galway who in turn provide the system for national use managed by ICHEC. As part of this agreement NUI Galway owns a percentage of system time.
Software
A large number of scientific software packages, libraries, compilers and other tools are installed and maintained on Stoney by ICHEC. A full listing of these will be made available on the software page.
Scheduling Policy
The following queues are available on Stoney:
| Job Size (cores) | Maximum Walltime | Maximum Available Cores | |
|---|---|---|---|
| short32 | 32 | 48 hours | 192 |
| short64 | 64 | 48 hours | 192 |
| short128 | 128 | 48 hours | 128 |
| long8 | 8 | 120 hours | 128 | long16 | 16 | 120 hours | 128 |
Note:
- Users should avoid specifying a queue in their PBS scripts. Specifying the walltime and processor requirements will result in the job being routed to the optimum queue.
- Processing cores on Stoney are allocated in multiples of eight (one node) and also in binary multiples of nodes. This means that users can only request core counts of 8, 16, 32, 64...
- Hard and soft throttling limits are in place to prevent a single user from flooding the entire system.
- Queued jobs gain priority at different rates depending on many factors.
Documentation
Introductory documentation for Stoney will be added in our documentation section.
Photos
Some photos of the Stoney system:
