This is the Daresbury Laboratory Web Server for the STFC Computational Science and Engineering Group for HPC and Grid Technology. You can find out about the projects we work on and the software we develop. Members of the Group can edit and update these pages.

HPC Grand Challenges, Software and Performance

• History of Computing at Daresbury
• Computing Grand Challenges
• Computational Research into Complex Systems - this is one of the grand challenges, perhaps for us the most important?
• Software for Agent Based Modelling and Simulation - this is a way to tackle complex systems.
  • Future Research Directions in Agent Based Modelling - a workshop held at University of Leeds 15/6/2010
  • ABM CCP development of a proposal for a collaborative computational project
  • Jeff Shank's ABM site more information on activities and researchers world wide
  • Videos about ABMs and Complexity
• Software for Computer Performance Modelling - this is also done using discrete event simulation.
• Research Data
  • Management and Analysis of Large Research Datasets - draft report
  • Data Intensive Computing - draft report

Grid, Cloud and e-Research Software

• A Book
• Some Notes on Clouds and Web 2.0
• Final Report of US Magellan Cloud Project

NW-GRID

The North West Grid, NW-GRID, is a collaboration between Daresbury Laboratory and the Universities of Lancaster, Liverpool and Manchester. With the help of NWDA funding, we established and operate a computational Grid comprising of high-performance computing systems coupled by a high-speed private fibre network between the sites. NW-GRID offers world-class services founded in the deployment and exploitation of Grid middleware technologies enabling the capabilities of the Grid to be realised in leading-edge computational science and engineering applications.

For more information see the NW-GRID Web site. You can log onto the portal if you have an account.

The following notes are mainly to remind me about things I did when installing system software.

• Environment Modules
• Globus GT-4
• Globus GT-5 coming soon. See http://www.globus.org/toolkit/docs/5.0/5.0.0/
• OpenMPI - building and using OpenMPI
• MPICH-2 - building and using MPICH-2
• Sun Grid Engine 6.2 - some hints when things go wrong
• Firewall Issues - network traffic
• Condor, see below under Campus Grid

Gentoo (and other) Linux

What is Linux?

• Gentoo (and other) Linux Hints and Tips
• LaTeX stuff
• Linux from Scratch - as recommended by Dave
• Using Autotools
• Grub and Lilo - boot managers
• sshd and tcpwrappers - more about TCP Wrappers
• NFS etc.
• Open Source Software - alternatives to popular packages

G-R-Toolkit

The G-R-Toolkit combines the best software developed at STFC during its e-Science Programme from 2001-2007. It allows users of many applications in computational research to manage their high-performance computing and data and information management tasks directly from their desktop systems. See here for more information and user manual.

Daresbury Science and Innovation Campus Grid

Here is a short introduction to our vision for the DSIC Campus Grid. An NGS Roadshow was held on the campus on 23/8/2010 see NGS Web site.

Some papers mostly relating to the use of Condor.

More information on Daresbury Condor resources is available on the DL Condor pool Web site. There are status pages for the Cockcroft Institute master ci-condor.dl.ac.uk and the DL master nodes condor-main.dl.ac.uk. A combined listing is provided here

Note that these pools are dynamic. From 12/3/2009-21/2/2010 all machines were registering with the CI Pool manager node. We are now investigating flocking to give more flexibility.

Further descriptions including installation notes are provided here.

• DL Condor Pool - Daresbury Laboratory
• CI Condor Pool - Cockcroft Institute

We are participating in the Campus Grid SIG.

• Manchester Condor Pool - some blog notes from Ian Cottam

Job Submission

This can be from a user's desktop machine using standard Condor scripts (requires the machine to be a submit node) or using the RMCS Web services client, see G-R-Toolkit documentation.

Current Work

We are investigating how to negociate the firewall between subnets on the DL site and how to flock between the DL and CI pools. See information from CamGrid http://www.escience.cam.ac.uk/projects/camgrid/firewall.html

We will later investigate back-filling the NW-GRID cluster in a similar way to how it is done in Manchester.

Virtual Research Environments and Portals

Sakai is deployed as a collaborative e-Research tool for a number of projects as follows:

• UK HPC training: http://congo.dl.ac.uk:8080/portal
• NW-GRID: the North West Grid: http://rhine.dl.ac.uk:8080/portal
• NCeSS: National Centre for e-Social Science: http://weaver.dl.ac.uk:8080/portal
• NeISS: National e-Institute for Social Simulation: http://weaver.dl.ac.uk:8080/portal/site/e-iss
• CSED: Computational Science and Engineering Department: http://cselnx8.dl.ac.uk:8080/portal
• Hartree Centre: http://yukon.dl.ac.uk:8080/portal
• Psi-k: http://cselnx9.dl.ac.uk:8080/portal

Sakai

Following is a link to the Developers' Resources

TWiki

You are using our TWiki and Web server at http://www.grids.ac.uk/twiki/bin/view

Older e-Science Software

See GridTech TWiki web

GridAndHPC Web Utilities

• - advanced search
• WebTopicList - all topics in alphabetical order
• WebChanges - recent topic changes in this web
• WebNotify - subscribe to an e-mail alert sent when topics change
• WebRss, WebAtom - RSS and ATOM news feeds of topic changes
• WebStatistics - listing popular topics and top contributors
• WebPreferences - preferences of this web