WebHome < GridAndHPC

| <img alt="" src="http://www.grids.ac.uk/Images/cclrc_dl_logo.gif" /> | <font color="#003153">This is the Daresbury Laboratory Web Server for the STFC Computational Science and Engineering Group for HPC and Grid Technology. You can find out about the projects we work on and the software we develop. Members of the Group can edit and update these pages.</font> |

---++ HPC Grand Challenges, Software and Performance

   * [[http://tardis.dl.ac.uk/computing_history][History of Computing at Daresbury]]
   * [[http://www.grids.ac.uk/Complex/grand_challenges][Computing Grand Challenges]]
   * [[http://www.grids.ac.uk/Complex/Complex][Computational Research into Complex Systems]] - this is one of the grand challenges, perhaps for us the most important?
   * [[http://www.grids.ac.uk/Complex/ABMS][Software for Agent Based Modelling and Simulation]] - this is a way to tackle complex systems.
      * [[LeedsABMWorkshop][Future Research Directions in Agent Based Modelling]] - a workshop held at University of Leeds 15/6/2010
      * [[http://www.softeng.rl.ac.uk/abm_ccp/][ABM CCP]] development of a proposal for a collaborative computational project
      * [[http://www.agent-based-models.com/blog/2010/03/30/agent-based-modeling/][Jeff Shank's ABM site]] more information on activities and researchers world wide
      * [[http://mass.leeds.ac.uk/2014/02/03/simsoc-recommendations-for-abm-videos/][Videos]] about ABMs and Complexity
   * [[http://www.grids.ac.uk/NWGrid/prediction][Software for Computer Performance Modelling]] - this is also done using discrete event simulation.
   * Research Data
      * [[http://www.grids.ac.uk/NWGrid/LargeData/][Management and Analysis of Large Research Datasets]] - draft report
      * [[http://www.grids.ac.uk/NWGrid/DataIntensive/][Data Intensive Computing]] - draft report

---++ Grid, Cloud and e-Research Software

   * [[http://www.woodheadpublishing.com/en/book.aspx?bookID=1892&ChandosTitle=1][A Book]]
   * [[http://www.grids.ac.uk/NWGrid/Clouds][Some Notes on Clouds and Web 2.0]]
   * [[http://science.energy.gov/~/media/ascr/pdf/program-documents/docs/Magellan_final_report.pdf][Final Report of US Magellan Cloud Project]]

---+++ NW-GRID
The North West Grid, NW-GRID, is a collaboration between Daresbury Laboratory and the Universities of Lancaster, Liverpool and Manchester. With the help of NWDA funding, we established and operate a computational Grid comprising of high-performance computing systems coupled by a high-speed private fibre network between the sites. NW-GRID offers world-class services founded in the deployment and exploitation of Grid middleware technologies enabling the capabilities of the Grid to be realised in leading-edge computational science and engineering applications.

For more information see the [[http://www.nw-grid.ac.uk][NW-GRID Web site]]. You can [[http://rhine.dl.ac.uk:8080/portal][log onto the portal]] if you have an account.

The following notes are mainly to remind me about things I did when installing system software.

   * [[Modules][Environment Modules]]
   * [[http://www.nw-grid.ac.uk/GT4.2.1][Globus GT-4]]
   * Globus GT-5 coming soon. See http://www.globus.org/toolkit/docs/5.0/5.0.0/
   * [[OpenMPI][OpenMPI]] - building and using OpenMPI
   * [[MPICH2][MPICH-2]] - building and using MPICH-2
   * [[SGE6][Sun Grid Engine 6.2]] - some hints when things go wrong
   * [[FirewallIssues][Firewall Issues]] - network traffic
   * Condor, see below under Campus Grid

---+++ Gentoo (and other) Linux

[[http://www.ibm.com/developerworks/linux/library/l-linuxuniversal/index.html?cmp=dw&cpb=dwlin&ct=dwnew&cr=dwnen&ccy=zz&csr=031512][What is Linux?]]

   * [[GentooLinux][Gentoo (and other) Linux Hints and Tips]]
   * [[LaTeX][LaTeX stuff]]
   * [[http://www.linuxfromscratch.org/][Linux from Scratch]] - as recommended by Dave
   * [[AutoTools][Using Autotools]]
   * [[http://www.ibm.com/developerworks/linux/library/l-lpic1-v3-102-2/index.html?ca=dnw-1115&ca=dth-lx][Grub and Lilo]] - boot managers
   * [[http://www.ibm.com/developerworks/aix/library/au-ssh_restrict/index.html?cmp=dw&cpb=dwaix&ct=dwnew&cr=dwnen&ccy=zz&csr=100710][sshd and tcpwrappers]] -  [[http://www.ibm.com/developerworks/aix/library/au-tcpwrapper/index.html?cmp=dw&cpb=dwaix&ct=dwnew&cr=dwnen&ccy=zz&csr=071411][more about TCP Wrappers]]
   * [[http://www.ibm.com/developerworks/linux/library/l-network-filesystems/?cmp=dw&cpb=dwlin&ct=dwnew&cr=dwnen&ccy=zz&csr=111810][NFS etc.]]
   * [[OSSoftware][Open Source Software]] - alternatives to popular packages

---+++ G-R-Toolkit

The G-R-Toolkit combines the best software developed at STFC during its e-Science Programme from 2001-2007. It allows users of many applications in computational research to manage their high-performance computing and data and information management tasks directly from their desktop systems. See here [[GRToolkit][for more information]] and user manual.

---+++ Daresbury Science and Innovation Campus Grid

Here is a [[http://www.grids.ac.uk/NWGrid/dsic_grid/index.html][short introduction]] to our vision for the DSIC Campus Grid. An NGS Roadshow was held on the campus on 23/8/2010 see [[http://www.ngs.ac.uk/cockcroft-institute][NGS Web site]].

[[http://epubs.cclrc.ac.uk/search?st=browse-by-person&pid=1160][Some papers]] mostly relating to the use of Condor.

More information on Daresbury Condor resources is available on the [[http://tardis.dl.ac.uk/Condor/][DL Condor pool Web site]].
There are status pages for the Cockcroft Institute master [[http://tardis.dl.ac.uk/Condor/cgi-bin/CIStatus.cgi][ci-condor.dl.ac.uk]]
and the DL master nodes [[http://tardis.dl.ac.uk/Condor/cgi-bin/CondorStatus.cgi][condor-main.dl.ac.uk]]. A [[http://tardis.dl.ac.uk/Condor/cgi-bin/DSICStatus.cgi][combined listing]] is provided here

Note that these pools are dynamic. From 12/3/2009-21/2/2010 all machines were registering with the CI Pool manager node. We are now investigating flocking to give more flexibility.

Further descriptions including installation notes are provided here.

   * [[DLPool][DL Condor Pool]] - Daresbury Laboratory
   * [[CIPool][CI Condor Pool]] - Cockcroft Institute

We are participating in the Campus Grid SIG.

   * [[http://www.walkingrandomly.com/?p=4411][Manchester Condor Pool]] - some blog notes from Ian Cottam

*Job Submission*

This can be from a user's desktop machine using standard Condor scripts (requires the machine to be a submit node) or using the RMCS Web services client, see [[GRToolkit][G-R-Toolkit documentation]].

*Current Work*

We are investigating how to negociate the firewall between subnets on the DL site and how to flock between the DL and CI pools. See information from !CamGrid http://www.escience.cam.ac.uk/projects/camgrid/firewall.html

We will later investigate back-filling the NW-GRID cluster in a similar way to how it is done in Manchester.

---++ Virtual Research Environments and Portals

Sakai is deployed as a collaborative e-Research tool for a number of projects as follows:

   * UK HPC training: http://congo.dl.ac.uk:8080/portal
   * [[http://www.nw-grid.ac.uk][NW-GRID]]: the North West Grid: http://rhine.dl.ac.uk:8080/portal
   * !NCeSS: National Centre for e-Social Science: http://weaver.dl.ac.uk:8080/portal
   * [[http://www.neiss.org.uk][NeISS]]: National e-Institute for Social Simulation: http://weaver.dl.ac.uk:8080/portal/site/e-iss
   * [[http://www.cse.scitech.ac.uk/][CSED]]: Computational Science and Engineering Department:  http://cselnx8.dl.ac.uk:8080/portal
   * Hartree Centre: http://yukon.dl.ac.uk:8080/portal
   * Psi-k: http://cselnx9.dl.ac.uk:8080/portal

---+++ Sakai

Following is a link to the 
[[http://weaver.dl.ac.uk:8080/access/content/group/a85ffaa1-eb48-4eac-806b-434daf910ea9/][Developers' Resources]]

---+++ TWiki

You are using our TWiki and Web server at http://www.grids.ac.uk/twiki/bin/view

---++ Older e-Science Software

See [[http://tyne.dl.ac.uk/twiki/bin/view/GridTech][GridTech TWiki web]]

---++ <nop>%WEB% Web Utilities

<form action='%SCRIPTURLPATH{"search"}%/%WEB%/'>
   * <input type="text" name="search" size="22" /> <input type="submit" class="twikiSubmit" value="%MAKETEXT{"Search"}%" /> - [[WebSearchAdvanced][%MAKETEXT{"advanced search"}%]]
   * WebTopicList - all topics in alphabetical order
   * WebChanges - recent topic changes in this web
   * WebNotify - subscribe to an e-mail alert sent when topics change
   * WebRss, WebAtom - RSS and ATOM news feeds of topic changes
   * WebStatistics - listing popular topics and top contributors
   * WebPreferences - preferences of this web
</form>
This topic: GridAndHPC > WebHome
Topic revision: r72 - 04 Feb 2014 - RobAllan